lsnu/openpi-pi05-vladaptor-oven-keypose-full-state

OpenPI pi0.5 VLAdaptor/PerAct2 Oven Keypose Full Fine-Tune State

Private archive for the corrected keypose training run on the PerAct2/VLAdaptorBench take tray out of oven task.

This repo is intended to make the work resumable on another machine. It contains the full OpenPI Orbax checkpoint directories for the full pi0.5 fine-tune, including both inference parameters and optimizer/training state.

What This Run Is

• OpenPI config: pi05_vladaptor_oven_keypose
• Experiment: vladaptor_oven_pi05_keypose_full_b32_20k
• Base init: pi0.5_base, full fine-tune, not LoRA
• Model config: Pi0Config(pi05=True, action_dim=32, action_horizon=1)
• Dataset: lsnu/vladaptor_take_tray_out_of_oven_keypose_train
• Target: next PerAct/VLAdaptor keypose, not dense next-frame EE deltas
• Action representation: 16D bimanual absolute EE keypose with quaternions
• Deltas: disabled
• EMA: disabled
• Batch size: 32
• Eval episodes: 80-99 (10 test + 10 heldout)

Checkpoints

The checkpoint upload is rooted at:

5000/
10000/
15000/

Each step directory contains:

params/       # inference parameters
train_state/  # full resumable training state, including optimizer state
assets/       # norm stats / assets used by the policy

There is no valid 20000/ checkpoint. The continuation past 15k was stopped around step 16.2k after offline eval regression and sim evidence that 10k was the better checkpoint.

Results Summary

Full sim evals on episodes 80-99:

Checkpoint	Eval	Successes	Errors	Notes
5k	original	2/20	0	first milestone
10k	original	4/20	0	best original milestone
10k	confirmation	6/20	0	best confirmed checkpoint
15k	original	2/20	0	lower than 10k
15k	confirmation	4/20	1	episode 90 hung twice; counted as failed/error

Current recommendation: use 10000/ as the best checkpoint. Do not treat the stopped 20k continuation as a better model.

Uploaded Layout

5000/, 10000/, 15000/
  Full Orbax checkpoint directories uploaded from:
  /workspace/workspace/checkpoints/openpi_vladaptor_keypose/pi05_vladaptor_oven_keypose/vladaptor_oven_pi05_keypose_full_b32_20k

code/
  openpi_source_snapshot.tar.gz
  sim_runtime_eval_source_snapshot.tar.gz
  openpi_tracked_changes.diff
  openpi_git_status.txt
  openpi_untracked_files.txt
  sim_runtime_repos_git_status.txt
  *_changes.diff

logs/
  openpi_vladaptor_keypose/
  legacy_dense_openpi_vladaptor/

assets/
  openpi_vladaptor_keypose/

data/
  lerobot_keypose_home/
  vladaptorbench_lowdim/
  runtime_assets/

manifests/
  source_sizes.txt
  full_checkpoint_files.tsv
  staged_metadata_files.tsv

evals/
  step10000_global_camera_episodes80-99/
  step15000_global_camera_episodes80-99/

replication/
  final_audit_20260419/

Post-Hoc Video And Replication Audit Artifacts

After the initial checkpoint archive, two global-camera video suites were added:

• evals/step10000_global_camera_episodes80-99/: all 20 per-episode global-camera videos, 20-up montage, eval JSONL/summary, and video integrity reports for checkpoint 10k.
• evals/step15000_global_camera_episodes80-99/: all 20 per-episode global-camera videos, 20-up montage, eval JSONL/summary, and video integrity reports for checkpoint 15k.

The final replication audit is in:

replication/final_audit_20260419/

That audit bundle includes current source diffs/copies, simulator eval source, environment captures, local-vs-remote inventory checks, post-video run logs, and SHA-256 manifests for the audit bundle and all 5k/10k/15k checkpoint files.

Resume Notes

After downloading the repo, restore the code snapshot, install the OpenPI environment, and point --checkpoint-base-dir to the parent of the restored checkpoint tree.

The original persistent checkpoint path was:

/workspace/workspace/checkpoints/openpi_vladaptor_keypose/pi05_vladaptor_oven_keypose/vladaptor_oven_pi05_keypose_full_b32_20k

To resume from 10k or 15k, keep this directory structure:

checkpoints/openpi_vladaptor_keypose/
  pi05_vladaptor_oven_keypose/
    vladaptor_oven_pi05_keypose_full_b32_20k/
      5000/
      10000/
      15000/

The key training/eval entrypoints are in the OpenPI source snapshot:

scripts/run_vladaptor_keypose_training_with_sim_eval.py
scripts/watch_and_launch_vladaptor_keypose.sh
scripts/validate_vladaptor_keypose_pipeline.py
scripts/verify_vladaptor_oven_keypose_dataset.py
scripts/convert_vladaptor_oven_keyposes_to_lerobot.py
src/openpi/policies/rlbench_ee_policy.py
src/openpi/training/config.py

The corrected config and validation reports verify:

• action_horizon=1, so the target is a single future keypose.
• The OpenPI target action matches the next discovered keypose.
• Dense next-frame targets are not used as the supervised label.
• Quaternion train/eval representation is consistent.
• Eval converts OpenPI left/right action order to RLBench right-first order.

OpenPI Subtask Full Fine-Tune Run Notes

The machine workspace archive uploaded on 2026-04-22 is under:

workspace_archives/machine_workspace_archive_20260422_fast_sharded/

The OpenPI subtask training code and run configuration are in the archived workspace, not in the root of this model repo. Restore the archive so the paths below resolve under /workspace/workspace.

Relevant files and directories for launching the openpi_subtask keypose run:

Purpose	Archive path	Notes
Code repo	workspace/repos/openpi_subtask/	Fork of Ke-Wang1017/openpi_subtask used for subtask-conditioned training.
Training configs	workspace/repos/openpi_subtask/src/openpi/training/config.py	Use pi05_vladaptor_oven_keypose_subtask_base_15k as the primary full fine-tune config.
Standard trainer	workspace/repos/openpi_subtask/scripts/train.py	Tyro/OpenPI trainer entrypoint.
Trainer with validation hooks	workspace/repos/openpi_subtask/scripts/train_with_eval.py	Adds eval loss, action sample MSE, text legibility checks, optional checkpoint mirroring, and optional HF upload. This is the preferred entrypoint for monitoring subtask legibility.
Subtask dataset summary	workspace/artifacts/base_subtask_15k/base_subtask_keypose_dataset_summary.json	Documents the PerAct2 keypose split and ordinal-to-subtask mapping.
Norm stats asset	workspace/assets/openpi_vladaptor_oven_keypose_subtask_base/pi05_vladaptor_oven_keypose_subtask_base_15k/lsnu/vladaptor_take_tray_out_of_oven_keypose_subtask_train/norm_stats.json	Normalization statistics used by the subtask keypose configs.
Train dataset repo	lsnu/vladaptor_take_tray_out_of_oven_keypose_subtask_train	80 train episodes, 2,622 keypose rows.
Test dataset repo	lsnu/vladaptor_take_tray_out_of_oven_keypose_subtask_test	10 test episodes, 341 keypose rows.
Heldout dataset repo	lsnu/vladaptor_take_tray_out_of_oven_keypose_subtask_heldout	10 heldout episodes, 331 keypose rows.
pi0.5 base weights	/dev/shm/openpi_gsutil/openpi-assets/checkpoints/pi05_base/params	Expected by the full fine-tune config. Download/restore before launch if absent.
Full fine-tune output root	workspace/checkpoints/openpi_vladaptor_oven_keypose_subtask_base_15k/ or an override path	Output location for the single-model full fine-tune. Override --checkpoint-base-dir if you want a separate 10k run directory.
Text/eval logs	workspace/logs/openpi_vladaptor_oven_keypose_subtask_base/	Contains smoke logs and text-eval metrics from this machine. New full-run metrics should be written here or to a new clearly named path.

The base subtask dataset maps keypose ordinals to these low-level prompts:

0: Move the right arm to the oven handle
1-2: Grasp the oven handle with the right gripper
3: Pull the oven door open with the right arm
4: Release the oven handle with the right gripper
5: Move the right arm away from the oven door
6-7: Move the left arm to the tray and grasp it
8: Pull the tray out of the oven with the left arm
9: Lift the tray out of the oven with the left arm
10-11: Move the tray to the finish position and release it
>=12 fallback: Move the left arm away from the tray

Recommended high-memory launch for a single full fine-tune:

cd /workspace/workspace/repos/openpi_subtask

# This is the intended run: full fine-tuning from pi0.5_base, with no frozen
# reasoner/action split. Use 10000 steps for a 10k reproduction, or omit
# --num-train-steps to use the config default of 15000.
XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 uv run scripts/train_with_eval.py \
  --config-name pi05_vladaptor_oven_keypose_subtask_base_15k \
  --exp-name vladaptor_oven_pi05_keypose_subtask_full_b32_10k \
  --batch-size 32 \
  --num-workers 8 \
  --num-train-steps 10000 \
  --checkpoint-base-dir /workspace/workspace/checkpoints/openpi_vladaptor_oven_keypose_subtask_full_10k \
  --eval-repo-id lsnu/vladaptor_take_tray_out_of_oven_keypose_subtask_heldout \
  --eval-interval 1000 \
  --sample-eval-interval 1000 \
  --sample-steps 8 \
  --text-eval-interval 1000 \
  --text-eval-max-samples 12 \
  --metrics-path /workspace/workspace/logs/openpi_vladaptor_oven_keypose_subtask_base/full_b32_10k_metrics.jsonl \
  --text-eval-output-path /workspace/workspace/logs/openpi_vladaptor_oven_keypose_subtask_base/full_b32_10k_text_eval.jsonl \
  --hf-cli-path /workspace/.hf-cli/venv/bin/hf

The relevant config details are:

• pi05_vladaptor_oven_keypose_subtask_base_15k is the intended full fine-tune config from pi0.5 base.
• It trains the subtask text head and action policy together: subtask_loss_weight=0.5, flow_matching_loss_weight=1.0, and no freeze filter.
• It uses bfloat16, action_dim=32, action_horizon=1, and max_token_len=128.
• The archived config default is 15,000 steps and batch size 1; for a high-memory machine, override batch size upward and set --num-train-steps 10000 for a 10k run.
• During training, inspect the *_text_eval.jsonl output at every validation step. Subtasks should be legible and semantically close to the canonical prompts above.

Machine Workspace Archive Shards

Each shard is a tar.zst archive split into up to 8 GiB parts. The table reports apparent uncompressed size from shard_manifest.json. Exact per-shard file lists are uploaded in workspace_archives/machine_workspace_archive_20260422_fast_sharded/shard_lists/.

Shard	Apparent GB	Entries	Primary contents	Exact list
shard_00	32.41	1	workspace/checkpoints/openpi_vladaptor_keypose/pi05_vladaptor_oven_keypose/vladaptor_oven_pi05_keypose_full_b32_20k/5000 (32.4 GB)	shard_lists/shard_00.list
shard_01	32.39	1	workspace/checkpoints/openpi_vladaptor_keypose/pi05_vladaptor_oven_keypose/vladaptor_oven_pi05_keypose_full_b32_20k/15000 (32.4 GB)	shard_lists/shard_01.list
shard_02	32.38	1	workspace/checkpoints/openpi_vladaptor_keypose/pi05_vladaptor_oven_keypose/vladaptor_oven_pi05_keypose_full_b32_20k/10000 (32.4 GB)	shard_lists/shard_02.list
shard_03	19.99	35	workspace/checkpoints/openpi_vladaptor_keypose/pi05_vladaptor_oven_keypose/vladaptor_oven_pi05_keypose_lora_smoke_fast_b32 (19.2 GB); keypose/open-more/regrasp LeRobot homes and smaller artifacts	shard_lists/shard_03.list
shard_04	20.00	35	workspace/hf_staging (19.1 GB); workspace/logs; open-more LeRobot homes and smaller artifacts	shard_lists/shard_04.list
shard_05	19.99	35	Restored Open More finetune audit bundle (18.8 GB); HF CLI env; open-more LeRobot homes and smaller artifacts	shard_lists/shard_05.list
shard_06	23.39	2	workspace/checkpoints/openpi_open_more_recovery (18.8 GB); workspace/sim_runtime (4.6 GB)	shard_lists/shard_06.list
shard_07	28.21	2	workspace/checkpoints/openpi_regrasp_retry_v16_postrelease_contact_best_stage2_recovered (18.8 GB); retry V12 checkpoint step 1199 (9.4 GB)	shard_lists/shard_07.list
shard_08	28.21	2	workspace/checkpoints/openpi_regrasp_retry_v16_postrelease_contact_best_stage2 (18.8 GB); retry V12 checkpoint step 900 (9.4 GB)	shard_lists/shard_08.list
shard_09	25.13	2	workspace/.cache (15.7 GB); strong recovery V1 checkpoint step 1000 (9.4 GB)	shard_lists/shard_09.list
shard_10	24.60	2	workspace/repos (15.2 GB), including openpi_subtask; recovery-only V2 checkpoint step 400 (9.4 GB)	shard_lists/shard_10.list
shard_11	23.80	2	workspace/tmp (14.4 GB); retry V12 checkpoint step 600 (9.4 GB)	shard_lists/shard_11.list
shard_12	21.88	2	workspace/checkpoints/openpi_vladaptor_from_hf (12.5 GB); strong recovery V1 checkpoint step 800 (9.4 GB)	shard_lists/shard_12.list
shard_13	20.94	2	workspace/data (11.5 GB); strong recovery V1 checkpoint step 1200 (9.4 GB)	shard_lists/shard_13.list
shard_14	19.99	33	workspace/checkpoints/openpi_vladaptor (10.3 GB); recovery-only V2 checkpoint step 800 (9.4 GB); smaller open-more data	shard_lists/shard_14.list
shard_15	19.99	36	LoRA smoke checkpoint (9.6 GB); strong recovery V1 checkpoint step 600 (9.4 GB); pip cache and smaller data	shard_lists/shard_15.list
shard_16	19.99	37	Restored generated-weighted finetune run (9.4 GB); recovery-only V2 checkpoint step 1200 (9.4 GB); uv cache and smaller data	shard_lists/shard_16.list
shard_17	19.99	36	Restored Open More 2k finetune run (9.4 GB); recovery-only V2 checkpoint step 1600 (9.4 GB); HF CLI venv and smaller data	shard_lists/shard_17.list
shard_18	19.99	34	Actual failed recovery smoke checkpoint (9.4 GB); strong recovery V1 checkpoint step 400 (9.4 GB); .hf cache and smaller artifacts	shard_lists/shard_18.list
shard_19	19.99	34	Open Action V1 checkpoint (9.4 GB); retry V12 checkpoint step 300 (9.4 GB); gsutil env and smaller data	shard_lists/shard_19.list
shard_20	19.99	13	Regrasp retry V13 checkpoints (9.4 GB); preserved open-more multiphase V5 checkpoint (9.4 GB); .hf_cache	shard_lists/shard_20.list
shard_21	20.62	3	Regrasp retry V15 actual-contact checkpoint (9.4 GB); regrasp retry V16 best checkpoint (9.4 GB); workspace/outputs	shard_lists/shard_21.list
shard_22	19.99	35	Regrasp retry V15 long checkpoint (9.4 GB); selected open-more checkpoint (9.4 GB); plot env and smaller retry data	shard_lists/shard_22.list
shard_23	20.03	3	Strong recovery V1 checkpoint step 200 (9.4 GB); regrasp retry V14 balanced checkpoint (9.4 GB); workspace/hf_upload	shard_lists/shard_23.list

Important Caveats

• This repo is a private work archive, not a polished public release.
• The simulator stack still needs local CoppeliaSim/RLBench/PyRep setup on a new machine.
• Raw and converted data artifacts are included for convenience, but the primary training dataset is also referenced by the OpenPI config as lsnu/vladaptor_take_tray_out_of_oven_keypose_train.
• The old dense-delta run is not included as a model checkpoint because it was the wrong objective. Its small logs are included under logs/legacy_dense_openpi_vladaptor/ for forensic context.