add-custom-act-policy

[wilsonchenghy]

No description provided.

[Copilot]

Pull request overview

Adds support for running a locally trained “custom ACT” policy checkpoint in the SO101 vial-task Isaac Lab eval script, and introduces a new humanoid_act Python package (training, dataset utilities, checkpointing, and a vendored ACT/DETR backbone) under autonomy/il.

Changes:

• Extend lerobot_eval.py with a custom_act policy type that loads a local .pt checkpoint and runs inference in sim.
• Add humanoid_act package with training (humanoid-act-train), smoke test (humanoid-act-smoke), checkpoint I/O, normalization, and sim stepping integration.
• Update autonomy/il packaging to ship humanoid_act and expose new CLI entry points.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
autonomy/simulation/so101_vial_task/scripts/lerobot_eval.py	Adds custom_act option and wires sim eval to a local ACT policy.
autonomy/il/pyproject.toml	Exposes new humanoid-act-* scripts and packages humanoid_act.
autonomy/il/humanoid_act/train.py	Implements a basic training loop for ACT on LeRobot datasets.
autonomy/il/humanoid_act/smoke.py	Adds a quick dataloader smoke-test entry point.
autonomy/il/humanoid_act/policy.py	Wraps ACT model + loss (L1 + KL) and image normalization.
autonomy/il/humanoid_act/normalize.py	Loads/saves normalization stats and reads them from LeRobot meta.
autonomy/il/humanoid_act/eval.py	Implements LocalCustomACTPolicy used by lerobot_eval.py.
autonomy/il/humanoid_act/detr/util/misc.py	Vendored DETR/torchvision utilities (distributed helpers, etc.).
autonomy/il/humanoid_act/detr/util/init.py	Initializes vendored util package.
autonomy/il/humanoid_act/detr/models/transformer.py	Vendored transformer implementation.
autonomy/il/humanoid_act/detr/models/position_encoding.py	Vendored positional encoding implementation.
autonomy/il/humanoid_act/detr/models/detr_vae.py	Implements DETR-VAE model used by ACT.
autonomy/il/humanoid_act/detr/models/backbone.py	Vendored backbone wrapper around torchvision ResNet.
autonomy/il/humanoid_act/detr/models/init.py	Exposes model builders for the vendored code.
autonomy/il/humanoid_act/detr/build.py	Builds ACT model + optimizer without argparse side effects.
autonomy/il/humanoid_act/detr/init.py	Top-level entry for the vendored DETR/ACT module.
autonomy/il/humanoid_act/dataset.py	Converts LeRobot datasets into ACT training batches.
autonomy/il/humanoid_act/config.py	Defines ACTConfig and derives it from dataset metadata.
autonomy/il/humanoid_act/checkpoint.py	Save/load checkpoint payloads, config, and stats.
autonomy/il/humanoid_act/init.py	Re-exports humanoid_act public API symbols.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.