Skip to content
Open
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,16 @@ embodichain.lab.sim.atomic_actions
Affordance
InteractionPoints
ObjectSemantics
HeldObjectState
MoveObjectTarget
ActionCfg
AtomicAction
MoveActionCfg
MoveAction
PickUpActionCfg
PickUpAction
MoveObjectActionCfg
MoveObjectAction
PlaceActionCfg
PlaceAction
AtomicActionEngine
Expand All @@ -37,6 +41,14 @@ Core
:members:
:show-inheritance:

.. autoclass:: HeldObjectState
:members:
:show-inheritance:

.. autoclass:: MoveObjectTarget
:members:
:show-inheritance:

.. autoclass:: ActionCfg
:members:
:exclude-members: __init__, copy, replace, to_dict, validate
Expand Down Expand Up @@ -66,6 +78,15 @@ Actions
:members:
:show-inheritance:

.. autoclass:: MoveObjectActionCfg
:members:
:exclude-members: __init__, copy, replace, to_dict, validate
:show-inheritance:

.. autoclass:: MoveObjectAction
:members:
:show-inheritance:

.. autoclass:: PlaceActionCfg
:members:
:exclude-members: __init__, copy, replace, to_dict, validate
Expand Down
34 changes: 34 additions & 0 deletions docs/source/overview/sim/atomic_actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,13 @@ AtomicActionEngine ◄─────────────── PlanResult
- `affordance` — *how* to interact with the object (e.g. antipodal grasp poses)
- `entity` — a live reference to the simulation object, so actions can read its current pose

**`HeldObjectState`** is runtime state produced after a successful semantic pickup. It stores
the held object's semantics and object-to-end-effector transform so later actions can move the
object without recomputing the grasp. It is intentionally separate from `ObjectSemantics`,
which remains a reusable object description rather than per-execution robot state.

**`MoveObjectTarget`** describes an object-centric target pose for an already-held object.

**`Affordance`** is a data class that encodes a specific interaction capability. The built-in affordance types are:

| Class | Use case |
Expand All @@ -67,6 +74,7 @@ The following actions are available out of the box:
|---|---|---|---|
| `MoveAction` | `MoveActionCfg` | `Tensor (4,4)` — EEF pose | Move arm to pose |
| `PickUpAction` | `PickUpActionCfg` | `ObjectSemantics` or `Tensor (4,4)` | Approach → close gripper → lift |
| `MoveObjectAction` | `MoveObjectActionCfg` | `MoveObjectTarget` | Move held object and keep gripper closed |
| `PlaceAction` | `PlaceActionCfg` | `Tensor (4,4)` — EEF release pose | Lower → open gripper → retract |

### `MoveAction`
Expand Down Expand Up @@ -101,6 +109,26 @@ Three-phase grasp motion: *approach → close gripper → lift*.

---

### `MoveObjectAction`

Moves a held object to an object-centric target pose while preserving the grasp. It consumes
the `HeldObjectState` produced by a prior semantic `PickUpAction`.

`HeldObjectState` and `MoveObjectTarget` are intentionally kept separate from
`ObjectSemantics`: `ObjectSemantics` describes the object and affordances, while these
types describe runtime held-object state and action-specific targets.

| Config field | Default | Description |
|---|---|---|
| `hand_close_qpos` | `None` | **Required.** Gripper closed joint positions |
| `hand_control_part` | `"hand"` | Robot control part for the gripper |
| `sample_interval` | `50` | Number of waypoints in the trajectory |

**Target:** `MoveObjectTarget` or `dict` with `"object_target_pose"` containing a `torch.Tensor`
of shape `(4, 4)` or `(n_envs, 4, 4)`.

---

### `PlaceAction`

Three-phase release motion: *lower → open gripper → retract*. Mirrors `PickUpAction`.
Expand All @@ -119,6 +147,8 @@ from embodichain.lab.sim.atomic_actions import (
ObjectSemantics,
AntipodalAffordance,
PickUpActionCfg,
MoveObjectActionCfg,
MoveObjectTarget,
PlaceActionCfg,
MoveActionCfg,
)
Expand All @@ -131,6 +161,7 @@ pickup_cfg = PickUpActionCfg(
hand_close_qpos=torch.tensor([0.025, 0.025]),
)
place_cfg = PlaceActionCfg(...)
move_object_cfg = MoveObjectActionCfg(hand_close_qpos=torch.tensor([0.025, 0.025]))
move_cfg = MoveActionCfg(control_part="arm")

# 2. Build the engine — action order matches target_list order
Expand Down Expand Up @@ -228,8 +259,10 @@ is_success, traj = engine.execute_static(target_list=[target_pose])
|---|---|
| `torch.Tensor (4,4)` or `(n_envs,4,4)` | EEF pose, broadcast across envs |
| `ObjectSemantics` | Passed directly to the action |
| `MoveObjectTarget` | Passed directly to `MoveObjectAction` |
| `str` (object label) | Looked up in `SemanticAnalyzer` cache |
| `dict` with `"pose"` key | Unwrapped to tensor |
| `dict` with `"object_target_pose"` key | Wrapped as `MoveObjectTarget` |
| `dict` with `"label"` key | Analyzed via `SemanticAnalyzer` |

---
Expand All @@ -239,3 +272,4 @@ is_success, traj = engine.execute_static(target_list=[target_pose])
- {doc}`planners/motion_generator` — the trajectory planner used by every action
- {doc}`sim_robot` — how control parts and IK solvers are configured

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add docs and unitest for the new atomic action

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added both.
Docs now cover:

  • MoveObjectAction
  • MoveObjectActionCfg
  • MoveObjectTarget
  • HeldObjectState
  • target resolution for MoveObjectTarget / {"object_target_pose": ...}
    Tests now cover:
  • MoveObjectAction target resolution
  • missing held-state failure
  • closed-hand trajectory padding
  • held-state preservation
  • engine held-state propagation/clearing
  • batched pickup success handling

- Tutorial: `scripts/tutorials/sim/atomic_actions.py`
- Move object demo: `scripts/tutorials/atomic_action/move_object_atomic_actions.py`
33 changes: 32 additions & 1 deletion docs/source/tutorial/atomic_actions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ Key Features
- **Semantic-aware execution** — actions accept either a raw pose tensor or an
``ObjectSemantics`` descriptor that bundles affordance data (grasp poses, interaction
points) with the simulation entity.
- **Three built-in primitives** — ``MoveAction``, ``PickUpAction``, and ``PlaceAction``
- **Built-in primitives** — ``MoveAction``, ``PickUpAction``, ``MoveObjectAction``,
and ``PlaceAction``
cover the most common tabletop manipulation workflows out of the box.
See the :ref:`supported_atomic_actions` table for configs and target types.
- **Extensible registry** — custom actions can be registered globally with
Expand Down Expand Up @@ -53,6 +54,7 @@ Setting up the engine
from embodichain.lab.sim.atomic_actions import (
AtomicActionEngine,
PickUpActionCfg,
MoveObjectActionCfg,
PlaceActionCfg,
MoveActionCfg,
)
Expand All @@ -78,6 +80,11 @@ Setting up the engine
hand_control_part="hand",
lift_height=0.15,
)
move_object_cfg = MoveObjectActionCfg(
hand_close_qpos=hand_close,
control_part="arm",
hand_control_part="hand",
)
move_cfg = MoveActionCfg(control_part="arm")

engine = AtomicActionEngine(
Expand Down Expand Up @@ -139,6 +146,30 @@ Executing a pick-place-move sequence
robot.set_qpos(trajectory[:, i])
sim.update(step=4)

Moving a held object
~~~~~~~~~~~~~~~~~~~~

``MoveObjectAction`` consumes the runtime ``HeldObjectState`` produced by a previous
semantic ``PickUpAction``. The target is object-centric, so the caller specifies where the
object should move, and the action converts that pose into an end-effector target while
keeping the gripper closed.

.. code-block:: python

from embodichain.lab.sim.atomic_actions import MoveObjectTarget

engine = AtomicActionEngine(
motion_generator=motion_gen,
actions_cfg_list=[pickup_cfg, move_object_cfg],
)

object_target_pose = torch.eye(4, dtype=torch.float32, device=device)
object_target_pose[:3, 3] = torch.tensor([0.3, -0.2, 0.25], device=device)

is_success, trajectory = engine.execute_static(
target_list=[semantics, MoveObjectTarget(object_target_pose=object_target_pose)]
)

Registering custom actions
~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
9 changes: 8 additions & 1 deletion embodichain/lab/sim/atomic_actions/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,18 @@
AntipodalAffordance,
InteractionPoints,
ObjectSemantics,
HeldObjectState,
MoveObjectTarget,
ActionCfg,
AtomicAction,
)
from .actions import (
MoveAction,
MoveObjectAction,
PickUpAction,
PlaceAction,
MoveActionCfg,
MoveObjectActionCfg,
PickUpActionCfg,
PlaceActionCfg,
)
Expand All @@ -47,16 +51,19 @@
__all__ = [
# Core classes
"Affordance",
"GraspPose",
"InteractionPoints",
"ObjectSemantics",
"HeldObjectState",
"MoveObjectTarget",
"ActionCfg",
"AtomicAction",
# Action implementations
"MoveAction",
"MoveObjectAction",
"PickUpAction",
"PlaceAction",
"MoveActionCfg",
"MoveObjectActionCfg",
"PickUpActionCfg",
"PlaceActionCfg",
# Engine
Expand Down
Loading