Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
d5d6136
Add arguments for 2D pose model overrides in opts class
xiu-cs May 15, 2026
3aea87e
Add configuration file for HRNet-w32 backbone fine-tuned on Animal3D
xiu-cs May 15, 2026
0d7d16f
Add initial configuration file for animal 2D detector and HRNet-w32 f…
xiu-cs May 15, 2026
cfe20da
Enhance SuperAnimalConfig with fine-tuning options and detailed mode …
xiu-cs May 15, 2026
f66e63f
Update vis_animals.sh to include saved_2d_model_path and reset saved_…
xiu-cs May 15, 2026
59c960d
Refactor 2D pose estimation in vis_animals.py to use SuperAnimalEstim…
xiu-cs May 15, 2026
06cbfc1
Refactor resolve_weights_path function in weights.py for improved cla…
xiu-cs May 15, 2026
832c2d4
Update README.md to clarify pre-trained model usage and auto-download…
xiu-cs May 15, 2026
c439d22
Remove unused joint variables from vis_animals.sh
xiu-cs May 15, 2026
674afbe
Update model weights path resolution to include file extension
xiu-cs May 15, 2026
e4ec633
Enhance SuperAnimalEstimator to support fine-tuned model loading and …
xiu-cs May 15, 2026
090dcfc
Update README.md to enhance SuperAnimalEstimator description with fin…
xiu-cs May 15, 2026
95293b4
Update model weights path resolution to include file extension for lo…
xiu-cs May 15, 2026
87dc5f4
Remove unused modules and classes related to 3D pose estimation
xiu-cs May 15, 2026
ed3f084
Update DatasetConfig references from "rat7m" to "animal3d" and adjust…
xiu-cs May 15, 2026
bc4a886
Remove references to Rat7M dataset from Graph class and related docum…
xiu-cs May 15, 2026
f32ca40
Update action placeholder in main_animal3d.py for Animal3D dataset
xiu-cs May 15, 2026
54d1cb6
Update dataset default value and root path in arguments.py for Animal…
xiu-cs May 15, 2026
ff801e5
Add SuperAnimalConfig support and unit tests for fine-tuned mode in S…
xiu-cs May 15, 2026
3279e88
Refactor get_pose2D function to streamline 2D pose estimation using S…
xiu-cs May 15, 2026
4c20e0f
Refactor 2D and 3D pose estimation functions to build estimators once…
xiu-cs May 15, 2026
54ffab2
Add 'mot' to ignore words list in Codespell workflow
xiu-cs May 17, 2026
e77e7f0
Refactor model loading to use device-agnostic code for CUDA compatibi…
xiu-cs May 17, 2026
6335c0b
Refactor HRNetPose2d to use device-agnostic code for model and input …
xiu-cs May 17, 2026
4ebd20c
Update Python version requirement and refine dependency constraints i…
xiu-cs May 17, 2026
e3fef16
Remove unused import of coco_h36m from utilitys.py
xiu-cs May 17, 2026
621b6a6
Remove unnecessary configuration files and associated data
xiu-cs May 17, 2026
34a824f
Update README.md to clarify Python version requirement and add PyTorc…
xiu-cs May 17, 2026
c6b1c4a
Fix checkpoint directory handling and improve weight loading logic in…
xiu-cs May 19, 2026
6131b5e
Refactor model path handling in test_animal3d.sh to clarify usage of …
xiu-cs May 19, 2026
f77f5e8
Update fmpose3d/animals/models/graph_frames.py
xiu-cs May 26, 2026
ba130a8
Refactor build_2d_estimator function to simplify model path handling …
xiu-cs May 26, 2026
d6d26ac
Refactor weight resolution logic in SuperAnimalEstimator to improve s…
xiu-cs May 26, 2026
882ea0b
Fix test_auto_download_finetuned to reuse resolved path and update ca…
xiu-cs May 26, 2026
aa3df33
Refactor SuperAnimalEstimator to streamline fine-tuned model configur…
xiu-cs May 26, 2026
a5ecbc1
Update fmpose3d/animals/configs/__init__.py
xiu-cs May 27, 2026
9048629
Remove unused estimator and postprocessor from FMPose3DInference setup
xiu-cs May 27, 2026
e45fe2c
Add assertions for auto_download_finetuned and pose_snapshot_path in …
xiu-cs May 27, 2026
30ff652
Refactor mock function in SuperAnimalPrediction tests to handle multi…
xiu-cs May 27, 2026
6e7337d
Update docstring in build_3d_lifter to clarify return value as eval m…
xiu-cs May 29, 2026
d4c2061
Enhance 2D and 3D pose estimation by including scores and validity ch…
xiu-cs May 29, 2026
760cdf7
Remove redundant import of resolve_weights_path in build_3d_lifter fu…
xiu-cs May 29, 2026
0db80b0
Fix docstring formatting in weights.py for clarity
xiu-cs May 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/codespell.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ jobs:
- name: Codespell
uses: codespell-project/actions-codespell@v1
with:
ignore_words_list: fmpose, mpjpe, uvd, xyz, hm36, cpn, dbb
ignore_words_list: fmpose, mpjpe, uvd, xyz, hm36, cpn, dbb, mot
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ FMPose3D creates a 3D pose from a single 2D image. It leverages fast Flow Matchi

### Set up an environment

Make sure you have Python 3.10+. You can set this up with:
Make sure you have Python 3.10. The installation and demos are tested with Python 3.10. You can set this up with:
```bash
conda create -n fmpose_3d python=3.10
conda activate fmpose_3d
Expand All @@ -45,6 +45,8 @@ For the animal pipeline, install the optional DeepLabCut dependency:
pip install "fmpose3d[animals]"
```

> **PyTorch/CUDA note.** FMPose3D pins `torch>=2.4.1,<2.5` and `torchvision>=0.19.1,<0.20`, which use CUDA 12.1 wheels by default on Linux. If your driver does not support CUDA 12.1, or if you need a specific CUDA build, install PyTorch first using the matching command from [pytorch.org](https://pytorch.org/get-started/locally/), then install `fmpose3d`.

## Demos

### Testing on in-the-wild images (humans)
Expand Down Expand Up @@ -108,7 +110,7 @@ FMPose3D also ships a high-level Python API for end-to-end 3D pose estimation fr

## Experiments on non-human animals

For animal training/testing and demo scripts, see [animals/README.md](animals/README.md).
For animal training/testing and demo scripts, see [animals/README.md](animals/README.md). The animal demo **auto-downloads both checkpoints** (a 26-joint SuperAnimal-Quadruped fine-tuned on Animal3D for 2D pose, and the FMPose3D animal flow-matching lifter for 3D) from [Hugging Face](https://huggingface.co/MLAdaptiveIntelligence/FMPose3D) on first run — no manual setup needed.

## Citation

Expand Down
8 changes: 5 additions & 3 deletions animals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,10 @@ In this part, the FMPose3D model is trained on [Animal3D](https://xujiacong.gith

This visualization script is designed for single-frame based model, allowing you to easily run 3D animal pose estimation on any single image.

Before testing, make sure you have the pre-trained model ready.
You may either use the model trained by your own or download ours from [here](https://drive.google.com/drive/folders/1kL4aOyWNq0o9zB0rSTRM8KYgkySVmUTk?usp=drive_link) and place it in the `./pre_trained_models` directory.
Both pre-trained checkpoints are **auto-downloaded from [Hugging Face](https://huggingface.co/MLAdaptiveIntelligence/FMPose3D)** on first run and cached under `~/.cache/huggingface/`. No manual downloads required.

- **3D lifter** (`fmpose3d_animals.pth`) — Animal3D 26-joint flow-matching 2D→3D lifter. Override: set `saved_model_path` in `vis_animals.sh` to a local `.pth`.
- **2D pose model** (`sa_finetune_hrnet_w32.pt`) — SuperAnimal-Quadruped HRNet-w32 fine-tuned on Animal3D for the 26-joint Animal3D output layout. Override: set `saved_2d_model_path` in `vis_animals.sh` to a local `.pt`.

Next, put your test images into folder `demo/images`. Then run the visualization script:
```bash
Expand Down Expand Up @@ -49,7 +51,7 @@ Place the downloaded files in the `dataset/` folder of this project:
## Training
The training logs, checkpoints, and related files of each training time will be saved in the './checkpoint' folder.

For trainig on the two datasets:
For training on the two datasets:

```bash
cd animals
Expand Down
Loading
Loading