Single gpu disk offload PTQ for DSR1/Ultra - #2008

Open

Fridah-nv wants to merge 27 commits into

mainfrom

fridah/single-gpu-disk-offload-ptq

Commits on Aug 4, 2026

feat(ptq): single-GPU disk-offload layerwise PTQ (G1/G2)
Fridah-nv
committed
abd5b6c
fix(export): materialize non-decoder disk-offloaded tensors before save
Fridah-nv
committed
c3b9dc6
fix(export): correct offloaded export path — MoE ordering, lm_head handler, memory
Fridah-nv
committed
8ffe1d6
feat(export): streaming shard writer for 80 GB CPU RAM target

Fridah-nv
and
claude
committed
b04ced8
refactor(export): simplify streaming-export additions per /simplify review

Fridah-nv
and
claude
committed
dfc3bd8
fix(export): harden streaming export path per code review

Fridah-nv
and
claude
committed
e833829
fix(export): avoid save_pretrained shared-tensor crash on MoE models

Fridah-nv
and
claude
committed
99afbf9
refactor(export): remove dead offloaded branch, early-return dispatch, shard regression test

Fridah-nv
and
claude
committed
cec8335
chore: document new offload recipe and fix pre-commit failures

Fridah-nv
and
claude
committed
b69d2f6
fix(export): guard streaming shards against aliases; make compat shims opt-in

Fridah-nv
and
claude
committed
112a2ed
fix(ptq): scope the DeepSeek load-path change and respect --trust_remote_code

Fridah-nv
and
claude
committed
eb254a6
revert(ptq): drop transformers compat shims in favor of a correct environment

Fridah-nv
and
claude
committed
49f6a5e
fix(export): scope data_ptr tie identity to resident tensors

Fridah-nv
and
claude
committed
a91c87f
fix(export): stream extra_state_dict tensors in the offload path

Fridah-nv
and
claude
committed
bd28239
fix(export): remove _tied_cache from fused-expert export to prevent cross-layer aliasing

Fridah-nv
and
claude
committed
0f9ab7a
fix(export): reconcile tied-cache reset with FSDP2 dedup opt-out

Fridah-nv
and
claude
committed
dcbddcf
refactor(export): decide materialization via the shared dispatcher

Fridah-nv
and
claude
committed
90d0671
fix(export): skip sharded DTensors in sync_tied_input_amax, not just meta

Fridah-nv
and
claude
committed
c4f0883
fix(export): disable tie dedup whenever weights are not resident

Fridah-nv
and
claude
committed
f09965d
refactor(export): consolidate offload detection into one helper

Fridah-nv
and
claude
committed
7900bb5
test(export): pin the dedup opt-outs, and correct the DTensor rationale

Fridah-nv
and
claude
committed
e7f61cb
fix(export): address open review findings on the offload path

Fridah-nv
and
claude
committed
098465d
refactor(export): narrow the offload changes to the streaming path

Fridah-nv
and
claude
committed
4e4ca2b
refactor(export): share model-level export prep between both exporters

Fridah-nv
and
claude
committed
5cd9dd5
test(export): consolidate overlapping offload export tests

Fridah-nv
and
claude
committed
1eea332
refactor(export): move the streaming offload exporter to its own module

Fridah-nv
and
claude
committed
a8afef2
docs(recipes): correct the general/ptq recipe count after rebase

Fridah-nv
and
claude
committed
8bc3030

Navigation Menu

Uh oh!

Single gpu disk offload PTQ for DSR1/Ultra - #2008

Single gpu disk offload PTQ for DSR1/Ultra#2008
Fridah-nv wants to merge 27 commits into
mainfrom
fridah/single-gpu-disk-offload-ptq

Commits on Aug 4, 2026

feat(ptq): single-GPU disk-offload layerwise PTQ (G1/G2)

fix(export): materialize non-decoder disk-offloaded tensors before save

fix(export): correct offloaded export path — MoE ordering, lm_head handler, memory

feat(export): streaming shard writer for 80 GB CPU RAM target

refactor(export): simplify streaming-export additions per /simplify review

fix(export): harden streaming export path per code review

fix(export): avoid save_pretrained shared-tensor crash on MoE models

refactor(export): remove dead offloaded branch, early-return dispatch, shard regression test

chore: document new offload recipe and fix pre-commit failures

fix(export): guard streaming shards against aliases; make compat shims opt-in

fix(ptq): scope the DeepSeek load-path change and respect --trust_remote_code

revert(ptq): drop transformers compat shims in favor of a correct environment

fix(export): scope data_ptr tie identity to resident tensors

fix(export): stream extra_state_dict tensors in the offload path

fix(export): remove _tied_cache from fused-expert export to prevent cross-layer aliasing

fix(export): reconcile tied-cache reset with FSDP2 dedup opt-out

refactor(export): decide materialization via the shared dispatcher

fix(export): skip sharded DTensors in sync_tied_input_amax, not just meta

fix(export): disable tie dedup whenever weights are not resident

refactor(export): consolidate offload detection into one helper

test(export): pin the dedup opt-outs, and correct the DTensor rationale

fix(export): address open review findings on the offload path

refactor(export): narrow the offload changes to the streaming path

refactor(export): share model-level export prep between both exporters

test(export): consolidate overlapping offload export tests

refactor(export): move the streaming offload exporter to its own module

docs(recipes): correct the general/ptq recipe count after rebase