Allow non-MLLM calibration datasets when not quantizing vision#2065
Open
aquilarubra wants to merge 2 commits into
Open
Allow non-MLLM calibration datasets when not quantizing vision#2065aquilarubra wants to merge 2 commits into
aquilarubra wants to merge 2 commits into
Conversation
aquilarubra
force-pushed
the
pr/mllm-dataset-fix
branch
from
July 19, 2026 14:20
79d663a to
589feba
Compare
…vision MLLMCalibrator unconditionally routed to get_mllm_dataloader whenever the model was multimodal, which KeyErrors on any dataset not in its MLLM_DATASET registry (its own "os.path.isfile(dataset) or dataset in MLLM_DATASET" guard still indexes MLLM_DATASET with the raw file path). But a local text dataset (file/dir/HF text set) is perfectly valid calibration for a VLM whose non-text modules are NOT being quantized (quant_nontext_module=False): the full-model forward runs text-only, and the loop's generic-dict branch already just feeds model(**batch) regardless of dataset shape. When quant_nontext_module is False and the dataset isn't a known MLLM_DATASET entry, use the standard text get_dataloader() instead of falling through to get_mllm_dataloader. Signed-off-by: Fabrizio del Tin <devotedmystic@gmail.com>
aquilarubra
force-pushed
the
pr/mllm-dataset-fix
branch
from
July 19, 2026 14:31
589feba to
123b0e7
Compare
Author
|
/azp run Performance-Test-AutoRound |
|
Commenter does not have sufficient privileges for PR 2065 in repo intel/auto-round |
The prior /azp run Performance-Test-AutoRound comment failed with 'Commenter does not have sufficient privileges' (external contributors can't trigger Azure Pipelines re-runs by comment on this repo). Pushing an empty commit instead, since a new commit re-runs CI regardless of commenter privileges. See PR discussion: the only failing check (Qwen3_W4A16 peak VRAM, +4.11%% vs baseline) is on a non-multimodal model that never exercises this PR's is_mllm=True-gated change in calibration/mllm.py. Signed-off-by: Fabrizio del Tin <devotedmystic@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Standalone fix, no dependency on the other PRs in this stack.
MLLMCalibratorunconditionally routes toget_mllm_dataloaderwhenever the model is multimodal, whichKeyErrors on any dataset not in itsMLLM_DATASETregistry (its ownos.path.isfile(dataset) or dataset in MLLM_DATASETguard still indexesMLLM_DATASETwith the raw file path). But a local text dataset (file/dir/HF text set) is perfectly valid calibration for a VLM whose non-text modules are not being quantized (quant_nontext_module=False): the full-model forward runs text-only, and the loop's generic-dict branch already just feedsmodel(**batch)regardless of dataset shape.Change
When
quant_nontext_moduleisFalseand the dataset isn't a knownMLLM_DATASETentry, use the standard textget_dataloader()instead of falling through toget_mllm_dataloader.