Skip to content

fix: prevent NOTSET sentinel leaking as test ID in parametrize with callable ids#14548

Closed
rusthype wants to merge 2 commits into
pytest-dev:mainfrom
rusthype:fix/parametrize-notset-id-leak
Closed

fix: prevent NOTSET sentinel leaking as test ID in parametrize with callable ids#14548
rusthype wants to merge 2 commits into
pytest-dev:mainfrom
rusthype:fix/parametrize-notset-id-leak

Conversation

@rusthype
Copy link
Copy Markdown

@rusthype rusthype commented Jun 2, 2026

Summary

Fixes #13235

When @pytest.mark.parametrize is used with an empty argvalues list and a callable ids parameter, two bugs occur:

  1. The internal NOTSET sentinel string leaks into test output as test_foo[NOTSET]
  2. The user's ids callable gets invoked with the NOTSET sentinel, causing an AttributeError or ValueError
@pytest.mark.parametrize("matrix", [], ids=lambda m: m.id)
def test_foo(matrix):
    pass

Before: test_foo[NOTSET] in output (string leaks) or crash

After: test_foo[matrix0] — clean, matches the auto-naming convention

Root Cause

In ParameterSet._for_parametrize(), when argvalues=[], a placeholder ParameterSet is created with id="NOTSET". This string was then yielded verbatim in _resolve_ids() (since id is not None), leaking the internal name. If the user provided a callable ids, it was also called on the NOTSET sentinel value before id="NOTSET" was set (in older versions) or was bypassed (current main), but the string still leaked.

Fix

src/_pytest/mark/structures.py — use id=None for the placeholder ParameterSet so standard ID generation runs:

# Before
ParameterSet(values=(NOTSET,) * len(argnames), marks=[mark], id="NOTSET")

# After
ParameterSet(values=(NOTSET,) * len(argnames), marks=[mark], id=None)

src/_pytest/python.py — guard _idval_from_function to skip calling the user's callable when val is NOTSET:

def _idval_from_function(self, val, argname, idx):
    if self.idfn is None:
        return None
    if val is NOTSET:      # ← new guard
        return None
    ...

This follows the existing pattern in _idval_from_value which already has a special case for NOTSET.

Test

Added TestMetafuncFunctional::test_parametrize_empty_argvalues_callable_ids in testing/python/metafunc.py verifying:

  • No NOTSET appears in output
  • The test is properly skipped (1 skipped)

Checklist

…allable ids

When @pytest.mark.parametrize is used with an empty argvalues list and a
callable ids parameter, pytest internally creates a placeholder ParameterSet
with NOTSET sentinel values. Previously this placeholder used id='NOTSET',
which caused the internal string to leak into the test output as 'test_foo[NOTSET]'.
Additionally, if an ids callable was provided, it would be invoked with the
NOTSET sentinel and crash with an AttributeError or similar.

Fix:
- Use id=None for the placeholder ParameterSet so the standard ID generation
  runs (produces e.g. 'test_foo[x0]' matching the existing workaround behavior)
- Guard _idval_from_function to return None early when val is NOTSET,
  preventing the user's callable from being invoked with the sentinel

Fixes pytest-dev#13235
@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Jun 2, 2026
@bluetech
Copy link
Copy Markdown
Member

bluetech commented Jun 2, 2026

The NOTSET in this case is intentional.

@bluetech bluetech closed this Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pytest.mark.parametrize ids with empty parameter

2 participants