Skip to content

gh-150505: fix data race in Unpickler_set_memo under free-threading#150550

Open
AhmedMagdy9876 wants to merge 1 commit into
python:mainfrom
AhmedMagdy9876:gh-150505-fix-unpickler-memo-race
Open

gh-150505: fix data race in Unpickler_set_memo under free-threading#150550
AhmedMagdy9876 wants to merge 1 commit into
python:mainfrom
AhmedMagdy9876:gh-150505-fix-unpickler-memo-race

Conversation

@AhmedMagdy9876
Copy link
Copy Markdown

@AhmedMagdy9876 AhmedMagdy9876 commented May 28, 2026

Problem

Unpickler_set_memo modified self->memo and self->memo_size without any critical section. Under free-threading (--disable-gil), two threads assigning to .memo on the same Unpickler instance concurrently could race: one thread could set self->memo = NULL while another was still indexing into it via _Unpickler_MemoPut, causing a NULL-pointer dereference and segfault.

TSAN confirmed the race:

WARNING: ThreadSanitizer: data race
  Read of size 8 by thread T2:
    #0 _Unpickler_MemoPut Modules/_pickle.c:1631
  Previous write of size 8 by thread T1:
    #0 Unpickler_set_memo Modules/_pickle.c:7816

Fix

  • Wrap the memo population loop (all _Unpickler_MemoPut calls) in a single Py_BEGIN_CRITICAL_SECTION / Py_END_CRITICAL_SECTION on self, so the entire assignment is atomic with respect to other threads
  • Swap self->memo and self->memo_size under the same critical section
  • Free the old memo array and decref its contents outside the lock, since Py_XDECREF can trigger arbitrary Python code and should not be called while holding the lock

Testing

Verified with a TSAN build (--disable-gil + -fsanitize=thread) using the reproducer from the issue. TSAN no longer reports a data race after this fix. Added a multithreaded test to Lib/test/test_pickle.py to catch regressions.

Fixes #150505

…ding

Unpickler_set_memo modified self->memo and self->memo_size without any
critical section, allowing concurrent threads to race on the same
Unpickler instance. Wrap the memo population loop and the pointer swap
in Py_BEGIN_CRITICAL_SECTION/Py_END_CRITICAL_SECTION to serialize
access.
@AhmedMagdy9876 AhmedMagdy9876 force-pushed the gh-150505-fix-unpickler-memo-race branch from be6a4a2 to 194ca8e Compare May 28, 2026 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Segfault in _Unpickler_MemoPut (_pickle.c) from Unpickler.memo assignment with free-threading

1 participant