Skip to content

fix(setup): don't force abi3 wheel tag on free-threaded builds#341

Open
JSap0914 wants to merge 1 commit into
tree-sitter:masterfrom
JSap0914:fix/free-threaded-wheel-tag
Open

fix(setup): don't force abi3 wheel tag on free-threaded builds#341
JSap0914 wants to merge 1 commit into
tree-sitter:masterfrom
JSap0914:fix/free-threaded-wheel-tag

Conversation

@JSap0914

Copy link
Copy Markdown

Problem

On free-threaded CPython (Py_GIL_DISABLED), tree-sitter-python builds an extension without the limited API — this is already correct, because the Extension sets:

py_limited_api=not get_config_var("Py_GIL_DISABLED")

So on a free-threaded interpreter the produced .so uses the full, version-specific ABI (e.g. _binding.cpython-314t-darwin.so).

However, BdistWheel.get_tag unconditionally rewrote any cp* wheel tag to cp310/abi3:

if python.startswith("cp"):
    python, abi = "cp310", "abi3"

The result is a wheel tagged as stable-ABI (cp310-abi3) whose contents are a non-abi3, free-threaded extension. The wheel tag lies about the ABI, which breaks installation/import on free-threaded interpreters (reported in #337 as ImportError: symbol not found in flat namespace '_tree_sitter_python_external_scanner_create' / an abi3 wheel that is unusable on 3.14t).

Fix

Gate the abi3 rewrite on the same Py_GIL_DISABLED condition already used for py_limited_api, so the wheel tag matches the ABI the extension was actually built against. This mirrors the fix suggested by the reporter (@ngoldbaum) in #337, expressed against the existing get_config_var import so the two ABI decisions stay in sync.

-        if python.startswith("cp"):
+        if python.startswith("cp") and not get_config_var("Py_GIL_DISABLED"):
             python, abi = "cp310", "abi3"
  • GIL builds are unaffected: still tagged cp310-abi3.
  • Free-threaded builds are now tagged to match their real ABI (e.g. cp314-cp314t).

Verification

Built and installed the wheel with cpython-3.14.6+freethreaded and a regular GIL 3.13:

Interpreter Wheel tag before Wheel tag after import tree_sitter_python
3.14t (free-threaded) cp310-abi3 (wrong) cp314-cp314t (correct) OK
3.13 (GIL) cp310-abi3 cp310-abi3 (unchanged) OK

After the fix, on free-threaded 3.14t the wheel WHEEL metadata reports Tag: cp314-cp314t-..., matching the bundled _binding.cpython-314t-darwin.so, and the existing binding test (bindings/python/tests/test_binding.py) passes on both interpreters.

Closes #337

On free-threaded CPython (Py_GIL_DISABLED), the extension is already
built without the limited API (py_limited_api = not Py_GIL_DISABLED),
so the produced .so uses the full, version-specific ABI
(e.g. _binding.cpython-314t-darwin.so). BdistWheel.get_tag, however,
unconditionally rewrote any cp* wheel tag to cp310/abi3, mislabeling the
free-threaded wheel as a stable-ABI (abi3) wheel. This produces a wheel
whose tag does not match its contents, which breaks installation/import
on free-threaded interpreters (tree-sitter#337).

Gate the abi3 rewrite on the same Py_GIL_DISABLED condition already used
for py_limited_api so the wheel tag matches the ABI the extension was
actually built against. GIL builds are unaffected and still tag cp310/abi3.

Closes tree-sitter#337

Signed-off-by: JSup <hanjisang0914@gmail.com>
@ngoldbaum

Copy link
Copy Markdown

Thanks!

Free-threaded 3.15 and newer does support limited API builds, so you might as well account for that right now and make the build skip it only on 3.14 and older.

@ngoldbaum

Copy link
Copy Markdown

Although I don't know offhand if tree-sitter's C extensions actually do build on abi3t, see https://docs.python.org/3.15/howto/abi3t-migration.html#abi3t-migration-howto

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: tree-sitter-python cannot be imported on free-threaded 3.14

2 participants