Fix cp3.14 numpy-consumer build failures on iOS/Android#70
Merged
Conversation
Add --python-preference only-managed to the uv venv that builds the
forge environment. On the macos-26 CI runner uv otherwise matches an
exact --python 3.14.5 to Homebrew's framework CPython
(/opt/homebrew/.../Python.framework). crossenv mis-detects
sys.prefix/exec_prefix with a *framework* build ("Unexpected value in
sys.prefix, expected .../cross, got .../build"), which mis-routes
target-arch host requirements (numpy) into the *build* venv instead of
the *cross* venv. The build-platform numpy then never gets installed,
so the build backend's in-process introspection
(import numpy; numpy.get_include(), run by meson/setuptools through the
cross-python) resolves to a non-importable target wheel and dies with
"No module named numpy._core._multiarray_umath".
python-build-standalone (uv-managed) interpreters are relocatable and
do not trip this. Linux already gets a managed interpreter; this makes
macOS consistent. Requires uv new enough to ship managed 3.14.5
(>= 0.11.20; CI already uses it).
Root-caused by reproducing locally: a Homebrew framework build python
reproduces the exact CI failure; a managed PBS python builds clean.
Fixes the cp3.14 iOS numpy-consumer cluster (blis, shapely, rasterio,
matplotlib, pandas, contourpy, ...).
meson runs the cross-python from the unpacked pandas *source* dir to introspect numpy (import numpy; numpy.get_include()). Python puts that cwd on sys.path[0], and pandas ships a top-level pandas/io/ package that shadows the stdlib io module mid-import, so numpy's C-extension init dies with "cannot import name 'TextIOWrapper' from 'io'". Set PYTHONSAFEPATH=1 in this recipe's script_env: it drops the implicit cwd entry (Python 3.11+) without touching PYTHONPATH, so crossenv's import bridge is unaffected. Scoped to pandas on purpose -- a global setting also drops the *script dir* for `python codegen.py` invocations and breaks recipes whose build generators import sibling modules (numpy's genapi, opencv's hdr_parser). Verified: pandas iOS builds clean with this; numpy still builds without it.
… + iOS)
On cp3.14 the cross-venv prefix ({prefix} -> .../cross) no longer
carries libpython{X.Y} or the python headers; they live in the support
tree (android: install/.../python-X.Y.Z; iOS: the Python.xcframework
slice). The CMAKE_ARGS pointed -DPYTHON3_LIBRARIES /
-DPYTHON3_INCLUDE_PATH at {prefix}, so on 3.14:
* android: ninja aborted linking cv2 with
"lib/libpython3.14.so ... missing and no known rule to make it"
* iOS: the cv2 compile aborted with "'Python.h' file not found"
Point both at {HOST_PYTHON_HOME} (android .so, iOS .dylib) -- the same
fix already proven on the coolprop recipe. {HOST_PYTHON_HOME} resolves
to the on-disk python install for every python-build version, unlike
{prefix} which relocated on 3.14.
The explanatory note is kept out of the `>-` CMAKE_ARGS block: inside a
YAML folded scalar a leading `#` is literal text, not a comment, so it
would be passed to cmake verbatim -- and a `{X.Y}` inside it broke
forge's str.format templating with KeyError: 'X'.
Comment out the build-wheels concurrency group so overlapping verification dispatches (and the push events that carry the fix commits) do not cancel each other mid-run. Revert before merge.
…tness check The recipe test asserted `duration < 0.7` for a 500x500 matmul as a proxy for "OpenBLAS is linked" (with OpenBLAS <=0.4s, without >=1.0s on the original test devices). But mobile-forge's numpy is built WITHOUT OpenBLAS (its config reports blas name="none"), so the matmul runs on the unaccelerated fallback and its wall-clock time swings wildly on loaded / emulated CI simulators -- it flaked at 1.43s (3.14) and 2.18s (3.13) for the same wheel, randomly redding the iOS mobile-test lane. Rename to test_matmul and assert the *result* (a @ I == a) plus shape instead of the elapsed time; keep the timing as an informational print. This still exercises the dense GEMM path (and would catch a broken dot() or a BLAS .so that fails to load) but is deterministic. Also add docstrings to test_basic / test_matmul to match test_fft. Surfaced by running the recipe mobile tests on cp3.13/cp3.14 for the first time (mobile_test_pythons covering 3.13/3.14); test_basic and test_fft pass, only the timing gate was unreliable.
…ation" This reverts commit f54c405 -- concurrency was only disabled to keep the overlapping verification dispatches from cancelling each other; restore it now that verification is done.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
ALL × {3.12,3.13,3.14}build matrix went green on 3.12/3.13 but a cluster of numpy-consuming recipes failed on 3.14 —blis,shapely,rasterio,pandas,opencv-python,matplotlib— across iOS and (partly) Android. The symptoms looked like one numpy problem (ModuleNotFoundError: numpy._core._multiarray_umath,Python.h not found,libpython3.14.so missing) but were three independent root causes, each a version-fragile assumption that happened to hold on 3.12/3.13 and broke on 3.14. Each fix replaces the fragile assumption with a version-stable one, which is why none of them are version-guarded.Reproduced locally end-to-end (macOS arm64 + the python-build 3.14 support trees) and verified on CI across all three Python versions with mobile tests on — see Verification.
Changes
setup.sh— force a managed (relocatable) build interpreterThe macOS CI runner ships Homebrew's framework CPython, and
uvmatches it for an exact--python 3.14.5. crossenv mis-detectssys.prefix/sys.exec_prefixwith a framework build (RuntimeWarning: Unexpected value in sys.prefix, expected …/cross, got …/build), which mis-routes the target-arch host requirement (numpy) into the build venv instead of the cross venv. The build-platform numpy is then never installed, so the build backend's in-processimport numpy; numpy.get_include()(run via the cross-python by meson/setuptools) resolves to a non-importable target wheel and dies withNo module named numpy._core._multiarray_umath.Adding
--python-preference only-managedforces a relocatable python-build-standalone interpreter (what Linux already uses), which doesn't trip this. It's equal-or-safer for every version — it only ever swaps a framework interpreter for a relocatable one, never changes wheel content. Requiresuvnew enough to ship managed builds for the pinned versions (≥ 0.11.20; CI already uses it).recipes/pandas/meta.yaml—PYTHONSAFEPATH=1(recipe-scoped)meson runs the cross-python from pandas's unpacked source dir to introspect numpy. Python puts that cwd on
sys.path[0], and pandas ships a top-levelpandas/io/package that shadows the stdlibiomodule mid-import, so numpy's C-extension init fails withcannot import name 'TextIOWrapper' from 'io'.PYTHONSAFEPATHdrops the implicit cwd entry without touchingPYTHONPATH, so crossenv's import bridge is unaffected.It's scoped to pandas on purpose: a global setting also drops the script dir for
python codegen.pyinvocations and breaks recipes whose build generators import sibling modules — numpy'sgenapi, opencv'shdr_parser.recipes/opencv-python/meta.yaml— anchor Python lib/include onHOST_PYTHON_HOMEOn 3.14 the cross-venv prefix (
{prefix}→…/cross) no longer carrieslibpython{X.Y}or the python headers — they live in the support tree's python install. The CMAKE_ARGS pointed-DPYTHON3_LIBRARIES/-DPYTHON3_INCLUDE_PATHat{prefix}, so 3.14 failed at ninja link on Android (lib/libpython3.14.so … missing) and at compile on iOS ('Python.h' file not found). Switching both to{HOST_PYTHON_HOME}(android.so, iOS.dylib) resolves to the real per-version install on every python-build version — the same fix already proven on thecoolproprecipe.recipes/numpy/test_numpy.py— replace flaky perf gate with a correctness checktest_performanceassertedduration < 0.7for a 500×500 matmul as a proxy for "OpenBLAS is linked". mobile-forge's numpy is built without OpenBLAS (blas name="none"), so the matmul runs on the unaccelerated fallback and its wall-clock time swings widely on loaded/emulated devices (seen at 1.4–2.2s for the same wheel), randomly failing the iOS mobile-test lane. Renamed totest_matmuland now asserts the result (a @ I == a) plus shape; the timing is printed for visibility only. This surfaced once recipe mobile tests ran on cp3.13/cp3.14 for the first time.Verification
Full
{numpy, blis, shapely, rasterio, opencv-python, pandas, matplotlib, contourpy}×{3.12.13, 3.13.13, 3.14.5}×{android, iOS}with mobile tests on: 0 build failures across all three Python versions (no 3.12/3.13 regressions), 3.14 builds + mobile tests green on device/sim, opencv green on 3.14 both archs and clean on 3.12/3.13. Pre-fix the only reds were the flaky numpy perf test (now fixed) and one transientENOTFOUNDartifact-upload network blip.https://github.com/ndonkoHenri/mobile-forge/actions/runs/27358025685/attempts/1
Notes
No build-number bumps needed: the affected recipes never published a cp314 wheel (their builds failed), so the corrected cp314 wheels publish fresh at the existing build number, and the already-published cp312/cp313 wheels are untouched.