Skip to content

Fix CI dependency caching: per-platform cache keys and fallbacks [androidbuild] [iosbuild]#41

Merged
ptesavol merged 2 commits into
mainfrom
ci/vcpkg-cache-keys
Jul 4, 2026
Merged

Fix CI dependency caching: per-platform cache keys and fallbacks [androidbuild] [iosbuild]#41
ptesavol merged 2 commits into
mainfrom
ci/vcpkg-cache-keys

Conversation

@ptesavol

@ptesavol ptesavol commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

You suspected the dependency caching does not really work in our CI runs — the suspicion was correct, and the evidence is unambiguous.

What was wrong

The cache key was built from the runner's processor architecture and operating system plus a hash of the dependency manifests. But three different jobs run on the same macOS runner image: the normal macOS build, the iOS build and the Android build. They therefore computed the same cache key while needing completely different dependency sets. GitHub's cache is write-once per key, so the first job to save — always the fast host build — owned the key forever, and the iOS and Android saves were rejected on every run (the "Cache save failed." messages visible in the logs).

Measured impact: every Android run spends twenty-three to thirty-one minutes and the iOS run thirty-five minutes, because their cross-compiled dependencies (folly, protobuf, openssl and the rest) are rebuilt from source every single time. The host macOS build, whose cache works, takes fourteen minutes.

Two further weaknesses fixed along the way: there was no fallback key, so any change to the dependency manifests or overlay files threw away the entire cache instead of reusing most of it; and the vcpkg tool's own version was not part of the key, so a tool upgrade silently reused stale installed trees.

The fix

  • The target platform (host, iOS or Android) is now part of both cache keys, so each platform owns its own cache and all saves succeed.
  • Fallback keys: on any exact miss the newest previous cache for the same platform is restored — vcpkg's own per-package hashing makes stale entries harmless, so a manifest change now rebuilds only what actually changed.
  • The vcpkg tool version (submodule commit) is part of the exact key.

What to expect

The first run with the new keys — this pull request, which runs all five platforms via the title keywords — is a deliberate cold build that populates each platform's cache for the first time. The improvement shows from the next run onwards: Android should drop from about thirty minutes to roughly twelve to fifteen, iOS similarly, since dependency installation becomes a quick unpack of cached binaries.

About the tutorial you linked (binary caching via GitHub Packages)

It is the right long-term design and this repository qualifies for it well (public repository, so the package storage is free). It gives per-package cache granularity in a registry with no total-size limit and no write-once semantics. I recommend it as a follow-up if the fixed key scheme proves insufficient — the five per-platform caches together may approach GitHub's ten-gigabyte total cache budget, and if evictions start appearing, the registry-based cache is the escape. The reasons not to jump straight to it: it needs the NuGet client on all runners (including installing it on your self-hosted Linux machine), token wiring with package-write permission, and this small fix likely captures most of the benefit already.

🤖 Generated with Claude Code

The macOS host, iOS and Android jobs all run on the same macos runner
image and therefore computed identical cache keys. GitHub's cache is
write-once per key, so the first job to save (the fast host build)
owned the key forever and the iOS/Android saves failed ("Cache save
failed." in the logs) - their cross-compiled dependencies (folly,
protobuf, openssl, ...) were rebuilt from source on every run:
measured ~23-31 min per Android run and ~35 min for iOS, versus ~14
min for the host build with a working cache.

- The target platform (host / --ios / --android, from ARCHFLAGS) is now
  part of both cache keys, so each platform owns its own cache.
- restore-keys prefix fallbacks: a manifest or overlay change now
  reuses the newest previous cache instead of rebuilding everything
  (vcpkg's per-package ABI hashing ignores stale archive entries, and
  vcpkg install reconciles a stale installed tree).
- The vcpkg submodule commit is part of the exact key (a tool bump
  previously reused stale installed trees silently).
- Saves are skipped when the exact key was already restored.

The first run with the new keys is a cold build by design (it
populates each platform's cache); the speedup shows from the next run
onwards.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 4, 2026

Copy link
Copy Markdown

Bugbot is not enabled for this team, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@github-actions github-actions Bot added the ci Pull requests that update Continuous Integration build label Jul 4, 2026
The NOT CMAKE_CROSSCOMPILING guard (added when Android modules were
enabled) is not enough on iOS: the iOS package builds keep the host
CMAKE_SYSTEM_NAME by design (Homebrew compiler, only the SDK/sysroot
is swapped), so CMake does not consider them cross-compiling and the
host-only protoc plugin was configured against the arm64-ios vcpkg
tree, which provides no protobuf::libprotoc. Restore the explicit IOS
condition (set by toolchains/ios.toolchain.cmake) alongside the
cross-compiling check. First iOS run since that commit caught it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 4, 2026

Copy link
Copy Markdown

Bugbot is not enabled for this team, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@ptesavol ptesavol merged commit ac64194 into main Jul 4, 2026
6 checks passed
ptesavol added a commit that referenced this pull request Jul 5, 2026
… restore the per-platform CI cache keys [androidbuild] [iosbuild] (#48)

streamr-json: one streamr.json.X sub-module per former header, no
umbrella, include/ tree deleted; the logger units' temporary textual
toJson.hpp include from C-6 becomes import streamr.json.toJson.

CI fix folded in per owner request: the C-2 commit had accidentally
reverted PR #41's per-platform vcpkg cache keys, making the Android and
iOS jobs share one write-once key with the host macOS leg — they have
been rebuilding all dependencies from source since (Android 48 -> 139
min as old caches were evicted). PR #41's key scheme (ARCHFLAGS
platform component, vcpkg submodule SHA, restore-keys fallbacks,
skip-save-on-exact-hit) is restored verbatim.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Pull requests that update Continuous Integration build proto-rpc

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant