probe: measure the GitCode upload path (TEMPORARY — do not merge)

[Sunrisepeak]

Temporary diagnostic branch — do not merge. All other workflows are deleted here on purpose so only the probe runs. Two rounds have now answered both the why and the what to do.

Problem

publish-ecosystem's GitCode leg has blown its 180s per-asset cap on four releases — 0.0.94, 0.0.97, 0.0.105, 2026.7.28.2 — always on the biggest tarballs, each time costing ~5 min of manual gtc release upload. Every previous fix guessed at the cause and tuned MIRROR_UPLOAD_TIMEOUT. This branch measured it.

Round 1 — why (run 30374882109)

Upload from a GitHub-hosted US-East runner to file.gitcode.com is a flat ~0.012 MB/s, independent of file size and of transport:

transport	1 MB	4 MB	16 MB	rate
urllib (what gtc does today)	88.8 s	failed put=-1	1529.9 s	0.010–0.011 MB/s
curl (streamed from disk)	83.1 s	330.7 s	1218.4 s	0.012–0.013 MB/s

4×8 MB serial: 672 / 650 / 675 / 599 s, wall 2618 s.

Controls:

direction	endpoint	rate
↑ upload, same runner	file.gitcode.com	0.012 MB/s
↓ download, same runner	gitcode.com	3.87 MB/s (~320×)
↑ upload, same runner	github.com	16 MB/s
↑ upload, mainland-CN host	file.gitcode.com	1.84 MB/s (~150×)

So it is not the Python client (curl matches it exactly) and not cross-border bandwidth in general (the same runner downloads from the same host at 3.87 MB/s). It is inbound rate limiting on the GitCode side for this egress.

At 0.012 MB/s the 34.8 MB asset needs ~48 minutes. The 180s cap was never within an order of magnitude of viable — which is why four rounds of tuning it failed.

Round 2 — what to do (run 30378781240)

The budget is per-connection, not per-host. Concurrency scales almost linearly:

N concurrent 1 MB	wall	aggregate	vs N=1
1	76 s	0.013 MB/s	1×
4	80 s	0.050 MB/s	3.8×
8	93 s	0.086 MB/s	6.6×

Per-connection rate stays 0.012–0.015 MB/s at every N.

But a single file cannot be split. The upload_url contract is a plain OBS V2 single-PUT presigned URL:

keys:         ['headers', 'url']
obs host:     file.gitcode.com
query params: ['AccessKeyId', 'Expires', 'Signature']
headers:      x-obs-meta-project-id, x-obs-acl, x-obs-callback, Content-Type

No uploads / partNumber / uploadId, and the API exposes no multipart endpoint — the signature cannot cover multipart operations. So intra-file parallelism is unavailable, and the largest asset is an irreducible ~45 min on GH-hosted runners.

No fixed timeout can be safe. 8 sequential 1 MB uploads: 0.011, 0.012, 0.012, 0.013, 0.015, 0.025, 0.051 MB/s — a ~4.6× spread. That is exactly why some releases got the mid-size assets in under 180s and others did not.

Concurrency adds an error rate. n4-3 returned put=502 with serves=false — a genuine failure, so any parallel upload path needs bounded retry.

The obs_callback false negative is now explained, not guessed. The x-obs-callback header carries base64 JSON with "callbackUrl": "https://api.gitcode.com/api/v5/repos/xlings-res/mcpp/releases/<tag>/obs_callback?access_token=***". OBS stores the object, then calls that URL; when the callback response can't be read, OBS returns 400 (... code:400 ... EOF) for a PUT whose object did land. Every probe therefore reports put_ok and serves independently, and this is reproducible.

Proposed solution

1. mirror_res.sh: upload a host leg's assets in parallel (measured 6.6× at N=8). Leg wall time becomes ≈ the largest asset (~45 min) instead of the sum (~100 min).
2. Delete the per-asset MIRROR_UPLOAD_TIMEOUT cap; use a whole-leg deadline instead (e.g. github 10 min, gitcode 75 min). The per-asset cap is what abandons assets and then fails the completeness gate; with a ~4.6× spread, no per-asset value can be both safe and useful.
3. gtc: re-probe the download URL before reporting an upload failure. The 400/EOF is a confirmed false negative; today it makes _upload_one sys.exit. Add bounded retry for the 502 class.
4. Split publish-ecosystem: the GitHub leg is fast and should gate the index bump PR; give the GitCode leg its own job with a ~90 min budget so a slow-but-progressing mirror can no longer take the release job down. This removes the manual step entirely — at the cost of one long-running job per release.
5. The actual speed fix, if wanted: run the GitCode leg on a CN-side self-hosted runner. Measured 1.84 MB/s from a mainland host ⇒ all 8 assets in ~45 s instead of ~45 min.

Not explored: GitCode may have another upload path (their web UI's) beyond the documented v5 upload_url endpoint that gtc uses. If one exists with multipart support, item 5 becomes unnecessary.

Footprint

Probe assets were random bytes under probe-<run_id>-* tags on xlings-res/mcpp, deleted by the cleanup job (verified: all deletes returned 200, and the release list shows no leftover probe tags).

[Sunrisepeak]

Analysis complete — the findings are implemented in #302 (S1: strip the shipped payload, 34.81MB → 4.62MB; S2: concurrent per-asset mirror upload + leg deadline). This branch was measurement-only and is not meant to merge; the full two-round report stays in the PR description for reference.