Skip to content

perf(packages): overlap download/extract/install in cold pioarduino-resolve (~185-237s serialized) #953

Description

@zackees

Context

#942 Phase-1 baseline, cold scenario: pioarduino-resolve costs 185–237 s of the 700–808 s cold build. Harness stderr shows repeated sequential esp32-xtensa-gcc ... downloading lines followed by a single extracting — the pipeline downloads, then extracts, then installs, largely serially. fbuild-packages has a download_all() that spawns per-URL tasks, but the ESP32 toolchain/platform/framework ensure path doesn't fan out, and extraction doesn't overlap with the next download.

Additionally fw-libs (162 s cold) only starts after resolve completes — compile of framework libs could begin as soon as its toolchain + sources are present rather than waiting for all packages.

Proposal

  • Pipeline the package ensure path: concurrent downloads (bounded), extract-while-downloading (stream or start extract per completed archive), install-while-extracting.
  • Start dependent build phases as soon as their specific inputs are ready instead of barriering on full resolve (e.g. fw-libs can start when toolchain + core sources exist, while libraries still download).

Acceptance criteria

Part of #942.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions