Every bar on the perf page is one number, so "the tool is slow" and "Node is slow to start" are indistinguishable. For the JS competitors, startup is most of the bar — and on the two cells where FerrFlow currently looks slower, it is the whole story.
Worked from the cold data in run 29416210326:
| cell |
total |
startup |
work |
mono-large ferrflow |
1157 ms |
~5 ms |
~1152 ms |
mono-large changesets |
987 ms |
~700 ms |
~287 ms |
mono-large commit-and-tag-version |
2039 ms |
~520 ms |
~1519 ms |
Reading the totals alone, changesets "beats" us on 200 x 10k. Reading the work, we beat commit-and-tag-version on the analysis itself and 71% of the changesets bar is Node booting.
FerrLabs/Benchmarks#173 derives startup_ms / work_ms per cell, but it needs a floor fixture to subtract: 1 package, 1 commit since the last tag, benchmarked with each tool's normal command. --version would be the obvious floor and is wrong — it skips lazy requires, undercounts startup by ~286 ms, and bills the difference as the competitor's work, so the error flatters us.
Needed here: add benchmarks/fixtures/definitions/floor.json, copy it into every shard (startup is per-runner and shards land on different machines), and re-pin the action.
Every bar on the perf page is one number, so "the tool is slow" and "Node is slow to start" are indistinguishable. For the JS competitors, startup is most of the bar — and on the two cells where FerrFlow currently looks slower, it is the whole story.
Worked from the cold data in run 29416210326:
mono-largeferrflowmono-largechangesetsmono-largecommit-and-tag-versionReading the totals alone, changesets "beats" us on
200 x 10k. Reading the work, we beatcommit-and-tag-versionon the analysis itself and 71% of the changesets bar is Node booting.FerrLabs/Benchmarks#173 derives
startup_ms/work_msper cell, but it needs afloorfixture to subtract: 1 package, 1 commit since the last tag, benchmarked with each tool's normal command.--versionwould be the obvious floor and is wrong — it skips lazy requires, undercounts startup by ~286 ms, and bills the difference as the competitor's work, so the error flatters us.Needed here: add
benchmarks/fixtures/definitions/floor.json, copy it into every shard (startup is per-runner and shards land on different machines), and re-pin the action.