Bro competitor benchmark — at a glance
Four rounds, each a different question. Round 4 is the bottom line.
- Round 4 — realistic workload (the headline). When tasks take real time (~2s each), Bro's content-addressed + remote cache is decisive: a clean CI worker with a primed remote cache rebuilds the whole graph in ~0.2s while make/just/go-task re-run ~4–34s of work (they have no remote cache). The per-run overhead that looked large in Round 2 is noise next to seconds of skipped work.
- Rounds 1–3 map the boundaries. R1 (a narrow Go pipeline) is Bro's accidental worst case. R2 (a polyglot monorepo) shows the per-package subgraph skip + the remote win at scale, and an honest local loss to make on the pure no-change cell (content hashing vs mtime — a deliberate correctness tradeoff). R3 (vs turbo) shows Bro wins the local cells but turbo's JS-niche remote cache is faster.
- Bro's defensible position: correct (content-addressed, dependency-output sound), universal (one config across Go/Python/TS/shell), remote-capable, low cold-start. Not "fastest at everything."
Bro vs make / just / go-task — Round 1 Benchmark
Round 1 of the competitor comparison defined in internal-docs/benchmark-plan.md. Everything here is reproducible from benchmarks/ alone:
benchmarks/setup.sh # clone corpus at pinned SHA, fetch tools, warm Go cache
benchmarks/run.sh 5 # run the matrix, 5 runs per cell
benchmarks/render.py # regenerate benchmarks/results.json + this documentMachine-readable output: benchmarks/results.json (raw per-run data in benchmarks/results-raw.jsonl). Bro at git 6219e4f.
Corpus
go-task/task @ d04748f0a4c0 — a real, single-module Go project (140 .go files). All four tools run the same three-task pipeline inside it:
vet (go vet ./...) -> test (go test ./...) -> build (go build -o bin/task ./cmd/task)Configs: benchmarks/configs/{Makefile,justfile,Taskfile.yml,bro.yaml} (copied into the corpus by setup.sh).
Environment
| Date | 2026-08-04T07:11:36+08:00 |
| CPU | 12th Gen Intel(R) Core(TM) i5-12400F (8 cores) |
| RAM | 15Gi |
| OS | Linux 6.6.87.2-microsoft-standard-WSL2 x86_64 GNU/Linux |
| Go | go1.26.1 |
| Bro | dev (git 6219e4f) |
| GNU make | 4.3 |
| just | 1.58.0 |
| go-task | 3.52.0 |
Methodology
- 5 runs per cell, median reported, min/max shown. Wall-clock measured around the full pipeline invocation (
bro run build/make/just build/task build), tool output discarded. - The Go build/test cache is shared infrastructure and stays warm for all tools and all runs — including "cold". "Cold" here means the tool's own state is cleared (bro:
.bro/+ declared outputs; make: stamp files +bin/task; go-task:.task/+bin/task; just:bin/task). Wiping$GOCACHEbetween runs would measurego vet/go testcompilation, not the task runners. This choice narrows absolute times equally for everyone. - No-change prepends one untimed full run, then measures the re-run.
- Leaf change appends a one-line comment to
variables.go(leaf file); root change does the same totask.go(root package, imported bycmd/task). Files are restored withgit checkoutafter each run. - go-task is configured with
sources:+method: checksum— its idiomatic up-to-date checking, i.e. its best foot forward. just has no such feature and always re-runs; that is the property being measured. - Each tool's config is the same three tasks; LOC counted with
wc -lbelow. - Change-scenario medians come out below cold medians for every tool: the measured run there immediately follows an untimed prep run of the same pipeline, so the page cache and Go's per-package test cache are as warm as possible, whereas cold runs follow other tools' activity.
Results
1. Cold full build
Tool state cleared; shared Go cache warm.
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 5.22 s | 5.17 s | 7.50 s | 5 |
| GNU make | 4.68 s | 4.55 s | 4.86 s | 5 |
| just | 4.34 s | 4.30 s | 4.51 s | 5 |
| go-task | 4.68 s | 4.57 s | 5.79 s | 5 |
2. No-change rebuild
Second consecutive run, nothing changed.
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 0.50 s | 0.47 s | 0.58 s | 5 |
| GNU make | 0.03 s | 0.03 s | 0.03 s | 5 |
| just | 2.94 s | 2.79 s | 3.61 s | 5 |
| go-task | 0.22 s | 0.21 s | 0.24 s | 5 |
3. Leaf change (variables.go)
One comment appended to variables.go.
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 3.82 s | 3.68 s | 3.90 s | 5 |
| GNU make | 2.92 s | 2.85 s | 2.95 s | 5 |
| just | 2.90 s | 2.75 s | 2.96 s | 5 |
| go-task | 3.12 s | 3.11 s | 3.26 s | 5 |
4. Root change (task.go)
One comment appended to task.go.
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 3.75 s | 3.68 s | 3.99 s | 5 |
| GNU make | 2.95 s | 2.81 s | 3.07 s | 5 |
| just | 2.89 s | 2.77 s | 3.13 s | 5 |
| go-task | 3.18 s | 3.08 s | 3.25 s | 5 |
Config size
| Tool | Config | Lines (wc -l) |
|---|---|---|
| Bro | bro.yaml | 23 |
| GNU make | Makefile | 24 |
| just | justfile | 17 |
| go-task | Taskfile.yml | 31 |
Analysis
Where Bro wins. On no-change rebuilds Bro is 6× faster than just (0.50 s vs 2.94 s): just has no up-to-date checking at all and re-runs go vet + go test + go build every time (only Go's own caches keep that at seconds instead of minutes). Bro's content-addressed cache turns the whole pipeline into a fingerprint lookup.
Where Bro does not win — stated plainly.
- Cold full build is parity, not a win. Bro is 1.12× make's median and 1.20× just's — just is fastest cold (4.34 s), being a bare command runner with zero bookkeeping, while Bro pays for DAG scheduling and input fingerprinting (hashing 140 files). The real work (
go vet/go test/go build) dominates and is identical for all four tools, exactly as predicted ininternal-docs/benchmark-plan.md§4. - No-change: make and go-task are faster in absolute terms. make's "Nothing to be done" (0.03 s) and go-task's checksum pass (0.22 s) beat Bro's 0.50 s. make only stats mtimes; go-task also hashes file contents but skips the extra work Bro does per run — one
go versionsubprocess per task (input_cmds) plus content-addressed cache bookkeeping. Sub-second either way; Bro pays for stronger correctness guarantees (mtime lies, e.g.git checkoutof an older commit) and for keying the cache on the toolchain version, which neither make nor go-task can express. - Leaf and root changes: Bro's clearest loss. Every task in this pipeline declares repo-wide inputs (
**/*.go), so one comment anywhere invalidates all three tasks — for Bro and for make's stamp files and go-task's checksums. With zero cache hits to harvest, Bro pays its costs (content hashing, storing the new cache entries) for no benefit and lands last: 3.82 s vs make 2.92 s and just 2.90 s on the leaf scenario (1.31× make, consistent across all 5 runs of both change scenarios; go-task sits between at 3.12 s). The takeaway is architectural, not a tuning miss: task-level caching only pays when a change leaves some task unaffected, and this 3-task pipeline has no unaffected branch to skip. A wider, finer-grained task graph (per the polyglot monorepo corpus ininternal-docs/benchmark-plan.md§2) is where the leaf-change scenario should flip to a Bro win; that is Round 2 material. - Go's own caches compress every gap. With
$GOCACHEwarm,go testre-uses cached test results for unchanged packages andgo buildis content-addressed internally — which is why even just's full re-run is only ~2.94 s. Bro deliberately does not try to beat the Go toolchain at caching Go compilation (seeinternal-docs/benchmark-plan.md§1); its value is the steps with no native cache, which a Go-only pipeline under-represents.
Out of scope for Round 1.
- turbo / nx are JS-monorepo tools; this corpus is a single Go module with no JS packages, so there is no meaningful equivalent pipeline to run. They belong in the polyglot-corpus round.
- Bazel deserves one honest paragraph: the comparison was attempted and abandoned at setup. A
go vet && go test && go buildpipeline in Bazel requiresrules_go+gazelle, aWORKSPACE/MODULE.bazelpinning the Go SDK and every dependency of the corpus, and generatedBUILDfiles per package — on the order of a hundred lines of Starlark plus a multi-hundred-MB first fetch of the toolchain and external repos, before the first action runs. That setup cost is the finding: for a pipeline this size, Bazel's per-action caching and remote execution are real, but its onboarding and config weight (the "config size" and "onboarding time" metrics ininternal-docs/benchmark-plan.md§5) disqualify it for the audience Bro targets. Quantifying that onboarding cost is left as a documented follow-up rather than silently dropping the cell.
Reproduce
benchmarks/setup.sh && benchmarks/run.sh 5 && benchmarks/render.pyPinned: corpus go-task/task@d04748f0a4c07139f46c82f5d493f48ff96a1124, just 1.58.0, go-task 3.52.0. benchmarks/corpus/ and benchmarks/tools/ are git-ignored; configs, scripts, and results are committed.
文档版本:0.1日期:2026-08-04
Bro vs make / just / go-task — Round 2 Benchmark (polyglot monorepo)
Round 2 of the competitor comparison defined in internal-docs/benchmark-plan.md. Where Round 1 ran a narrow 3-task Go pipeline (Bro's accidental worst case), Round 2 runs a synthetic polyglot monorepo with a wide task graph, then adds the remote-cache CI scenario no tool in this set can match.
Reproducible from benchmarks/ alone:
benchmarks/setup-r2.sh # generate corpus, fetch tools, warm shared caches
benchmarks/run-r2.sh 5 3 # matrix: 5 fast / 3 slow runs per cell
benchmarks/render-r2.py # results-r2.json + this sectionMachine-readable output: benchmarks/results-r2.json (raw per-run data in benchmarks/results-r2-raw.jsonl). Bro at git 6eaf3d2.
Corpus
A generated monorepo (benchmarks/gen-monorepo.py): 48 service packages (32 Go + 16 Python) plus a shared Go core at the DAG root — 146 tasks total. Each service runs a three-task pipeline, gen -> build -> test, where gen is a cache-less codegen step (no native tool caches it):
core-build (DAG root; root-change target)
└─ per service: gen (codegen) -> build -> test
└─ all (aggregator)Every per-package task uses dir: packages/<svc> with inputs: scoped to that package, so editing one package leaves the other ~48 packages' fingerprints unchanged — the subgraph skip Round 1's narrow graph could not exercise. Inputs/fingerprint are language-appropriate (Go tasks also fold go version into the key via input_cmds).
Configs (generated into the corpus): bro.yaml, Makefile (per-package stamp targets, mtime-based), justfile (no up-to-date check), Taskfile.yml (per-package sources: + method: checksum).
Environment
| Date | 2026-08-04T14:56:07+08:00 |
| CPU | 12th Gen Intel(R) Core(TM) i5-12400F (8 cores) |
| RAM | 15Gi |
| OS | Linux 6.6.87.2-microsoft-standard-WSL2 x86_64 GNU/Linux |
| Go | go1.26.1 |
| Bro | dev (git 6eaf3d2) |
| GNU make | 4.3 |
| just | 1.58.0 |
| go-task | 3.52.0 |
| Corpus | 48 services (32 go + 16 py), 146 tasks |
Methodology
- 5 runs per fast cell (nochange, leaf), 3 per slow cell (cold, root, remote), median reported, min/max shown. Wall-clock around the full
allinvocation; tool output discarded. - The Go build/test cache is shared infrastructure and stays warm for all tools and runs (as in Round 1); "cold" means the tool's own state is cleared (bro:
.bro+ outputs; make:.stamps+ outputs; go-task:.task+ outputs; just: outputs). Python's__pycache__is likewise shared and warm. - Leaf change appends a comment to
packages/svc-32/main.go(a leaf Go package); root change does the same topackages/core/main.go(core, imported by the whole graph via the task dependency). Files restored withgit checkout. - Remote cache (scenario 5): Bro's
httpbackend points at a tiny filesystem-backed server (benchmarks/cache-server.py) implementing Bro's REST protocol. The remote is primed by one cold Bro run, then each measured run is a clean worker (bro clean --outputs— local cache + outputs gone, remote untouched). make/just/go-task have no remote cache, so their clean worker is a full cold rebuild. - go-task and make are configured idiomatically (per-package checksum / stamp targets) so their incremental skip is their best form — Bro is not given an unfair opponent.
Results
1. Cold full build
Tool state cleared; shared caches warm.
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 2.74 s | 2.69 s | 3.07 s | 3 |
| GNU make | 2.67 s | 2.65 s | 2.69 s | 3 |
| just | 11.44 s | 10.83 s | 12.66 s | 3 |
| go-task | 3.31 s | 3.23 s | 3.34 s | 3 |
2. No-change rebuild
Second consecutive run, nothing changed.
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 0.50 s | 0.48 s | 0.53 s | 5 |
| GNU make | 0.03 s | 0.02 s | 0.03 s | 5 |
| just | 8.71 s | 8.65 s | 9.03 s | 5 |
| go-task | 0.22 s | 0.21 s | 0.22 s | 5 |
3. Leaf-package change
One comment appended to packages/svc-32/main.go.
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 0.72 s | 0.69 s | 0.77 s | 5 |
| GNU make | 0.36 s | 0.35 s | 0.36 s | 5 |
| just | 9.04 s | 8.94 s | 9.47 s | 5 |
| go-task | 0.55 s | 0.53 s | 0.56 s | 5 |
4. Root-package change (core)
One comment appended to packages/core/main.go (fans out to the whole graph).
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 0.91 s | 0.89 s | 0.97 s | 3 |
| GNU make | 2.06 s | 2.05 s | 2.07 s | 3 |
| just | 9.20 s | 8.59 s | 9.40 s | 3 |
| go-task | 0.35 s | 0.35 s | 0.35 s | 3 |
5. CI clean worker + primed remote cache
Clean CI worker (local state cleared); Bro's remote cache primed by a prior run, the others have no remote cache.
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 1.22 s | 1.21 s | 1.22 s | 3 |
| GNU make | 2.58 s | 2.45 s | 2.77 s | 3 |
| just | 11.25 s | 10.57 s | 11.60 s | 3 |
| go-task | 3.52 s | 3.08 s | 3.82 s | 3 |
Analysis
The headline — remote cache. On a clean CI worker (local state cleared) with a primed remote cache, Bro rebuilds all 146 tasks in 1.22s — 145 of them replayed from the remote, only the all aggregator executes. make/go-task/just have no remote cache, so the same clean worker is a full cold rebuild (2.58s / 3.52s / 11.25s). Bro is 2.1× faster than the best competitor on the one cell that matters most for CI — where every run starts on a fresh machine — and none of make/just/go-task can do this at all. (turbo/nx can; see "Out of scope".) The margin is moderate rather than huge because this corpus's tasks are deliberately lightweight, so per-run fingerprinting (below) is a large share of Bro's wall-clock; with heavier realistic tasks the skipped execution dominates and the win widens.
Bro beats just everywhere, often by an order of magnitude. just has no up-to-date checking and runs recipes serially, so every invocation re-executes the whole 146-task graph: 18× slower than Bro on a no-change rebuild (8.71s vs 0.50s) and 13× slower on a leaf change (9.04s vs 0.72s). Same evidence as Round 1, now at monorepo scale.
Local incremental: an honest loss to make and go-task. This is the most important non-win. Bro matches make and go-task on capability — all three skip the ~48 unchanged packages on a leaf edit — but Bro is slower in absolute time: no-change 0.50s vs make's 0.03s, leaf 0.72s vs make's 0.36s and go-task's 0.55s. The cause is per-run fingerprinting at scale: Bro content-hashes every task's inputs and, for each of the 32 Go tasks, runs a go version subprocess (input_cmds) to key the cache on the toolchain. Across 146 tasks that is roughly 0.50s of overhead on every run, which make's mtime stat (0.03s) and go-task's checksum pass (0.22s) largely avoid. It is the correctness tax for content addressing and toolchain keying. Bro now single-flights input_cmds across tasks that share a directory (a package's build and test share one go version), which is why this cell stays well under a second; each package directory still triggers its own toolchain probe, because the memo key has to include dir to stay correct for dir-sensitive commands like git rev-parse HEAD.
Cold and root. Cold is parity with make/go-task (real work dominates; Bro is 1.02× make) and well under just's serial 11.44s. The root change exposes a dependency-model difference: Bro (0.91s) and make (2.06s) soundly re-run the whole graph — Bro because a dependency's output-hash change propagates to every dependent (spec §2.7), make via its stamp chain — and Bro rebuilds it markedly faster than make here, its worker pool re-running the independent packages in parallel without make's per-target stamp-file bookkeeping. go-task rebuilds only core (0.35s): its per-task method: checksum checks each task's own declared sources: and does not propagate a dependency's change to its dependents. In this corpus the services don't consume core's artifact, so all three produce identical correct results and go-task's skip is the cheap, correct choice here — but it is unsound the moment a task uses a dependency's output without declaring it as a source, the exact staleness Bro's dep-output-hash fingerprinting exists to prevent.
Soundness, summarized. Bro's correctness guarantees cost real time at this scale (content hashing, toolchain keying, dependency-output propagation). In return none of the failure modes that bite make/go-task apply: mtimes do not lie to Bro after a git checkout, a toolchain upgrade invalidates exactly the affected tasks, and a changed dependency always rebuilds its dependents.
The designed battlefield, vs Round 1. Round 1's 3-task pipeline declared repo-wide inputs, so any change invalidated everything and Bro lost the change cells. Round 2's per-package scoped inputs turn a one-package change into a 48-of-49 subgraph skip — the cell Round 1 could not represent — and add the remote-cache CI scenario that is Bro's decisive, unique win.
Out of scope for Round 2.
- turbo / nx are the genuine remote-cache competitors (both have content + remote caches). They are JS-focused, so on this Go/Python corpus they cover only a slice, and standing up their own remote-cache server is Round-1-Bazel territory. A head-to-head remote-cache cell against turbo on a TS-heavy corpus is settled in Round 3 below: turbo wins the remote cell (~2.6×), Bro wins cold / no-change / leaf.
- Bazel is excluded for the same reason as Round 1: its
rules_go/gazelle setup cost (hundreds of lines of Starlark + a large first fetch) is itself the finding — it disqualifies Bazel for Bro's audience on the config-size and onboarding axes.
Reproduce
benchmarks/setup-r2.sh && benchmarks/run-r2.sh 5 3 && benchmarks/render-r2.pybenchmarks/corpus/monorepo/ is gitignored (regenerated by gen-monorepo.py); configs, scripts, and results are committed.
Bro vs turbo — Round 3 Benchmark (remote-cache head-to-head, TS monorepo)
Round 3 narrows in on the one scenario Rounds 1–2 couldn't settle: the remote-cache CI cell, against the tool that actually has a remote cache too. turbo (and nx) were out of scope in Rounds 1–2 because make/just/go-task have no remote cache at all; here Bro and turbo go head-to-head on the battlefield turbo was built for — a TypeScript monorepo.
Reproducible from benchmarks/ alone:
benchmarks/setup-r3.sh # generate TS corpus, npm install turbo/typescript/server
benchmarks/run-r3.sh 5 3 # matrix: 5 fast / 3 slow runs per cell
benchmarks/render-r3.py # results-r3.json + this sectionMachine-readable output: benchmarks/results-r3.json (raw in benchmarks/results-r3-raw.jsonl). Bro at git 5bf71df.
Corpus
A generated TypeScript monorepo (benchmarks/gen-ts-monorepo.py): 16 packages, each a build task (tsc: src -> dist) — 17 tasks total. Both tools run the same per-package build on it, each with its own equivalent config and its own remote cache:
- turbo —
turbo.json+ per-packagepackage.json(scripts.build), remote viaTURBO_API(the open-sourceturborepo-remote-cacheserver on local filesystem storage). - Bro —
bro.yamlper-package tasks (dir+ scopedinputs+outputs: dist), remote via thehttpbackend (benchmarks/cache-server.py).
Environment
| Date | 2026-08-04T20:03:12+08:00 |
| CPU | 12th Gen Intel(R) Core(TM) i5-12400F (8 cores) |
| RAM | 15Gi |
| OS | Linux 6.6.87.2-microsoft-standard-WSL2 x86_64 GNU/Linux |
| Node | v22.19.0 |
| TypeScript | 7.0.2 |
| turbo | 2.10.8 |
| Bro | dev (git 5bf71df) |
| Corpus | 16 TS packages, 17 tasks |
Methodology
- 5 runs per fast cell (nochange, leaf), 3 per slow cell (cold, remote), median reported, min/max shown. Wall-clock around the full build; tool output discarded. The shared
tsccompile cache stays warm. - Remote (scenario 4): each tool primes its OWN remote from one cold run, then each measured run is a clean worker (Bro:
bro clean --outputs; turbo: wipenode_modules/.cache/turbo+.turbo+dist) that hits its own remote. No cross-tool cache sharing — this compares the machinery, not a shared store. - turbo cache-correctness note: turbo hashes every non-gitignored file in a package, so build outputs (
dist/,node_modules/,.turbo/) MUST be gitignored or the cache key changes when outputs appear/disappear. The corpus ships that.gitignore. (Bro scopes inputs explicitly, so it has no such requirement.)
Results
1. Cold full build
Tool state cleared; shared tsc cache warm.
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 0.29 s | 0.29 s | 0.30 s | 3 |
| turbo | 0.63 s | 0.60 s | 0.66 s | 3 |
2. No-change rebuild
Second consecutive run, nothing changed (local cache).
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 0.01 s | 0.01 s | 0.01 s | 5 |
| turbo | 0.04 s | 0.04 s | 0.04 s | 5 |
3. Leaf-package change
One comment appended to a leaf package's src/index.ts.
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 0.07 s | 0.07 s | 0.08 s | 5 |
| turbo | 0.20 s | 0.20 s | 0.23 s | 5 |
4. CI clean worker + primed remote cache
Clean CI worker (local state cleared); each tool hits its own primed remote.
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 0.15 s | 0.15 s | 0.15 s | 3 |
| turbo | 0.06 s | 0.05 s | 0.06 s | 3 |
Analysis
A genuine split — and that is the honest result. Round 1–2's remote-cache "win" was Bro versus tools that have no remote cache at all. Against turbo, which has one, the picture is nuanced:
- Bro wins cold and local no-change. Bro's per-invocation overhead is lower than turbo's: cold 0.29s vs 0.63s (2.2×) and no-change 0.01s vs 0.04s. turbo pays a Node/Rust-binary startup tax on every invocation; Bro's static Go binary starts faster. On a leaf change Bro is faster.
- turbo wins the remote-cache cell — the one this round exists to settle. On a clean worker with a primed remote, turbo rebuilds all 16 packages in 0.06s vs Bro's 0.15s (2.6× turbo). turbo's remote path (a Rust client, lean artifact fetch/restore) is markedly faster than Bro's for these small JS artifacts. Bro still hits its remote for every task and produces a correct build — it just pays more per task in the download/restore path than turbo does.
What this confirms about Bro's positioning. Bro's value in Rounds 1–2 (remote cache that make/just/go-task lack) stands. But turbo is the stronger tool within its niche — JS/TS monorepos where its remote cache and scheduling are purpose-built. Bro's edge is universality (the same bro.yaml runs the Go, Python, and shell steps turbo can't express), lower cold-start overhead, and explicit-input correctness; turbo's edge is raw JS-monorepo remote-cache speed. Neither dominates: pick by workload.
A shared correctness footnote. Both tools' caches are only as correct as their input model: turbo's hash is "everything not gitignored" (hence the mandatory output gitignore above), Bro's is "explicitly declared inputs plus dependency-output hashes." Bro's is sound by construction even when a task consumes an undeclared dependency output (spec §2.7); turbo's requires the developer to keep .gitignore truthful about what is generated.
Reproduce
benchmarks/setup-r3.sh && benchmarks/run-r3.sh 5 3 && benchmarks/render-r3.pybenchmarks/corpus/ts-monorepo/ (incl. node_modules/) is gitignored; configs, scripts, and results are committed.
Round 4 — realistic workload (heavy tasks)
Rounds 1–3 used lightweight tasks (tens of ms each), so Bro's per-run fingerprint+cache overhead looked large next to the work it skipped. Round 4 gives each task a realistic cost — sleep 2s, standing in for a real test or build that takes 2 seconds (a task runner cannot tell sleep from real work). With ~34s of real work in the graph, skipping it via cache is worth seconds, and Bro's ~0.4s overhead is noise. This is the trade Bro is designed for.
Reproducible from benchmarks/ alone:
benchmarks/gen-heavy-monorepo.py --packages 16 --sleep 2 --force
benchmarks/run-heavy.sh 5 3
benchmarks/render-heavy.pyMachine-readable output: benchmarks/results-heavy.json (raw in benchmarks/results-heavy-raw.jsonl). Bro at git 1bf3bb3.
Corpus
A generated monorepo: 16 service packages + a shared core, each build task sleeps 2s (16+2 tasks). All four tools run the same per-package build with equivalent configs (Bro: scoped inputs + cache; make: stamp files; go-task: sources+checksum; just: no up-to-date check). The cache servers are the same harness as Round 2 (Bro http backend + cache-server.py).
Environment
| Date | 2026-08-04T22:51:30+08:00 |
| CPU | 12th Gen Intel(R) Core(TM) i5-12400F (8 cores) |
| RAM | 15Gi |
| OS | Linux 6.6.87.2-microsoft-standard-WSL2 x86_64 GNU/Linux |
| Go | go1.26.1 |
| Bro | dev (git 1bf3bb3) |
| GNU make / just / go-task | 4.3 / 1.58.0 / 3.52.0 |
| Corpus | 16 packages, each task sleeps 2s (~34s of work) |
Methodology
5 runs per fast cell (nochange, leaf), 3 per slow cell (cold, root, remote), median reported. "Cold" = tool's own state cleared. The remote cell is the CI story: Bro's remote is primed by one run, then each measured run is a clean worker (local cache + outputs cleared, remote untouched) — i.e. "machine A primed it, machine B is a fresh checkout." make/just/go-task have no remote cache, so their clean worker is a full cold run.
Results
1. Cold full build
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 6.04 s | 6.03 s | 6.04 s | 3 |
| GNU make | 6.03 s | 6.02 s | 6.03 s | 3 |
| just | 34.06 s | 34.06 s | 34.06 s | 3 |
| go-task | 4.22 s | 4.19 s | 4.23 s | 3 |
2. No-change rebuild
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 0.01 s | 0.01 s | 0.01 s | 5 |
| GNU make | 0.00 s | 0.00 s | 0.01 s | 5 |
| just | 34.06 s | 34.05 s | 34.06 s | 5 |
| go-task | 0.19 s | 0.17 s | 0.23 s | 5 |
3. Leaf-package change
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 2.01 s | 2.01 s | 2.02 s | 5 |
| GNU make | 2.01 s | 2.01 s | 2.01 s | 5 |
| just | 34.06 s | 34.05 s | 34.06 s | 5 |
| go-task | 2.18 s | 2.17 s | 2.27 s | 5 |
4. Root-package change (core)
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 2.02 s | 2.02 s | 2.04 s | 3 |
| GNU make | 6.03 s | 6.03 s | 6.03 s | 3 |
| just | 34.05 s | 34.05 s | 34.06 s | 3 |
| go-task | 2.19 s | 2.19 s | 2.21 s | 3 |
5. CI clean worker + primed remote cache
Clean CI worker (local state cleared); Bro's remote primed, the others have none.
| Tool | Median | Min | Max | Runs |
|---|---|---|---|---|
| Bro | 0.23 s | 0.23 s | 0.23 s | 3 |
| GNU make | 6.03 s | 6.03 s | 6.03 s | 3 |
| just | 34.06 s | 34.06 s | 34.06 s | 3 |
| go-task | 4.21 s | 4.20 s | 4.24 s | 3 |
Analysis
The headline — with real work, Bro's cache is decisive. On a clean CI worker with a primed remote cache, Bro rebuilds the whole graph in 0.23s — it skips every task's 2s sleep by replaying outputs from the remote. make and go-task have no remote cache, so they re-run all ~34s of work (6.03s / 4.21s); just re-runs it serially (34.06s). Bro is 18× faster than the best competitor on the cell that matters most for CI. This is the same capability Rounds 1–2 established; Round 4 shows it at a scale where the skipped work is unambiguous.
Locally, skipping is everything; overhead is noise. No-change and leaf changes skip the unchanged packages' sleeps: Bro (0.01s), make (0.00s) and go-task (0.19s) all skip the same work, just re-runs it all (34.06s). Bro's no-change cost (0.01s) is 0.0% of the ~34s of work it skips — negligible. (Round 2's lightweight corpus made Bro's overhead look dominant only because the skipped work was ~tens of ms.)
Cold is real-work-bound; root reveals the propagation model. Cold (nothing cached) is parity — Bro, make and go-task run the 2s tasks in parallel (6.04s / 6.03s / 4.22s); just serializes (34.06s). The root change here is benign — core's output is an empty marker file, so editing core's source changes nothing its dependents consume — and the tools split: Bro (2.02s) and go-task (2.19s) rebuild only core (content / checksum), while make (6.03s) conservatively re-runs every dependent via its stamp chain. (A root change that actually changed the shared output would fan out for all three.)
Honest scope. The sleep tasks are a proxy for real commands; they model the task-runner's view faithfully (skip = don't run the command) but don't capture toolchain-internal caching (e.g. go test's own cache), which only narrows gaps further. The corpus is generated and small; the point is the shape of the result (skip seconds of work for sub-second overhead), which scales with task weight.
Reproduce
benchmarks/gen-heavy-monorepo.py --packages 16 --sleep 2 --force && benchmarks/run-heavy.sh 5 3 && benchmarks/render-heavy.pybenchmarks/corpus/heavy/ is gitignored (regenerated); scripts and results are committed.