Benchmarks

Everything here is Amber 2.4.1. Unless a section says otherwise it was measured on one laptop (Intel Core Ultra 7 255U, WSL2, one core per run) in one sitting, and every answer was checked before its time counted. 2.6 added threads inside primitives; these tables stay one core per engine, and what the threads buy is on the performance page.

The engine matrix: 24 operations, 12 engines

The widest run there is: 24 operations against every array language and columnar engine that installs on one machine (PeachQ, ngn/k, CBQN, J, NumPy, pandas, Polars, DuckDB and a hand-written C reference), all on one thread. The spec is bench/scout/SCOUT_SPEC.md, the driver bench/scout/scout.py, and this block is generated straight from its results.json, so it can't drift from the repo.

Against the 2.2.0 run on the same laptop: distinct_100k 4.7× faster, sort_presorted 2.7×, mmax_64 2.1×, distinct 2.0×, find 1.6×, the moving sums 1.4–1.5× and sort_f 1.35×.

Amber is faster on 18 of 24 operations, slower on 6. Bars are log-scaled; the centre line is parity.

Milliseconds, lower is better. Amber-coloured is the fastest single-threaded engine in that row. skip means the engine has no faithful way to express the operation under SCOUT_SPEC.md, not that it failed.

Amber (native build), milliseconds. A 100× growth for a 100× row count is linear; below 100× is superlinear cache behaviour paying off, above it is the algorithm's log factor or a cache cliff.

The CI table

Four workloads, ten engines, re-run by CI on every push (bench/run_comparative.py, spec in bench/SPEC.md). This one runs on a small 2-core cloud box, so it doesn't line up with the laptop numbers above. Median of 5 runs after 2 warm-ups, kernel time only, ms; the small number under each is the multiple of C.

Amber is in there twice on purpose: Amber is array-primitive code, the fair peer of ngn/k, CBQN, J and Uiua; Amber qSQL goes through select … by … from, the fair peer of DuckDB's planner. The gap between the two rows is the query layer's cost.

Measured by someone else

Everything else on this page is Amber measuring itself against a spec in this repo. The Array Language Benchmarks Game isn't: its author picks the problems, writes the rules, runs every entry on their own machine (an i5-1135g7, 2 GB cap) and publishes the table. Amber has four entries there next to BQN, growler/k, Goal, q, Kap and C: spectralnorm and collatz in the current round, knucleotide and nbody in round 1.

The figures aren't copied here because the round is open and they move every week; read them on the game's own site, next to every entry's source. There's no restriction on algorithms, so a row compares two programs at least as much as two languages.

Against NumPy, on a real workload

One application kernel instead of a synthetic suite: a structured-payoff backtest, against a line-for-line NumPy port of the same algorithm (same tensor, same three scans, np.maximum.accumulate where Amber uses maxs). NumPy 2.5.2, both on one core, warm, best of repeated runs. It isn't flattering everywhere, which is the point.

Building the tensor: NumPy wins

S = 1800 start dates, n = 12 observations. The tensor at U = 100 is 17.3 MB and the U = 400 price history 14.7 MB. The first build after a cold start costs 2–4× a warm one, so build it when a session opens, not on the first query.

Running a product: Amber wins once there are enough observations

Amber pays a fixed cost per operation (allocate, dispatch) that barely moves as the data inside it grows; NumPy's per-operation cost is tiny, so its time tracks the data. Time per basket:

Amber's cost goes up 2.3× while the data grows 20×; NumPy's goes up 40×. They cross at about 24 observations (it was ~43 a few releases ago). A three-year quarterly note has 12 and belongs to NumPy; a ten-year monthly one has 120, where Amber is 3.8× faster.

Float scans

Up to 2.0 every float +\, *\, &\ and |\ (so sums, prds, mins, maxs on doubles) went through a generic loop that boxed an atom per element. Since 2.1 they're native. Same laptop, 2.0.1 against 2.4.1, best of 7:

And the ones a quant writes all day, on a 1,000,000-point price path:

Min and max go through the same order-preserving integer domain as the reductions, so a scan and a reduce agree bit for bit on NaN and ±0.0.

Multicore: what peach buys you

A basket sweep is embarrassingly parallel. peach is a persistent thread pool now (no fork, no copying the tensor to a child), so even one basket per task pays off. 400 baskets through the payoff kernel above, best of 5:

Same 400 answers in every configuration. It tops out around 2.1× on this laptop (2 performance cores, the rest efficiency cores); on heavier per-item work it scales further, e.g. {avg x?1.0} over 8 × 800k draws goes from 25.1 ms to 6.2 ms on 8 threads (4.0×).

Sorted attributes

bench.k times 5,000 lookups with ? (find) on the same column, with and without the sorted attribute. Without it, find builds a hash of the whole column every call; with it (asc sets it) it's a binary search per lookup. Same answers, very different times:

A real HFT run

demo/hft_demo.k on 500,000 trades and a million quotes: generate the tape, VWAP per symbol, a 50-period EMA and an as-of join of every trade to its quote. The script prints this itself:

== benchmark summary ========================================
stage      ms
gentq   418.7
vwap     10.2
ema       0.1
asof     15.0
------------------------------------------------------------
total: 444.1 ms for 500000 trades end-to-end

What each release moved

Each row is a before/after of one change, measured on the machine of its day, so read rows on their own, not against each other. The full tables are in docs/BENCHMARKS.md.

Running them yourself

./amber bench.k                        # sorted attributes vs plain find
./amber demo/hft_demo.k < /dev/null    # the HFT run (edit N at the top)
python3 bench/scout/scout.py --build    # the 24 x 12 engine matrix
python3 bench/run_comparative.py        # the ten-engine CI table
bash bench/run_suite.sh                 # vs numpy, pandas, Polars, DuckDB
./amber bench-std.k                     # moving windows, sorts

The per-engine query files are in bench/queries/, separately tuned scripts rather than one file reused, and each amber_*.k says in its header what was tried and why.