Writing fast Amber

There is one rule, and almost everything else follows from it: a primitive call costs about 60–100 nanoseconds before it touches a single element. Fast Amber is not about clever primitives. It is about making fewer calls on longer vectors.

The cost model in one table

Every figure below is one operation on 21,600 doubles, measured on the same machine in the same run. Read the right-hand column, not the left.

Operation	Time	ns / element
`+/v` sum reduce	2.4 µs	0.11
`\|/v` max reduce	8.3 µs	0.38
`v+v` vector add	14.9 µs	0.69
`v>0.5` compare	13.6 µs	0.63
`maxs v` running max	85 µs	3.9

A quarter of a nanosecond per element is memory bandwidth. You are not going to beat it, and you do not need to. What you need to avoid is paying the fixed cost 1,800 times when you could pay it eleven times.

The number to remember

One primitive call on a 12-element vector costs about the same as one primitive call on a 12,000-element vector. If you find yourself iterating over rows, you are buying dispatch, not arithmetic.

The mistake: `each` over the short axis

Suppose you have a matrix with many rows and few columns — 1,800 observations of 12 values each — and you want a running maximum across the 12 within every row. The obvious spelling is an each:

the slow shape

M: (1800; 12) # data     / 1800 rows, 12 columns
r: maxs'M                / running max within each row

That is 1,800 separate calls to maxs, each doing twelve elements of work. Measured: 4,430 µs. The same 21,600 elements as a single flat scan take 85 µs. The arithmetic is identical; the difference is 1,800 dispatches and 1,800 result allocations.

The tell is diagnostic and easy to check: if making the inner vectors longer barely changes your runtime, you are dispatch-bound. Grow the columns from 12 to 120 and a dispatch-bound kernel gets 1.9× slower while doing 10× the work.

The fix: transpose the problem, not the data

Store the same values as 12 columns of 1,800 instead of 1,800 rows of 12. The running max across observations becomes a scan over a list of vectors, which is eleven whole-column operations — identical element count, 150× fewer calls.

the fast shape

C: +M                    / 12 columns of 1800
r: (|\)C                 / 11 whole-column maxes

The same restructuring applies to every adverb. (+/)C sums across observations in eleven vector adds. C >' lvl compares each column against its own threshold in twelve calls rather than 1,800. And the last observation — which as a row-wise *|' was 1,800 calls — is just *|C, one index.

Rule of thumb

Arrange your data so the long axis is the one inside each primitive call, and the short axis is the one you iterate. If a dimension is small and fixed — observations, legs, tenors, buckets — it should be the list, not the vector.

Case study: a payoff kernel, 10× without touching the engine

A structured-payoff backtest computes, for each of 1,800 start dates, whether a note redeemed early, how many coupons it banked, and what it paid — three cumulative scans over a performance tensor. The first implementation stored the tensor row-major and was written entirely in each. Rewriting it column-major changed no arithmetic and no answer:

U = 100, S = 1800, n = 12	Row-major	Column-major	Speed-up
Tensor build	33.0 ms	10.4 ms	3.2×
Basket min-reduce	1,423 µs	206 µs	6.9×
Lifecycle kernel	1,311 µs	91 µs	14×
Per basket, total	2,734 µs	298 µs	9.2×
200-basket sweep	397 ms	39.0 ms	10.2×

Outputs were bit-identical across a 500-basket differential test on every field. For context, a faithful NumPy port of the same algorithm runs at 626 µs per basket — so the row-major version was 2.3× slower than NumPy and the column-major version is 2.1× faster, with the same engine and the same maths.

Before you blame the engine

The engine change that shipped alongside this work — native float scans, worth 15–36× on maxs/sums over doubles — made no measurable difference to this kernel, because once it was column-major its scans ran on booleans and integers, which were already fast. The layout was worth 10×; the engine fix was worth nothing here. Measure before you patch C.

What 2.1 fuses for you

Since 2.1.0 the compiler recognises the shapes below and runs each as one pass with no intermediate vector, so the idiomatic spelling is also the fast one. Prefer these over hand-rolled alternatives:

Write	Instead of	Why
`+/x*y`	materialising `x*y` first	fused dot product: no 80 MB temporary at 10M rows (14 ms → 7)
`+/x<50`, `+/x=y`	`#&x<50`	counts without a mask or index vector
`x@&m`, `+/x@&m`	explicit `&m` then index	compress by mask in one pass; the sum never writes the compressed vector
`y+2.5*x`	two statements	one pass, product rounded then added (identical bits)
`gsum[k;v]`, `select sum v by k from t`	`(+/v@)'=k`	one-pass group aggregate: 5–7× at 100–100k groups; the where-clause becomes a mask
`x in y`	`~^y?x`	one probe per element, one byte out (5×)
`asc x`, `x@<x`	grade-then-index by hand	counting sort for integral-valued data; the result is flagged `s, so the next `?`/`aj` on it is O(log n)

Everything here is a general rewrite of the bytecode: a fused primitive falls back to the primitives it replaces for any operand shape it does not handle, so it never changes an answer. \disasm shows whether an expression fused.

A short checklist

Count your calls, not your flops. An each over n rows is n dispatches plus n allocations. Multiply by 100 ns and compare that to your total runtime before optimising anything else.
Hoist loop-invariant work out of the each. A divisor, a threshold vector, a base index — compute it once outside and pass it in as a projected argument.
Prefer reduce over scan when you only need the last value. |/v is ten times cheaper than *|maxs v and says what you mean.
Let booleans stay booleans. Comparisons produce a compact type with its own fast kernels; multiplying by 1.0 to "make it numeric" throws that away.
Pre-warm large allocations. The first build of a big intermediate costs roughly twice the steady state. Do it when a session opens, not on the user's first query.
Set attributes when you can honour them. `s on a sorted column turns a linear find into a binary search — see one byte at offset −13.
Reach for more processes before more threads. peach forks, and on a small core count the copy can cost more than the parallelism returns.