Writing fast Amber

One rule carries most of the weight here: a primitive call costs about 60–100 nanoseconds before it touches a single element. Writing fast Amber means making fewer calls on longer vectors. Clever primitives matter much less.

The cost model in one table

Every figure below is one operation on 21,600 doubles, measured on the same machine in the same run. Read the right-hand column, not the left.

A quarter of a nanosecond per element is memory bandwidth. You are not going to beat it, and you do not need to. What you need to avoid is paying the fixed cost 1,800 times when you could pay it eleven times.

The mistake: each over the short axis

Suppose you have a matrix with many rows and few columns, 1,800 observations of 12 values each, and you want a running maximum across the 12 within every row. The obvious spelling is an each:

M: (1800; 12) # data     / 1800 rows, 12 columns
r: maxs'M                / running max within each row

That is 1,800 separate calls to maxs, each doing twelve elements of work. Measured: 4,430 µs. The same 21,600 elements as a single flat scan take 85 µs. The arithmetic is identical; the difference is 1,800 dispatches and 1,800 result allocations.

The tell is diagnostic and easy to check: if making the inner vectors longer barely changes your runtime, you are dispatch-bound. Grow the columns from 12 to 120 and a dispatch-bound kernel gets 1.9× slower while doing 10× the work.

The fix: transpose the problem, not the data

Store the same values as 12 columns of 1,800 instead of 1,800 rows of 12. The running max across observations becomes a scan over a list of vectors. That is eleven whole-column operations, identical element count, 150× fewer calls.

C: +M                    / 12 columns of 1800
r: (|\)C                 / 11 whole-column maxes

The same restructuring applies to every adverb. (+/)C sums across observations in eleven vector adds. C >' lvl compares each column against its own threshold in twelve calls rather than 1,800. The last observation, a row-wise *|' that cost 1,800 calls, becomes a single index: *|C.

Case study: a payoff kernel, 10× without touching the engine

A structured-payoff backtest computes, for each of 1,800 start dates, whether a note redeemed early, how many coupons it banked, and what it paid. That is three cumulative scans over a performance tensor. The first implementation stored the tensor row-major and was written entirely in each. Rewriting it column-major changed no arithmetic and no answer:

Outputs were bit-identical across a 500-basket differential test on every field. For context, a faithful NumPy port of the same algorithm runs at 626 µs per basket, so the row-major version was 2.3× slower than NumPy and the column-major version is 2.1× faster, with the same engine and the same maths.

What the compiler fuses for you

Since 2.1.0 the compiler recognises the shapes below and runs each as one pass with no intermediate vector, so the idiomatic spelling is also the fast one. Prefer these over hand-rolled alternatives:

Everything here is a general rewrite of the bytecode: a fused primitive falls back to the primitives it replaces for any operand shape it does not handle, so it never changes an answer. \disasm shows whether an expression fused. How the fusion pass was built, and why a rewrite that can silently change an answer is worse than no rewrite at all, is in two point one: the engine gets fused.

Threads and fused trees (2.6)

Since 2.6 big vectors use every core without you asking. Sort and grade, find and in, the aj match, gather, distinct, group aggregates and float sum/dot/max/min split over AMBER_THREADS threads once the input is big (32K items for most of them, 1M for the float reductions). And inside a lambda, a whole tree like (a*b)-c%2.0 over float or int vectors runs as one cache-sized pass instead of one full vector per verb. On a 14-core laptop, 10M items, against 2.5.0 on one core:

About 1.6× over the whole 24-op set. Things that are one fast pass already (int sum, scans) don't move, because memory is the limit there, not the core. Small vectors never touch the pool, and AMBER_THREADS=1 turns it all off. One thing to know: float sums of 1M+ items add in fixed 64K blocks, so the last digits can differ from a plain left-to-right sum, but they're the same at any thread count.

A short checklist

• Count your calls, not your flops. An each over n rows is n dispatches plus n allocations. Multiply by 100 ns and compare that to your total runtime before optimising anything else.
• Hoist loop-invariant work out of the each. A divisor, a threshold vector, a base index. Compute it once outside and pass it in as a projected argument.
• Prefer reduce over scan when you only need the last value. |/v is ten times cheaper than *|maxs v and says what you mean.
• Let booleans stay booleans. Comparisons produce a compact type with its own fast kernels; multiplying by 1.0 to "make it numeric" throws that away.
• Pre-warm large allocations. The first build of a big intermediate costs roughly twice the steady state. Do it when a session opens, not on the user's first query.
• Set attributes when you can honour them. `s on a sorted column turns a linear find into a binary search. See one byte at offset −13.
• Give peach real work per item. It's a thread pool, so there's no copy to pay, but a task still has a handoff cost: chunk tiny items so each task does at least tens of microseconds. A 400-basket sweep goes 451 ms → 215 ms on 14 threads.