Blog
Longer-form writing on array programming, columnar engines, and the parts of market-data infrastructure that only look simple.
Two point nine: my kdb+ tickerplant, on Amber
Amber now talks to q: its wire format, its logs and the databases it writes. A real kdb+ options tickerplant runs on it with its .q files unchanged, and then rewritten the Amber way its cep does a batch in 6.7 µs instead of 175.
Checking the four colour theorem with arrays
The Dürre–Heesch reducibility check on Steinberger's 2822 configurations, from a 2023 bachelor project with Abel, Tomas, Lex and Jordy, moved from Python loops to Amber columns: every ring colouring and every Kempe interchange at once, over 100× faster than our Python original, and running live in the page.
Two point one: the engine gets fused
A performance release with no new syntax: the compiler fuses the common idioms, group-by, membership and distinct become one pass, price-shaped sorts become counting sorts, the portable build runs AVX2. Results bit-identical, and 11 of 19 shared operations now faster than CBQN.
Two point oh: infix dyads, qSQL in every script, and a 17x symbol group-by
The version number crossed a whole integer on ergonomics, not vocabulary. The two-argument
library dyads go infix, bare qSQL runs inside loaded .k scripts, symbol group-by is
~17× faster, and two engine bugs die.
Why high-frequency trading systems demand array languages
Moving beyond row-oriented overhead. A trade is not a row. It is one value in each of nine columns, and every question you ask of a trading day is a question about columns. What that costs you in an object-per-row runtime, measured rather than asserted.
Zero-copy interop: bridging Amber memory buffers directly into Apache Arrow & Python
What "zero copy" has to mean if the claim is going to survive contact with a refcount, a batch that outlives its stream, and a 32-byte object header sitting immediately before every payload.
Navigating temporal offsets: why Amber counts days and nanoseconds from 2000-01-01
A thirty-year offset is the most boring kind of bug and the hardest to see. Where the epoch choice comes from, what it buys, and how every integration in the ecosystem is built so the conversion is impossible to forget silently.
From language to platform: -8!, one-report diagnostics, owning the terminal, and libamber.so
Six point releases that added no vocabulary at all. What they added instead was a wire format, an error report you can act on, a line editor, an extension seam, and a shared library: everything an ecosystem needs to exist outside the repository.
Down to the metal: temporal types, SIMD, the arena, and a branch-free as-of join
The point where Amber stopped being a library written in K and started being an engine written in C. Four releases, five kernels, and one 16 MB bump allocator that exists purely to keep malloc out of the tick path.
One byte at offset −13: column attributes in C
`s, `u, `p and `g are not metadata. They are a promise the kernel is allowed to believe, stored in a byte nobody has to look for, and they are the difference between 4.23 seconds and 1.9 milliseconds.
Teaching K to speak q: tables, the join family, and bare qSQL
K has dictionaries and a transpose. Everything else a tick shop needs, from table literals and keyed tables to eight joins and select … by … from, had to be built. Here is how, and where I put each piece.
The interpreter underneath: what ngn/k is, and why Amber starts there
Before any of my code existed there was a 10,000-line C interpreter that fits in a cache. This is what it does, why K is shaped the way it is, and what I inherited.