Chapter 8 of 8

Making it fast: fusion, windows, aggregates, charts and parallelism

Amber's position is that the idiomatic spelling should also be the fast one. Mostly that is true because the compiler recognises shapes and the library exposes one-pass kernels. This chapter is the list of both, so you know what you are already getting.

What the compiler fuses

These shapes compile to a single primitive with no intermediate vector. Write them, and you get the kernel:

write	instead of	what it saves
`+/x*y`	materialising `x*y`	the product vector, in a fused dot product
`+/x<50`	`#&x<50`	the mask and the index vector
`x@&m`	`&m` then index	the index vector
`+/x@&m`	compress then sum	both, in one pass
`a+s*b`	two statements	one pass, product rounded then added
`+/(a+s*b)@&m`	three statements	the whole chain in one pass
`x@<x`	grade then gather	a value sort with no gather

x: 1000 ? 100.0
y: 1000 ? 100.0
+/ x * y               / fused dot product
+/ x @ & x > 50        / fused masked sum
x @ < x                / fused value sort

You can see it for yourself, because \disasm prints the bytecode:

\disasm +/x*y

Moving windows

Single-pass O(n) sliding windows, front-padded to the input length:

x: 1 2 3 4 5 6.0
msum[3; x]             / moving sum over a 3-wide window
mavg[3; x]
mmax[3; x]
mmin[3; x]
mcount[3; x]
mdev[3; x]  /  mvar[3; x]

These are not the fin.k ones

fin.k also ships movavg, movsum and friends. The m* family here is the native single-pass implementation in the standard library and is what you want by default.

Group aggregates in one pass

select ... by uses these underneath, and you can call them directly when you have the key and value vectors already:

k: `a`b`a`b`a
v: 1 2 3 4 5
gsum[k; v]             / sum of v within each distinct k
gavg[k; v]
gmin[k; v]  /  gmax[k; v]
gcount[k]              / rows per key -- keys only, there is no value argument
gfirst[k; v]  /  glast[k; v]

Charts, in the terminal

No dependencies, no window, no browser. Braille line charts, candles and histograms drawn straight into the REPL:

plot 14*{sin x%7}'!74
hist 1000?100.0
spark 1 3 2 5 4 6
xyplot ({sin x%9}'!60; {cos x%9}'!60)

Write these compactly

{sin x%7}'!74 parses; {sin x % 7} ' ! 74 does not. An adverb binds to the verb immediately to its left, so putting spaces around ' separates it from the lambda it is meant to modify.

Temporal axes are labelled as clocks and calendars rather than as the integers underneath:

tm: ptime'("09:30:00.000"; "09:31:00.000"; "09:32:00.000")
tplot (tm; 10 12 11.0)          / the x axis reads as a clock, not as integers
chart `x`y`xunit ! (0 1 2; 10 12 11.0; `num)

Across cores

Since 2.6 you get some of this for free: big vectors (sort, find, in, distinct, group sums, float sums, fused arithmetic inside a lambda) already split over all the cores. For your own per-item work there's peach. It runs on a pool of threads, real parallelism with no interpreter lock, and is worth it when the per-item work is heavy:

peach[{avg x ? 1.0}; 4 # 100000]

Set AMBER_THREADS to pin the thread count; 1 makes everything serial, which is the comparison you want when deciding whether it helps.

Measuring, rather than guessing

\t:100 +/ x * y        / time 100 runs, in milliseconds
\trace +/ x * y        / four-phase timing and memory
\ast   +/ x * y        / the parse tree
\disasm +/ x * y       / the bytecode

The engine's own benchmark matrix, methodology and the operations where other engines win are published in Benchmarks.

Where to go next

Language reference: every verb, adverb and clause
Gotchas: the silent failures worth knowing about
Applied tutorials: a tick store, Grafana, Arrow
Performance: how to make your own code fast
\l examples/tour.k at the prompt: a worked example of every function