Chapter 5 of 8

Dictionaries and tables: the column store

Amber's table is not a new primitive bolted on. It is two things you already have, a dictionary and a flip, composed. Understanding that is worth more than memorising the table functions.

A dictionary is two vectors

d: `a`b`c ! 1 2 3      / keys ! values
d`a                    / 1        -- index by key
d`a`c                  / 1 3
! d                    / `a`b`c   -- the keys
. d                    / 1 2 3    -- the values
# d                    / 3

A table is a flipped dictionary of columns

cd: `sym`px ! (`a`b`c; 10 20 30)   / call it cd, NOT cols -- cols is a builtin
t: + cd                / that flip IS the table
t                      / a 3-row table with columns sym and px
t: ([] sym: `a`b`c; px: 10 20 30)   / the literal syntax for the same thing

Because the columns are the storage, a column is a plain vector and everything you learned in chapters 3 and 4 applies to it directly:

t: ([] sym: `a`b`c; px: 10 20 30)
t`px                   / 10 20 30      -- a column is just a vector
+/ t`px                / 60
t`px                   / 10 20 30
# t                    / 3             -- rows
cols t                 / `sym`px
t 0                    / a row, as a dictionary

This is why it is fast

Selecting a column is a pointer, not a copy or a gather. Summing one column of a fifty-column table touches only that column's memory. A row store has to walk past the other forty-nine.

Inspecting

t: ([] sym: `a`b`c; px: 10 20 30)
meta t                 / column names, types and attributes
show t                 / grid-print it
5 # t                  / first 5 rows -- take works on tables
cols t                 / column names
istable t              / 1

Keyed tables

A keyed table is a dictionary whose key and value are both tables. That is what select ... by returns, and what joins look up against:

kt: ([sym: `a`b`c] px: 10 20 30)
keys kt                / the key column(s)
value kt               / the value table
unkey kt               / back to a flat table
xkey[`sym; t]          / key an existing table by a column

Restructuring

t: ([] sym: `a`b`a`b; px: 10 20 30 40)
xgroup[`sym; t]        / one row per sym, px becomes a nested column
ungroup xgroup[`sym; t]    / and back again
xasc[`px; t]           / sort the table by a column
xdesc[`px; t]
xcols[`px`sym; t]      / reorder columns