Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Sample a table

Draw a sample of a large table into memory and work on it: the table, the query, Analysis, charts and export all read the sample.

S at the table opens the Sample form; Enter draws it.

datui -c analysis.sample_memory_limit=8GiB https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2025-01.parquet
KeyDoes
SOpen the Sample form, on the view’s current sample or a new one
EnterDraw the sample. After a memory warning, press it again to draw anyway
Esc in the formClose it; the view’s sample stays as it was
Esc while it is drawnStop; the rows so far stay. Before the first rows come, the view stays as it was
Method → No sample, or RRemove the sample

The form’s settings (which rows, the method, the size, the seed) are the analysis sample’s. Here the Every row method is labeled No sample.

A step of the view

The sample sits between the source and the query: the query, filters and sort run over the sample’s rows. The footer shows the steps in that order:

yellow_tripdata_2025-01.parquet › sample 100,000 of 3.48M › query   1 / 41,208
FooterMeans
sample 100,000 of 3.48M100,000 rows of the 3.48M the scope holds
sample about 100,000 of 3.48MEach row kept by chance, so the size is about what was asked for
sample 1,234+Still being drawn
sample 58,700 of 3.48M, stoppedStopped before its end: by Esc, or by memory
query › sample 2,000 of 41,208Drawn from the query’s rows: S on a queried view

To sample part of a table, choose it under Rows from (a row range, partitions, files, a time range), or query first and press S on the queried view. Removing the sample keeps the query, filters and sort, which then run over the source again. Removing a sample drawn from a query’s rows returns to that query.

A pivot reads all of its input, so no sample can be drawn underneath one. Sample the pivoted view instead (Rows from All rows). When a pivot sits on top of a sample, R removes both the sample and the pivot.

The same seed keeps the same rows. A random sample of a stream keeps each row by chance when the total is known, and uses a reservoir when it is not. When it is drawn again, in the session or from a view, it is drawn the same way as the first time.

Rows as they arrive

The table shows rows while the sample is being drawn, without moving the view, as a pipe does. Until the first rows arrive, the previous view stays; a draw that fails or stops before then leaves it as it was. Rows show in the order they arrive, then in the source’s order once the sample is complete.

ReadRows show
One Parquet or IPC fileEach of the 50 seeded blocks as it arrives
First rows, every rowEach batch as it is read
Random over a stream, the total knownEach row kept with chance size ÷ total, as it is read: about the size asked for
Random over a stream, the total unknownAll at the end; the footer counts the rows read meanwhile
Equal per valueAll at the end

While it is drawn, moving around, find, the inspector and the Info panel work at once. Anything that needs every row (a sort, a query, Analysis, a chart, an export) waits until it is drawn.

What reads it

WhereReads
AnalysisThe sample, whole; s there edits the view’s sample
ChartsThe sample, whole; the chart’s Rows row is hidden, and its note names the sample and its seed
ExportThe sample’s rows, with the query, filters and sort: the way to keep a sample on disk
ViewsThe sample’s settings, never its rows: applied again, it draws the same rows from the seed

Memory

The sample is held in memory and never written to disk. The form’s size line shows the cost when the table has measured its rows: 100,000 rows · ~380.0 MiB.

Whendatui
The estimate is more than the memory available nowWarns on the form, naming the setting; Enter again draws anyway, with no running check
Memory runs low while it is drawnStops, keeps the rows so far, and says so: Sample stopped at 3.9 GiB (58,700 rows): memory ran low. A sample that holds its rows until the end (an unknown total, or equal per value) stops when what it holds could not fit in memory twice, and keeps those rows
There is no estimateDraws, with the running check as the backstop

analysis.sample_memory_limit sets a fixed limit, checked before and during the draw. Unset, the memory available at the time decides. 0 turns off both the warning and the stop:

[analysis]
sample_memory_limit = "8GiB"