Sample a table
Draw a sample of a large table into memory and work on it: the table, the query, Analysis, charts and export all read the sample.
S at the table opens the Sample form; Enter draws it.
datui -c analysis.sample_memory_limit=8GiB https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2025-01.parquet
| Key | Does |
|---|---|
| S | Open the Sample form, on the view’s current sample or a new one |
| Enter | Draw the sample. After a memory warning, press it again to draw anyway |
| Esc in the form | Close it; the view’s sample stays as it was |
| Esc while it is drawn | Stop; the rows so far stay. Before the first rows come, the view stays as it was |
| Method → No sample, or R | Remove the sample |
The form’s settings (which rows, the method, the size, the seed) are the analysis sample’s. Here the Every row method is labeled No sample.
A step of the view
The sample sits between the source and the query: the query, filters and sort run over the sample’s rows. The footer shows the steps in that order:
yellow_tripdata_2025-01.parquet › sample 100,000 of 3.48M › query 1 / 41,208
| Footer | Means |
|---|---|
sample 100,000 of 3.48M | 100,000 rows of the 3.48M the scope holds |
sample about 100,000 of 3.48M | Each row kept by chance, so the size is about what was asked for |
sample 1,234+ | Still being drawn |
sample 58,700 of 3.48M, stopped | Stopped before its end: by Esc, or by memory |
query › sample 2,000 of 41,208 | Drawn from the query’s rows: S on a queried view |
To sample part of a table, choose it under Rows from (a row range, partitions, files, a time range), or query first and press S on the queried view. Removing the sample keeps the query, filters and sort, which then run over the source again. Removing a sample drawn from a query’s rows returns to that query.
A pivot reads all of its input, so no sample can be drawn underneath one. Sample the pivoted view instead (Rows from All rows). When a pivot sits on top of a sample, R removes both the sample and the pivot.
The same seed keeps the same rows. A random sample of a stream keeps each row by chance when the total is known, and uses a reservoir when it is not. When it is drawn again, in the session or from a view, it is drawn the same way as the first time.
Rows as they arrive
The table shows rows while the sample is being drawn, without moving the view, as a pipe does. Until the first rows arrive, the previous view stays; a draw that fails or stops before then leaves it as it was. Rows show in the order they arrive, then in the source’s order once the sample is complete.
| Read | Rows show |
|---|---|
| One Parquet or IPC file | Each of the 50 seeded blocks as it arrives |
| First rows, every row | Each batch as it is read |
| Random over a stream, the total known | Each row kept with chance size ÷ total, as it is read: about the size asked for |
| Random over a stream, the total unknown | All at the end; the footer counts the rows read meanwhile |
| Equal per value | All at the end |
While it is drawn, moving around, find, the inspector and the Info panel work at once. Anything that needs every row (a sort, a query, Analysis, a chart, an export) waits until it is drawn.
What reads it
| Where | Reads |
|---|---|
| Analysis | The sample, whole; s there edits the view’s sample |
| Charts | The sample, whole; the chart’s Rows row is hidden, and its note names the sample and its seed |
| Export | The sample’s rows, with the query, filters and sort: the way to keep a sample on disk |
| Views | The sample’s settings, never its rows: applied again, it draws the same rows from the seed |
Memory
The sample is held in memory and never written to disk. The form’s size line
shows the cost when the table has measured its rows: 100,000 rows · ~380.0 MiB.
| When | datui |
|---|---|
| The estimate is more than the memory available now | Warns on the form, naming the setting; Enter again draws anyway, with no running check |
| Memory runs low while it is drawn | Stops, keeps the rows so far, and says so: Sample stopped at 3.9 GiB (58,700 rows): memory ran low. A sample that holds its rows until the end (an unknown total, or equal per value) stops when what it holds could not fit in memory twice, and keeps those rows |
| There is no estimate | Draws, with the running check as the backstop |
analysis.sample_memory_limit sets a
fixed limit, checked before and during the draw. Unset, the memory available
at the time decides. 0 turns off both the warning and the stop:
[analysis]
sample_memory_limit = "8GiB"