File upload
Ingest a dataset straight from a file on your machine.
The file upload connector ingests data straight from a file on your machine. It is the quickest way to get a one-off dataset or a small lookup table into Gaur, and it is the only connector that supports Excel files.
No secret needed
File upload is the one connector that needs no secret. The data comes directly from the file you upload, so there are no credentials to store.
Ingest a file
Choose file upload
In the Ingestion area, pick the file upload connector and select the file or files from your machine.
Set the format and options
Gaur detects the format from the file extension. Set any format options it needs, such as a CSV delimiter or an Excel sheet name. The options for each format are listed below.
Name the table and declare its primary key
Give the resulting table a name and declare its
primary key: the column or columns that
uniquely identify a row. When you map multiple files to multiple tables in
one operation, each file gets its own primary_key.
Gaur ingests the file, infers the schema, and creates the table.
Supported file formats
File upload accepts six formats.
| Format | Extensions | Notes |
|---|---|---|
| Parquet | .parquet, .parq | Columnar; schema and types are in the file. |
| CSV | .csv | Comma-separated text. |
| TSV | .tsv | Tab-separated text. |
| JSON | .json | A JSON document of records. |
| JSON Lines | .jsonl, .ndjson | One JSON record per line. |
| Excel | .xlsx, .xls | Spreadsheet workbooks. File upload only. |
Format options
Each format takes its own set of options.
CSV and TSV
| Option | Description |
|---|---|
delimiter | The field delimiter. Defaults to a comma for CSV and a tab for TSV. |
header | Whether the first row holds column names. |
ignore_errors | Skip rows that fail to parse instead of failing the ingestion. |
JSON and JSON Lines
| Option | Description |
|---|---|
records | How records are laid out in the document. |
maximum_object_size | The largest single JSON object size to accept. |
ignore_errors | Skip records that fail to parse instead of failing the ingestion. |
Parquet
Parquet has no format options. Column names and types are read directly from the file.
Excel
| Option | Description |
|---|---|
sheet | Which worksheet to read. Defaults to the first sheet. |
range | A cell range to read, for example A1:F500. |
header | Whether the first row of the range holds column names. |
all_varchar | Read every column as text rather than inferring types. |
empty_as_varchar | Treat empty cells as text rather than null. |
stop_at_empty | Stop reading at the first empty row. |
ignore_errors | Skip cells or rows that fail to parse instead of failing the ingestion. |
Refreshing
An uploaded file is a snapshot. Because the source is a file on your machine rather than a live system, keeping the table current means uploading new data. See Refreshes for the full and append modes available to file-based tables.