Gaur Docs

The S3 connector ingests files from Amazon S3 or any S3-compatible object storage.

Supported file formats

The S3 connector reads these formats from a bucket:

Format	Extensions
Parquet	`.parquet`, `.parq`
CSV	`.csv`
TSV	`.tsv`
JSON	`.json`
JSON Lines	`.jsonl`, `.ndjson`

Each format takes the same options described under file formats. Excel files are not supported over S3; use file upload for Excel.

Create a secret

The S3 connector needs an S3 secret to reach your bucket. Create one with connector type S3 and these fields:

Field	Required	Description
`bucket_name`	Yes	The S3 bucket to read from.
`region`	No	The bucket's region. Defaults to `us-east-1`.
`access_key_id`	No	Access key ID for the bucket.
`secret_access_key`	No	Secret access key for the bucket.
`session_token`	No	Session token, for temporary credentials.
`endpoint`	No	Custom endpoint URL, for S3-compatible providers.

access_key_id and secret_access_key go together: provide both or neither. Omit them only when the environment supplies credentials another way. Set endpoint when you point at an S3-compatible provider rather than AWS.

The credentials are encrypted and reused for every ingestion and refresh from this bucket.

Ingest a table

Choose the S3 connector

In the Ingestion area, pick S3 and select the S3 secret you created.

Point at the data

Give the path to the file or files in the bucket (an s3:// path). Gaur reads the format from the path; set format options such as a CSV delimiter if needed.