Adding Source Data

You can add new source datasets from cloud storage locations or database connections, or upload files directly into Tamr Cloud.

Before you begin, see Requirements for Source Datasets.

Adding a Source from a Database Connection

For BigQuery and Snowflake, you can add a single source table at a time.

To add a new source from a BigQuery or Snowflake:

  1. Navigate to Configurations > Sources.
  2. Select Add Source.
  3. Enter a Display Namefor the source and optional description.
  4. Select the type of connection from which you are adding the source (BigQuery or Snowflake).
  5. Select a configured connection for adding this source.
  6. Enter the source table.
  7. Select Add Source.

The new source is listed on the Sources page and can be used in a data product.

Adding a Source from a Cloud Storage Connection

For ADLS, Delta Lake, GCS, and S3, you can add either a single file or multiple files from the same location. If adding multiple files, all the files must have the same column headers. Tamr combines these files into a single source dataset.

To add a new source from a ADLS, GCS, or S3 connection:

  1. Navigate to Configurations > Sources.
  2. Select Add Source.
  3. Enter a Display Namefor the source and optional description.
  4. Select the type of connection from which you are adding the source (ADLS, GCS, or S3).
  5. Select a configured connection for adding this source.
  6. Enter the path to the file. If you are adding multiple files, enter the path to the location at which these files are stored. All files must be the same file type and have the same schema.
    Note: For GCS sources, the file path cannot include colons.
  7. Select the file format: Delta Lake, Avro, CSV, New-line Delimited JSON (NDJSON), or Parquet.
  8. If you are adding a CSV file, specify the following format options:
    • Delimiter character. Default is comma ,.
    • Quote character used in field values. Default is double quote ".
    • Escape character for quote characters inside field values. Default is double quote ".
  9. Select Add Source.

The new source is listed on the Sources page and can be used in a data product.

Adding a Source through File Upload

To add a new source through file upload:

  1. Navigate to Configurations > Sources.
  2. Select Add Source.
  3. Enter a Display Namefor the source and optional description.
  4. Select Upload to upload a file directly into Tamr Cloud.
  5. Browse to the file for upload.
  6. Specify the following format options:
    • Delimiter character. Default is comma ,.
    • Quote character used in field values. Default is double quote ".
    • Escape character for quote characters inside field values. Default is double quote ".
  7. Select Add Source.

The new source is listed on the Sources page and can be used in a data product.