Automating Scheduled Jobs
You can scheduling recurring jobs to automate updating data, running mastering flows, and exporting mastered datasets.
In Admin > Scheduler, you can schedule the following recurring jobs:
- Refresh Source: This job refreshes the data in Tamr Cloud for a selected source dataset. For example, you might schedule a recurring job to refresh the data stored in a specific Snowflake table each Monday morning.
- Update Data: This job runs the mastering flow for a selected data product, and delivers updated data to Tamr Cloud. For example, you might schedule a recurring job to run the mastering flow each Monday afternoon.
All records are processed when mastering flow runs. For data that has not changed since the previous flow run, the existing Tamr ID is retained.
Note: This option is applicable only for legacy data products. - Publish Data Product: This job exports data to a configured destination for a selected data product. For example, you might schedule a recurring job to export data each Monday evening.
Note: This option is applicable only for legacy data products.
When scheduling a job, you select the frequency with which to run the job. For best results, leave a buffer of several hours between each job. The duration of each job depends on the size of the dataset and complexity of the mastering flow. Monitor how long jobs take to complete, and use that information to ensure you have an adequate buffer between jobs.
You can set one of the following frequencies for each job:
- Every n minute (increments of 5)
Note: Flow completion takes at least 30 minutes, depending on the number of records and steps in the flow. - Hourly
- Daily
- Weekly
You can edit and delete scheduled jobs, as well as pause and resume scheduled jobs.
Only admin users have access to Admin > Scheduler.
Scheduling a New Job
To schedule a new job:
- Navigate to Admin > Scheduler.
- Select Add Job.
- Select the job type to schedule: Refresh Source, Update Data, or Publish Data Product.
- In the Job Information section, provide a name and optional description for this job.
- Also in the Job Information section, provide the following information for the type of job you are scheduling:
- For Refresh Source, select the source to refresh from the Source dropdown.
- For Update Data, select the data product for which to run this job from the Data Product dropdown.
- For Publish Data Product, select:
- The data product for which to run this job from the Data Product dropdown.
- The export destination from the Destination dropdown.
- In the Schedule section:
- From the Repeat every dropdown, select the frequency with which to run the job. Then select the time to run the job:
- Minute: Select minute interval to run the job.
- Hour: Select hour interval to run the job, and number of minutes past the hour to run it.
- Day: Select time to run the job every day.
- Week: Select time and which days of the week to run the job.
- In the Timezones dropdown, select the time zone in which you want your automated job to run according to the set frequency.
- From the Repeat every dropdown, select the frequency with which to run the job. Then select the time to run the job:
- Select Add to schedule the job.
This job appears in the Scheduler table.
Editing a Scheduled Job
To edit a scheduled job:
- Navigate to Admin > Scheduler.
- In the Scheduler table row for the job, at right, select Edit .
- Update the job settings as needed and select Update.
Pausing or Resuming a Job
To pause or resume a scheduled job:
- Navigate to Admin > Scheduler.
- Select the checkbox for the job in the table row for the job.
- Select either Pause or Resume. The job's status updates to reflect your choice.
Deleting a Scheduled Job
To delete a scheduled job:
- Navigate to Admin > Scheduler.
- Select the checkbox for the job in the table row for the job.
- Select Delete and confirm the deletion.
Updated 4 months ago