Steps Completed by Healthcare Providers
When you create a data product using the healthcare providers template, Tamr Cloud creates a mastering flow with steps specific to healthcare provider mastering.
The following table describes each step in the healthcare providers flow, and explains which steps may need to be edited for your data.
If you need to make changes to the mastering flow beyond those described in this documentation, contact Support at [email protected] for assistance.
Usually Requires Changes | Step | Description |
---|---|---|
Add Data | You verify that source data meets both general requirements and template-specific requirements. Then, you add source data. | |
Align to Healthcare Provider Model | You map input columns
to attributes in the supplied schema. If you add attributes to the unified schema, you will need to update the following steps, as described in the rows below, to ensure that these attributes appear in your final mastered entities:
|
|
Create tamr_record_id
|
This transformation step ensures that each source record has a unique primary key across all
source datasets by adding a new primary key field: tamr_record_id . The tamr_record_id
is a 128-bit hash value of the source dataset name and the source primary
key. See the
Tamr Core
documentation for a description of the function used to generate the hash value.
Important: If records within the same source dataset have duplicate primary key values, the tamr_record_id value for those records will also be duplicates.
If you mapped an empty placeholder column to the trusted_id attribute, add the following transformation: SELECT *, '' as trusted_id; .
|
|
Prepare Data for Enrichment | This step transforms the data in the unified dataset to match the expected inputs to the enrichers included in the mastering flow. | |
Enrich Address | This step standardizes and validates address information, and enriches addresses with latitude, longitude, and detailed address information. See Address Standardization, Validation, and Geocoding. | |
Prepare Address Fields | This step prepares address data to be used in the clustering model. | |
Prepare Data for Phone Enrichment | This step transforms the data in the unified dataset to match the expected inputs to the phone number data quality service included in the mastering flow. This step replaces empty country values in source records with the country returned by the Enrich Address step. | |
Enrich Phone | This step standardizes and validates phone number information. See Phone Number Enrichment. | |
Prepare for Clustering | This step transforms the data in the unified dataset to create the fields used by the
trained clustering model to identify similar and matching records.
The fields created as input to the model are prefixed with ml_ . Many of these ml_ fields are created as arrays of unified source
fields and fields added by the enrichment services. The model identifies
the most similar values across the arrays and assigns weights based on these similarities.
Because the cluster model uses full text values for specialty, some of the transformations in this step map input provider specialty codes to full text. If needed, you can add more mappings. See the Adding Provider Specialty Code Mapping section below. |
|
Apply Clustering Model | This step groups records that refer to the same entity into a cluster, using the trained model. See Features of Healthcare Providers. | |
Consolidate Records | This step applies rules to produce a single record, called the mastered entity record, that best represents a cluster. For most fields, these rules select the most common value from the clustered records.
Additionally, this step adds a Tamr ID ( tamr_id ) to each mastered entity record. The Tamr ID
is a unique, persistent id.
If you added new attributes in the Schema Mapping step, add lines in the transformations to tell Tamr Cloud what value to set for each attribute when creating the mastered entity. See Modifying Record Consolidation Transformations. |
|
NPPES Enrichment | This step enriches mastered healthcare provider entities with a matching National Provider Identifier (NPI) and detailed practice, credential, specialty, name, and other information from the National Plan & Provider Enumeration System (NPPES). See NPPES Enrichment. | |
Configure Attributes | You configure how mastered entity attributes appear in Tamr Cloud and published datasets. If you added new attributes in the Schema Mapping step, add and map those attributes in this step to include them in your final mastered entity output. See Configuring Data Display. |
Adding Provider Specialty Code Mappings
Because the cluster model uses full text values for specialty, some of the transformations in the Prepare for Clustering step map input provider specialty codes to full text, as shown in the code sample below. If needed, you can add more mappings, following the pattern in the transformation code.
//provider_specialty as ml_provider_specialty;
select *,
case
when provider_specialty == 'NRP' then 'Nurse Practitioner'
when provider_specialty == 'FM' then 'Family Medicine'
when provider_specialty == 'IM' then 'Internal Medicine'
when provider_specialty == 'PHA' then 'Physician Assistant'
when provider_specialty == 'OBG' then 'Obstetrics / Gynecology'
when provider_specialty == 'EM' then 'Emergency Medicine'
when provider_specialty == 'END' then 'Endocrinology, Diabetes and Metabolism'
when provider_specialty == 'RHU' then 'Rheumatology'
when provider_specialty == 'ID' then 'Infectious Disease'
when provider_specialty == 'PD' then 'Pediatrics'
when provider_specialty == 'ON' then 'Medical Oncology'
when provider_specialty == 'ORS' then 'Orthopedic Surgery'
when provider_specialty == 'CD' then 'Cardiovascular Disease'
when provider_specialty == 'N' then 'Neurology'
when provider_specialty == 'P' then 'Psychiatry'
when provider_specialty == 'HO' then 'Hematology/Oncology'
when provider_specialty == 'GP' then 'General Practice'
when provider_specialty == 'CHP' then 'Child and Adolescent Psychiatry'
when provider_specialty == 'GS' then 'General Surgery'
when provider_specialty == 'DR' then 'Diagnostic Radiology'
when provider_specialty == 'U' then 'Urology'
when provider_specialty == 'D' then 'Dermatology'
when provider_specialty == 'PUD' then 'Pulmonary Disease'
when provider_specialty == 'GE' then 'Gastroenterology'
when provider_specialty == 'PTH' then 'Anatomic/Clinical Pathology'
when provider_specialty == 'NS' then 'Neurological Surgery'
when provider_specialty == 'TS' then 'Thoracic Surgery'
when provider_specialty == 'NEP' then 'Nephrology'
when provider_specialty == 'OM' then 'Occupational Medicine'
when provider_specialty == 'GPM' then 'General Preventive Medicine'
when provider_specialty == 'MPD' then 'Internal Medicine/Pediatrics'
when provider_specialty == 'HEM' then 'Hematology'
when provider_specialty == 'PHO' then 'Pediatric Hematology/Oncology'
when provider_specialty == 'AN' then 'Anesthesiology'
when provider_specialty == 'PM' then 'Physical Medicine and Rehabilitation'
Updated 12 months ago