Different process involved in informatica MDM

Informatica MDM includes various processes for processing data from sources. The processes involved in Informatica MDM are:

Data can be loaded into MDM Hub and processed across multiple batch processes. You can move data to your MDM hub using a land process. You can then use the staging process to clean and prepare the data for the load process that loads the data into the base object. Once loaded, you can run the tokenization and matching process to identify duplicate records and consolidate them through the consolidation process.

Land Process:

This part describes concepts and tasks related to land processes in Informatica MDM Hub.

Stage Process:

The staging process transfers source data to staging tables associated with specific base objects. You can run the entire phase process on the MDM Hub or Informatica platform. During the staging process, source data is moved to a staging table associated with the base object. You can run a staging process to load data into staging tables directly from your source system or MDM Hub landing tables. When you integrate MDM Hub with the Informatica platform, you can load data directly from your source systems into MDM Hub staging tables. If you do not integrate MDM Hub with the Informatica platform, load source data into landing tables so that the staging process moves the data to staging tables.

Load Process:

In Informatica MDM Hub, a load operation moves data from a staging table to a corresponding target table (base object) in hub storage.

The load process uses the following information to decide what to do with the data in the staging table.

Whether the corresponding record already exists in the target table, and if so, whether the record in the staging table has been updated since the last load.

Whether trust is enabled for a particular column (base objects only). In this case, the loading process calculates confidence values ​​for the cell data.

Whether the data to be loaded is valid. Otherwise, the load process rejects the record instead.

Other configuration settings

Tokenize Process:

The tokenization process generates match tokens and stores them in the match key table associated with the base object. The match token is then used by the match process to identify potential matches.

Match Process:

The match process uses match columns and match rules to:

Identifies similar or identical records within a base object.

Identify records that are candidates for automatic consolidation.

Identify potential records for review by data stewards before integration.

Consolidate Process:

After the matching process identifies matched pairs, consolidation is the process of consolidating the data from the matching records into a single master record.

Delta Detection in informatica MDM and process of enabling delta detection.

Delta detection is used to detect new inserts and update existing source datasets in a complete data load process. Delta detection is done on specific columns that can be configured at the stage table level.

Considerations for Using Delta Detection:

Delta detection can be performed by comparing entire data sets or by using date columns. Last modified delta detection is the most efficient because Informatica MDM Hub can simply compare the last modified column of each incoming record to the record's previous last modified date.

With delta detection, you have the option of not including columns in the landing table that are associated with the last_update_date column in the staging table in delta detection. If you include the last_update_date column when you configure delta discovery, and only the last_update_date column changes, the MDM hub will perform a lot of unnecessary delta discovery and loading work.

If you process records by last modified date, do not use the Purge Now feature to compare last modified values ​​(for example, whether the source record's last modified date is earlier than the current system date). to test). Using Now in this manner can produce unexpected results.

Perform delta detection only on columns in the source whose last modified date is not an indicator of actual change. The Informatica MDM Hub stage job compares the entire source record to the latest corresponding record (previous load) in the PRL table. If the cells are different, the records are transferred to a staging table. Delta detection is done using PRL tables.

If delta detection is based on a date column (last_update_date, a system-defined date column, or a custom date column such as date of birth), only records with new date values ​​in the date column (compared to their corresponding records) The maximum date from the RAW table (instead of the PRL table) is displayed and inserted into the staging table.

If delta detection is based on a specific column, including a date column (such as DOB or last_update_date), a record will be detected every time the date in the specified column changes (not the maximum date, but the corresponding value in the PRL table).). in the RAW table) is inserted into the staging table.

If you check the delta of all columns during delta detection, only records with null primary key will be rejected. This is expected behaviour. All other records that fail the delta process are rejected by subsequent processes.

If delta detection is based on last modified date, changes to the last modified date or primary key are detected. Updates to values ​​that do not correspond to the last modified date or that are part of a concatenated primary key are not recognized. When using delta detection with mapped columns, duplicate primary keys are not considered in subsequent processes.

Rejection processing allows you to:

- View all rejected records for a specific staging table related to a batch job

- View all rejected records by day across all staging tables

- Query rejection table based on query filters

Delta Detection Enable for a Staging Table:

Start the Schema Manager.

Selection of the staging table that has be to configure.

Acquire a write lock.

Check the check box to enable delta detection for the table. To see this option, scroll down.

Specify how delta is detected.