PrepareSynchronizationTask

View Behavior Details

The task reads files from the source directory, usually the temp folder > ExportOutput folder.

Cleanse data

The following actions are performed on the CSV source files:

1. Remove columns that are not used in <EntityTypeMapping> or <EntityAssociationMapping>.
2. Remove entries that have a null primary key.
3. Remove duplicates.
4. Sort entries according to the primary key.

The result of the Prepare-Synchronization is stored in the export directory as three files:

• For every entity type of the relevant Connector involved in an <EntityTypeMapping> or an <EntityAssociationMapping> , a .sorted.csv file is generated, containing the final, cleansed and sorted result.
• Duplicates are kept in a separate .duplicates.csv file.
• Null primary key entries are kept in a separate .nullpk.csv file.

All files produced by the task are in the work folder > Collect directory.

Compute changes

In incremental mode, changes might need to be computed by the Agent:

• If the Export step has provided computed changes, no further process is required. The changes will be sent as-is to the server.
• If the Export step has provided a full extract of the managed systems, the Prepare-Synchronization step computes changes. This computation is based on the result of the last data cleansing, generated by the previous Prepare-Synchronization, and stored in the previous folder in the export directory.

For incremental mode, it is recommended, whenever possible, to use managed systems to compute changes. Dedicated workstations and knowledge of the inner data organization allow managed systems to compute changes with performance that Usercube can't match. Using managed systems for these operations avoids generating heavy files and alleviates Usercube's processing load.

The result is a set of clean lists of changes stored as a .sorted.delta file containing a command column. The command column can take the following values:

• insert
• update
• delete
• merge

These values are instructions for the Synchronization step to apply the changes to the database.

The .sorted file (that is, the original clean export file, not the changes) is stored in the previous folder inside the export directory. It will be used as a reference for the next incremental Prepare-Synchronization to compute the changes, if needed.

Tampering with the previous folder content would result in false changes leading to false computation. It would result in data corruption in the Usercube database. To restore the Usercube database and reflect the managed system data updates, a complete__Sync Up would be required.

Prepare the server

At the beginning of every Prepare-Synchronization process, the Server is notified via HTTP.

Upon receiving the notification, the server creates a directory on its host environment, identified by a unique GUID, to contain .sorted or .sorted.delta files that will be sent by the agent.

This is designed to prevent network errors that would cause an incremental database update to happen more than once.

This means that several Export and Prepare-Synchronization tasks can be executed simultaneously. These tasks will be processed by the server one at a time, in the right order.

Any notification of a complete Prepare-Synchronization would cancel the previous non-processed incremental Prepare-Synchronizations. As a complete Prepare-Synchronization reloads the whole database, it renders incremental changes computation moot.

Send clean exports

.sorted or .sorted.delta files are sent over HTTP to the Server for the last step.

Examples

<PrepareSynchronizationTask
    Identifier="HR_PrepareSynchronization"
    DisplayName_L1="HR PrepareSynchronization"
    OpenIdClient="Job"
    Connector="HR"
    Agent="Local"
    SynchronizationMode="Initial"
/>

Properties

Property	Details
Agent required	Type String Description Identifier of the agent on which the job will be launched. Warning: all jobs containing the task must be launched on the same agent or on the server.
Connector required	Type String Description Identifier of the connector involved in the task.
DisplayName_L1 required	Type String Description Display name of the task in language 1 (up to 16).
OpenIdClient required	Type String Description Connection client for the task.
SynchronizationMode required	Type DataCollectType Description Synchronization mode for collect and synchronization Task. List of Modes: - Initial = 0, - Complete = 1, - Incremental = 2
ColumnName optional	Type String Description If there is a delta in the synchronization, specifies the column name which stores the command
Identifier optional	Type String Description Unique identifier of the task.
Type default value: None	Type PrepareSynchronizationType Description Define the type of PrepareSynchronization to launch the correct executable in job.
WorkingDirectory optional	Type String Description Path of the working directory