Monitor Execution Performance
Learning Objectives
After completing this unit, you'll be able to:
- Navigate the IDMC Monitor service to track job histories and execution performance.
- Interpret task status indicators across Mapping Tasks, Synchronization Tasks, and Taskflows.
- Analyze Session Logs to diagnose pipeline failures.
- Use the Error Rows File to identify records that fail target validation.
The Monitor Service
Cassie's scheduled taskflows now automate data movement across AW Computing. However, production environments inevitably encounter operational disruptions—network timeouts, schema changes, expired API credentials, or malformed source data. When pipelines fail, Cassie relies on the Monitor service.
The Monitor service provides a centralized, real-time control tower for every job run across the organization—including Mapping Tasks, Synchronization Tasks, Replication Tasks, Data Transfer Tasks, and Taskflows.
Navigate the Monitor Service
When Cassie opens the Monitor service, she sees a job run list. She moves to the All Jobs tab which displays a chronological execution history of all recent runs and reviews the jobs that have run.
She can use filters to help narrow the view to the runs she needs.
-
Time range: Filter runs from the last hour, day, week, or a custom date range.
-
Status: Narrow the view to Success, Failed, Warning, Running, or Stopped.
-
Task type: Isolate Mapping Tasks, Synchronization Tasks, Taskflows, or Replication Tasks.
-
Project/folder: Filter execution runs by specific project workspace folders.
She can also search by task name and other options.
Understanding Status Indicators
Status Indicator |
Meaning |
Required Action |
|---|---|---|
Success |
The task completed cleanly and all rows were processed without errors. |
None—job completed as expected. |
Warning |
The task completed, but non-fatal errors occurred or some rows were skipped. |
Analyze the session log, and if it's a synchronization task, review the Error Rows File for rejected rows. |
Failed |
The task stopped unexpectedly before completing due to a fatal error. |
Open the Session Log to diagnose the root cause. |
Running |
The task is currently executing on the assigned Secure Agent. |
Monitor progress in real time. |
Stopped |
The task was manually halted by an administrator before completion. |
Verify why the job was terminated before re-triggering. |
Reading the Session Log
When a task displays Failed or Warning, Cassie clicks the job run record to inspect the diagnostic details. CDI provides two distinct diagnostic tools.
-
Session log: A detailed, timestamped execution log recording database connection attempts, SQL queries, transformation processing steps, and engine error codes.
-
Error rows file: A downloadable .csv file containing the exact data records that failed validation, along with the specific field-level error reason.
From these diagnostics, Cassie identifies two root causes. When a Synchronization Task fails with a Salesforce API error, the session log shows:
ERROR: INVALID_FIELD_FOR_INSERT_UPDATE: Field 'ExternalID__c' is not accessible on Contact
- This is a system permission issue: The integration user profile lacks write access to the Salesforce ExternalID__c field.
Alternatively, the Error Rows File lists the records along with the error:
- This is a data quality issue: Malformed record syntax caused 42 rows to fail target validation.
Resolving Issues and Rerunning Tasks
Once Cassie corrects the field permissions in Salesforce and asks the data steward to fix the malformed email address at the source, she doesn't need to rebuild or reconfigure her pipeline. She simply navigates back to the Monitor service, locates the failed run, and clicks Rerun:
- IDMC resubmits the job to the Secure Agent—the on-premises component that executes your jobs using the same configuration.
- The Secure Agent reevaluates the source data, processes all records cleanly, and updates the status. If the fix was correct, the rerun shows Succeeded. If new errors surface, the session log updates with fresh detail.
Knowing how to read a session log and act on it—fix, rerun, verify—is the core Monitor workflow Cassie uses every day in production.
Wrap It Up
Cassie has officially built an enterprise-grade, fully automated data integration platform for AW Computing.
-
Mappings and mapping tasks: Define data integration processes and run them on demand or on a schedule.
-
Reusable mapplets: Enforce logic consistency across mappings.
-
Router transformations: Split data streams in a single pass.
-
Taskflows: Orchestrate parallel and sequential execution.
-
Monitor service: Provide real-time visibility and error recovery.
The fragile, manual CSV spreadsheets and copy-paste processes that once consumed Cassie's workdays are gone. Her team has full visibility into every data movement—what ran, when, how many rows, and why anything failed. In their place stands a clean, scannable, and self-maintaining integration architecture that delivers accurate, timely data to AW Computing's business leaders every single day.
Resources