Notifications

247 views

Description

Data sources with import errors with loading the data (before the transformation), do not shown errors on the Import set nor transform history but hidden on the Import Log and will not indicate if additional lines were skipped.

Administrators should get an error status when the import fails on a schedule job. However, when an import is scheduled, if the data is loaded with errors, the data transformation gets started and no warnings are thrown after that even if some records were skipped.

 

Steps to Reproduce

1. Create a Data Source, to import a CSV file with 5 rows, and on line 5, add an extra quote.

e.g.

"TEST1","TEST2","TEST3"
"1","2","3"
"2","2","3"
"3","2","3"
"4","2","""
"5","2","3"
"6","2","3"

Note the line starting with 4, it has a extra quote ("). This file is attached here: test_error.csv

Save the data source.

2. Create a Transformation Map to incident and map TEST1 = Short Description, TEST2=Work-notes, TEST3=Additional comments.

3. Create a Scheduled Job that runs the data source and execute the Scheduled job.
Data source and Schedule

You will notice the import set shows GREEN processed, and it shows 3 records imported, and transformation history shows "total: 3, inserts 0, updates 3, ignored 0, skipped 0, errors 0". Transformation history shows NO errors.

However, the error when loading the data source is NOT shown.
Transformation history

Workaround

Keep in mind the source data count, and validate the same amount is loaded on the staging table manually. Finally, always review the import logs for any error messages related to data load from the import executed. Errors related to loading the data does not show on the Import set, nor the transformation history.

Import log 


Related Problem: PRB1242566

Seen In

There is no data to report.

Associated Community Threads

There is no data to report.

Article Information

Last Updated:2018-03-02 13:37:52
Published:2018-02-16