28 views

Import sets do not show loading data errors

Problem

Import set shows several messages and information related to the data imported and transformed. However, problems when loading the data are stored on a hidden form called Import logs (import_log table). However, these message might not indicate whether some records were skipped as a consequence of the error.

Symptoms

You will notice some data missing on your staging table. However:

  • The import set shows no errors, no skipped records, and it could show as Proceeded.
  • The Transform history shows no errors.
  • When manually counting the source data records (for example, on the Excel document), it shows more records than the ones available on the staging table.
Cause

When the data source is executed, a loader routine is executed to create and/or load the staging table. If an error happens at this time, it will be loaded on the Import Logs (import_log table) and will not be passed to the Import set for recording.

Resolution
For your initial testings, keep in mind the source data amount of records, and validate that the same count is loaded on the staging table by comparing them manually. Finally, always review the import logs for any error messages related to data load from the import executed. Errors related to loading the data does not show on the import set, nor the transformation history.

 Import log

Note: Always review the import logs for any error messages related to data load from the import executed. When using the "Load all records" on the Data source, these messages will also appear on the interface.

 

Article Information

Last Updated:2018-01-15 22:43:34
Published:2018-01-16