Tag: Enterprise transform and load
Ryan Van Fleet
Ryan Van Fleet
03 Dec, 2019

Kitchen Sink: Audit/Governance/Work Queue – Extract Transform and Load Part 6

In the previous articles, we discussed the main pieces to consider when designing and implementing an Extract Transform and Load pipeline. Submission Information Packages and High-level Validation: Data, End Points, and validation Canonical Transformation: Converting data to a common object base Business Rules: Low-level validation Identification: Does this data...
Ryan Van Fleet
Ryan Van Fleet
11 Jul, 2019

Data Identification and Persistence – Extract Transform and Load Part 5

The final two steps, identification and persistence, can make or break the data chain. Identification handles the deduplication of data and informs the system of it should be persistence. Understanding how data will be used by the system informs if the data is Single, Submitter, Transaction Source. Once data...
Ryan Van Fleet
Ryan Van Fleet
18 Dec, 2018

Extract Transform and Load Overview

Part 1 of the Data Ingest Series The process of extract transforms and load (ETL) goes by many names: Data acquisition Data ingest Enterprise transform and load But they all are about getting external data into the system. The problem that most businesses face is that there are no...