Author: Ryan Van Fleet
11 Jul, 2019
Data Identification and Persistence – Extract Transform and Load Part 5
The final two steps, identification and persistence, can make or break the data chain. Identification handles the deduplication of data and informs the system of it should be persistence. Understanding how data will be used by the system informs if the data is Single, Submitter, Transaction Source. Once data...
09 Apr, 2019
Business Rules – Extract Transform and Load Part 4
In the last article of the Extract Transform and Load (ETL) series, we talked about how using a canonical object can decouple data sources from the downstream components, making the pipeline reusable and easily maintained. In this article we will discuss one of these downstream components, the business rules....
19 Mar, 2019
Reusability and Sustainability Using a Canonical – Enterprise Transform and Load Part 3
Overview In the last article in my Extract Transform and Load (ETL) series, Submission Information Packages and High-level Validation, I covered the most important part of this process: the data. The data format will drive 80 percent of the design decisions for the ETL pipeline. In this article, I...
26 Feb, 2019
Extract Transform and Load Part 2 – Submission Information Packages and High-level Validation
The first two major pieces of an Extract Transform and Load pipeline are the Submission Information Package (SIP) and High-level validation. The major components as well as the benefits and usages of Extract Transform and Load where discussed in the first article in the series. It is recommended that...
18 Dec, 2018
Extract Transform and Load Overview
Part 1 of the Data Ingest Series The process of extract transforms and load (ETL) goes by many names: Data acquisition Data ingest Enterprise transform and load But they all are about getting external data into the system. The problem that most businesses face is that there are no...
01 Aug, 2018
Build a Responsive Navbar Using Angular 5 and Bootstrap 4
A UI that is responsive to device and browser size is critical to provide usable access to your website and services. One of the most important parts of the UI is the navigation bar (navbar), which allows users to easily find and access information. The good news is that...
06 Jun, 2018
XML Schema Definition:
A Definitive Quick Guide
The XML Schema Definition is a dense document that will take you a lot of time to parse to get to the heart of what you’re after. Even if you’ve worked with XML and XSDs for years, you may still need to drill down into it in order to...
10 Apr, 2018
Spring Boot JPA Relationship Quick Guide
This week we return to a more technical topic, Spring Boot JPA, and unwind its complexities into practical examples of common persistence patterns. Spring Boot JPA has a ton of documentation, references, and articles. That’s a lot of material to sort through, so I created this Spring Boot JPA...
02 Aug, 2017
The Compressed Java Zip Solution
Recently, I had a requirement on a project to systematically zip up a directory of reports to be stored on AWS S3 and later downloaded by the user. I thought it can’t be too hard to zip files using Java with the expectation that the APIs would reflect the...
20 Apr, 2017
Taming of the Subselect: Three ways to use this troublesome SQL Query
Subselects can be used in many ways: filtering, retrieving specific data, and creating custom joins. Writing effective SQL using subselects, though, can be frustrating. While SQL is ubiquitous in software engineering and is a valuable skill for testing, data analysis and reporting, nowadays a lot of SQL is abstracted...