Author: Ryan Van Fleet
13 Feb, 2020
Hacking Your Life with Task Types
I want to share an idea that has changed my life by mitigating burnout and feeling more fulfilled day-to-day, month-to-month, and year-to-year. The idea is task typing. In my next two blog posts, I will introduce you to the idea and show you a few ways to use it...
28 Jan, 2020
4 Ways to Learn Object-Oriented Design
Looking back over 2019 I realized I’ve had the privilege to work on a diverse set of software projects: Architecting and coding an Enterprise Transform and Load (ETL) project in Python and Django Building an SFTP platform using AWS lambda, Node, and Vue Adding major features and refactoring a...
14 Jan, 2020
Finding Diacritics in a Postgres Table
Recently I had the task to find all the Diacritics in a database table. I had to find an easy way to select every row that has a diacritic in the name. The solution was surprisingly simple, but a bit unintuitive. In this article, I will provide you with...
03 Dec, 2019
Kitchen Sink: Audit/Governance/Work Queue – Extract Transform and Load Part 6
In the previous articles, we discussed the main pieces to consider when designing and implementing an Extract Transform and Load pipeline. Submission Information Packages and High-level Validation: Data, End Points, and validation Canonical Transformation: Converting data to a common object base Business Rules: Low-level validation Identification: Does this data...
12 Nov, 2019
Basic Rules Engine Design Pattern
In today’s posts, I want to share with you one of my favorite design patterns. I’ve used this design pattern in countless situations in multiple different languages, but it is the most useful when creating and managing rules. In the following simplified scenario, I want to validate a Recording...
03 Oct, 2019
How to Debug Node Serverless Using JetBrains WebStorm
One of the most useful tools in a developer’s quiver is the debugger. The debugger allows a developer to not only step through code and track down bugs, but it is useful as a way to profile data structures. I find the ability to profile data structures to be...
11 Jul, 2019
Data Identification and Persistence – Extract Transform and Load Part 5
The final two steps, identification and persistence, can make or break the data chain. Identification handles the deduplication of data and informs the system of it should be persistence. Understanding how data will be used by the system informs if the data is Single, Submitter, Transaction Source. Once data...
09 Apr, 2019
Business Rules – Extract Transform and Load Part 4
In the last article of the Extract Transform and Load (ETL) series, we talked about how using a canonical object can decouple data sources from the downstream components, making the pipeline reusable and easily maintained. In this article we will discuss one of these downstream components, the business rules....
19 Mar, 2019
Reusability and Sustainability Using a Canonical – Enterprise Transform and Load Part 3
Overview In the last article in my Extract Transform and Load (ETL) series, Submission Information Packages and High-level Validation, I covered the most important part of this process: the data. The data format will drive 80 percent of the design decisions for the ETL pipeline. In this article, I...
26 Feb, 2019
Extract Transform and Load Part 2 – Submission Information Packages and High-level Validation
The first two major pieces of an Extract Transform and Load pipeline are the Submission Information Package (SIP) and High-level validation. The major components as well as the benefits and usages of Extract Transform and Load where discussed in the first article in the series. It is recommended that...