Apache Airflow
-
IBM Data Engineering Capstone Project
In this IBM sponsored project, I assumed the role of a Junior Data Engineer who has recently joined a fictional online e-Commerce company named SoftCart. Presented with real-world use cases, I was required to apply a number of industry standard data engineering solutions.
-
LAB: Watch me write an Airflow DAG pipeline that analyzes web server log files
In this lab, I was required to write an Airflow DAG pipeline that analyzes web server log files, extracts the required lines and fields, transforms and loads the data to a compressed file.