Devjyoti Patra is a staff engineer in the data engineering team at Qubole. In this talk, he explains how Qubole is using Airflow for operationalizing complex data pipeline with data quality checks & Qubole operator. These tools along with CI/CD CLI tool (https://github.com/qubole/afctl) are opensourced by Qubole.
Free access to Qubole for 30 days to build data pipelines, bring machine learning to production, and analyze any data type from any data source.
See what our Open Data Lake Platform can do for you in 35 minutes.