Prateek Shrivastava

Data Engineers and Enterprises continue to struggle with Stream Processing at scale. In our extensive discussions with customers and partners, we repeatedly found the following…

What’s keeping streaming data processing investments from yielding “speedy” results? There are multiple streaming data processing solutions out there but none are well equipped to…

How to Install Apache Airflow to Run Different Executors Now that we know about Airflow’s different components and how they interact, let’s start with setting…

Airflow Architecture diagram for Celery Executor-based Configuration Airflow Architecture Diagram Before we start using Apache Airflow to build and manage pipelines, it is important to…

This is an abridged version of the article that appears on NewStack BI tools have been the go-to for data analysts who help businesses track…

What is Apache Airflow? Apache Airflow is one of the most powerful platforms used by Data Engineers for orchestrating workflows. Airflow was already gaining momentum…

Key considerations for building a scalable transactional data lake Data-driven companies are driving rapid business transformation with cloud data lakes. Cloud data lakes are enabling…