Airflow Interview Questions for Data Engineers

01

The 5 kinds of Airflow question

Cut by what a question asks you to DO, not by the topic it names. The same topic scores differently when you have to explain it, build it, or work out why it broke — and those three fail separately.

• 20ConceptualThe model underneath the UI. What the scheduler is doing between runs, why a daily DAG for Monday runs on Tuesday, and what an executor actually decides.DAGs, tasks, operators · Scheduler & executors (Local / Celery / Kubernetes) · logical_date vs execution_date · Catchup & backfill→
• 20DAG AuthoringWrite the DAG. Dependencies, branching and a fan-out whose width is only known at runtime — plus the discipline that keeps a task idempotent so a retry is free.TaskFlow API · Dynamic task mapping · Trigger rules & branching · Idempotency→
• 20Debug & FixStuck in queued. Not triggering. Ran twice. Skipped when it should not have. Each one has a small set of real causes, and knowing them is the difference between reading logs for an hour and fixing it in five minutes.Pools, priority weights, concurrency limits · Trigger rules & branching · Catchup & backfill · Retries, alerting & SLAs→
• 22Operational ScenariosBackfill two years without starving the daily runs. Rotate a credential without editing every DAG. Rerun one task in the middle of a finished run. The questions somebody who has been on call can answer immediately.Catchup & backfill · Connections, Variables & secrets backends · Retries, alerting & SLAs · Datasets / data-aware scheduling→
• 20Scaling & ConcurrencyTwo hundred sensors holding every worker slot. Where the limits actually are — pools, DAG and task concurrency, parallelism, executor choice — and which one to move when the queue stops draining.Sensors, reschedule mode & deferrable operators · Pools, priority weights, concurrency limits · Scheduler & executors (Local / Celery / Kubernetes) · Airflow 2 vs 3→

02

The ones that keep coming up

Answered here, in short. The full reasoning, the wrong answer and the follow-up are one click away.

1. 01Do you actually need Airflow? Compare it to a managed workflow service like AWS Glue Workflows or Step Functions, and say when you would not run it.Airflow earns its place when you orchestrate across systems, need real backfill, and have someone to operate it. Five Glue jobs with no cross-system dependency do not need a scheduler and a metadata database.→
2. 02Two critical pipelines fail within 5 minutes of each other and you are on call alone. How do you decide which one to work first?Rank on impact and reversibility, not on which alert fired first. Check whether one caused the other, communicate both, then work the one where delay does the most irreversible damage.→

03

What Airflow interviews actually cover

• 01DAGs, tasks, operators
• 02Scheduler & executors (Local / Celery / Kubernetes)
• 03logical_date vs execution_date
• 04Catchup & backfill
• 05Sensors, reschedule mode & deferrable operators
• 06XComs & data passing
• 07TaskFlow API
• 08Dynamic task mapping
• 09Trigger rules & branching
• 10Pools, priority weights, concurrency limits
• 11Retries, alerting & SLAs
• 12Connections, Variables & secrets backends
• 13Idempotency
• 14Custom operators & hooks
• 15Datasets / data-aware scheduling
• 16Airflow 2 vs 3
• 17Testing DAGs

04

Learn it, then practise it