Managed Airflow, Airflow 3 & Migration

Two decisions close this track. Should your team run Airflow itself, or pay a company to run it? And how do you move from Airflow 2 to Airflow 3?

You follow the same DAG as before: shop_orders_daily. You also meet a made-up platform with 40 DAGs, the same company as in Executors Compared: Local, Celery, Kubernetes. Every number about it is made up.

Airflow 3 came out in April 2025. Services and versions change fast, so check the provider docs and the release notes for the latest facts. You answer four questions:

1. What does a managed service run for you, and what stays yours?
2. How do you count the real cost?
3. How do you plan the move to Airflow 3?
4. Which changes break loudly, and which change data without an error?

Managed Airflow means a company runs the Airflow parts for you. You still write the DAGs.

A bucket is a named place for files in cloud storage. Three managed services are common. Look at how each one receives your DAG files:

The DAG file is the same in all four cases. Only the way you ship it changes. Look at one DAG on three services:

# Amazon MWAA: copy the file to the DAGs folder in S3
aws s3 cp shop_orders_daily.py s3://shop-demo-airflow/dags/

# Google Cloud Composer: import the file to the environment bucket
gcloud composer environments storage dags import \
  --environment shop-demo --location europe-west1 \
  --source shop_orders_daily.py

# Astronomer: build the project image and deploy it
astro deploy

Now the split of work. Look at who owns each job under a managed service:

Managed services also have limits. Check each one in the provider docs, because they change often:

• Versions: a service often offers a new Airflow release months after Airflow ships it.
• Plugins and settings: some settings and custom code may be limited.
• Network: reaching a private database needs network set-up on your side.

A price list shows servers for self-hosting and a fee for managed. It leaves out the hours your team spends.

Think of the made-up platform with 40 DAGs. A platform engineer keeps Airflow running for the other teams. The hours below are made up, per year.

Managed still needs 24 hours for upgrades. You test your 40 DAGs on each new version, even when the provider does the upgrade.

Now compare in words. Self-hosting costs servers plus 300 hours. Managed costs a service fee plus 42 hours.

Ask what 258 hours of your team are worth.

Measure your real hours for one month before you decide. A guess made on day one can be far off.

Airflow 3 is the biggest change since Airflow 2.0. A migration is the move of a running system to a new version.

You met the Airflow 3 changes in Airflow Architecture: Scheduler, Executor, Workers, Metadata DB. The API server replaces the web server, and the DAG processor runs alone. A task no longer reads the metadata database.

This section is the plan to get there.

Follow these eight steps in order:

1. Inventory: list every DAG, plugin, custom operator, setting and script that calls the API.
2. Bridge: you can upgrade to Airflow 3 only from Airflow 2.7 or later. The upgrade guide advises the latest 2.x release first.
3. Fix on 2.x: remove deprecation warnings (notes that a feature will go away) and run the ruff checks (next section).
4. Back up the metadata database. Run airflow db clean first, so the database is smaller.
5. Stage: upgrade a copy of the platform and run real DAGs on it.
6. Run both: keep Airflow 2 running and move DAGs to Airflow 3 in waves.
7. Cut over: when the last wave works, switch off Airflow 2 for good.
8. Rollback plan: keep the old environment and the backup for a set time.

On Amazon MWAA and Google Cloud Composer, you cannot upgrade Airflow 2 to Airflow 3 in place. You create a new Airflow 3 environment and move DAGs to it.

Airflow 3 also records DAG versions. The UI shows which version of a DAG each run used.

Look at the waves for the made-up platform. It has 40 DAGs, so you move them in three groups:

Each DAG lives in one place at a time. Pause it on Airflow 2 before you turn it on in Airflow 3.

The rollback plan matters because you should treat the database change as one way. You go back by switching to the old environment, not by undoing the database change. Check the release notes for what your version supports.

Do not search your DAGs by hand. Ruff is a Python checker, and it has rules for Airflow. Airflow also checks your settings.

Look at the commands, then at an Airflow 2 file. Use ruff 0.13.1 or newer, as the upgrade guide says:

# Find code that Airflow 3 changes
ruff check dags/ --select AIR3

# Report settings that Airflow 3 changes
airflow config lint

# Show the setting changes, or apply them
airflow config update
airflow config update --fix

Now the file ruff reads. It is shop_orders_daily written for Airflow 2:

from datetime import datetime
from airflow.decorators import dag, task
from airflow.datasets import Dataset

orders_table = Dataset("postgres://shop/orders")

@dag(schedule_interval="0 0 * * *", start_date=datetime(2026, 1, 1))
def shop_orders_daily():
    @task(outlets=[orders_table])
    def load_orders(**context):
        day = context["execution_date"].date()
        print(f"loading {day}")

    load_orders()

shop_orders_daily()

Line 7 has a second problem that no tool reports. The next sections explain it. Look at the file after the fixes:

from datetime import datetime
from airflow.sdk import dag, task, Asset

orders_table = Asset("postgres://shop/orders")

@dag(schedule="0 0 * * *", start_date=datetime(2026, 1, 1), catchup=False)
def shop_orders_daily():
    @task(outlets=[orders_table])
    def load_orders(**context):
        day = context["logical_date"].date()
        print(f"loading {day}")

    load_orders()

shop_orders_daily()

A timetable is the rule that turns a schedule into runs. Read the changes, and do not trust them blindly.

In this file, logical_date does not mean the same day under the new default timetable. Two Silent Changes shows why.

A removed feature fails loudly. Your DAG file stops importing, and you see the error at once.

Look at what to search for. Each row has a replacement, and most replacements come from earlier chapters:

If a task counts earlier runs through the database, rewrite it to call the API. Also check scripts that call the old REST API path. Airflow 3 uses a new one.

Start-up scripts and health checks also change. A script that starts the webserver must now start the API server and the DAG processor.

Some changes run without any error. The DAG imports and the task turns green, but the data is different.

A tick is a moment when the schedule fires.

The first change is the timetable. A plain cron string used to make one run for each full day. In Airflow 3, it makes a run at the tick, and the start equals the end.

Use the six shop orders from Scheduling, Data Intervals & the Logical Date Trap. Look at the paid revenue query for the tick 2026-01-11 00:00:

SELECT DATE '{{ data_interval_start | ds }}' AS revenue_date,
       COALESCE(SUM(amount), 0) AS revenue
FROM   orders
WHERE  status = 'paid'
  AND  order_date >= '{{ data_interval_start | ds }}'
  AND  order_date <  '{{ data_interval_end | ds }}';

The Airflow 3 run is green, but it built the wrong day. Its 0.00 looks right only because order 103 is refunded. Order 102 on 2026-01-10 is never counted.

You have two fixes. Set create_cron_data_intervals = True in the scheduler settings to keep the old way.

Or pass the timetable in the DAG, as in Scheduling, Data Intervals & the Logical Date Trap. Look at the schedule line:

from airflow.timetables.interval import CronDataIntervalTimetable

@dag(
    schedule=CronDataIntervalTimetable("0 0 * * *", timezone="UTC"),
    start_date=datetime(2026, 1, 1),
    catchup=False,
)

The second change is catchup. Catchup decides whether Airflow builds the days that are already over. Its default was True in Airflow 2 and is False in Airflow 3.

Take a new DAG with start_date 2026-01-01, deployed on 2026-01-11, with no catchup setting. Airflow 2 builds the 10 old days, 2026-01-01 to 2026-01-10. Airflow 3 does not build them.

So a DAG that built its own history may now start with none. Set catchup on every DAG, so no default decides for you.

Most Airflow articles describe Airflow 2, and some describe Airflow 1. You can often date an article by its words.

Use the table in What Airflow 3 Removed as a check list. If an article uses a word from its left column, it was written for Airflow 2 or older.

An article about SubDAGs was written for an old Airflow. Treat the rest of its advice with care.

Upgrade in small steps. If you wait for five releases, the next upgrade becomes a project, and a security notice often forces it.

Run the ruff Airflow rules in your continuous integration (CI) checks all the time, not only on upgrade day. They show new deprecations while you have time to fix them.

Managed Airflow takes away the Airflow parts. Your DAGs, packages, secrets, alerts and bill stay yours.

Look at the answers you should know:

Try this alone. After an upgrade, a green DAG built 0.00 revenue for a day that had paid orders. What do you check first?

The answer is the timetable. Under the new default, the data interval start equals its end, so the window is empty.