We Ported an Airflow Pipeline to Kestra: Field Report

Why we ran this experiment

We have run Apache Airflow in production for several clients for years. Airflow is the market reference for orchestration — the foundation of most serious data platforms, the tool teams know, the one that has proven itself.

For a few months now, one question has kept coming back, both at our clients and in hallway conversations: what is Kestra actually worth? Kestra is a younger open-source orchestration platform, often pitched as "declarative Airflow" — YAML instead of Python, a more modern developer experience, native cloud integrations. To answer with more than an opinion, we ported a real production pipeline onto Kestra, alongside production, on the same cluster, running the same application binaries.

Here is what we learned. Not a benchmark, not an install guide — a field report aimed at any team or leadership team wondering whether Kestra is worth evaluating, and in which cases.

During the first hours of exploration, one phrase kept coming back: "Kestra is to Airflow what Terraform is to Ansible". More declarative, more modern in its iteration cycle, more oriented to cloud-native. The analogy has its limits, but it frames the shift we observed.

When Kestra makes sense, when Airflow is still preferable

Before we get into the build, let's set the scene. From what we observed, the choice between the two depends heavily on context:

Context	Our reading after the experiment
New project, team to onboard, need to iterate fast	Kestra has a real edge
Stable Airflow platform, trained team, custom patterns already built	Airflow stays preferable (existing capital)
Simple declarative pipelines with many standard integrations	Kestra (broad native plugin ecosystem)
Pipelines with complex application logic (ML, advanced transforms)	Airflow (native Python, more powerful)
Strong multi-tenancy, audit, broad governance expected	Airflow (broader community, multi-vendor commercial ecosystem)

Our goal isn't to settle the question in the abstract — it's to know what we're doing in each case. The rest of this article tells the concrete story of the build that validates that reading.

Step 1 — Real-world testing without touching production

The first constraint we set ourselves: don't build a separate "toy" test environment. We wanted to reproduce the real conditions of a potential client deployment, not a showroom demo. The method:

• Kestra deployed on the same cluster as production Airflow.
• Isolation by data only: dedicated database and object storage for the test environment.
• Zero modification of production application code. The same binaries run on both sides — Kestra orchestrates exactly what Airflow orchestrates, with different environment variables.
• Pure open-source edition assumed. No Enterprise features in scope. What we deploy is what we can redeploy at a client without commercial dependency.

This method is reproducible at any client who wants to evaluate an orchestrator without risk to production. It's what makes the comparison meaningful — the same data flows through two different orchestrators, and we look at what comes out.

Compared architecture

One of the most immediately visible differences is the number of components to operate:

• On the Airflow side, several services share responsibilities: a scheduler, an API server, a definition processor, an event trigger handler. Each runs in parallel and needs to be supervised.
• On the Kestra side, in standalone edition, a single service covers all of those roles.

On this specific point we can be factual without extrapolating: fewer moving parts to monitor and upgrade — that's observable. What we cannot claim at this stage — and we own it — is "Kestra uses fewer resources": the load conditions of the two environments were not equivalent.

Step 2 — The hard fact: Kestra produced the same data as Airflow

The build was pointless if Kestra delivered different results. So we framed a rigorous validation: on the same scope, over several independent days, we compare row-for-row what both orchestrators deliver at the end of the pipeline. The outputs are strictly identical. Downstream business checks — distributions, totals, aggregates used by end users — return the same results on both sides.

That's what we can claim without nuance after this build: on this scope, Kestra produces the same deliverable as Airflow. The transition is technically feasible.

What the validation did not settle. Raw performance under real multi-client load, operational cost at scale, stability over several months. We didn't honestly test those dimensions. At equivalent scope on the subset we were able to measure, Airflow keeps a slight execution-time edge; the sample is too small to derive a reliable number from, so we don't quantify it.

Step 3 — A developer experience that changes the daily routine

Beyond the deliverable, what counts for a team day-to-day is the time it loses waiting on its tools. On this specific point, Kestra has a clear edge.

Editing a pipeline on the Airflow side goes through a standard Git cycle — commit, push, wait for the sync, wait for re-parsing. On the Kestra side, you edit, you push, the pipeline is available almost instantly. Over an afternoon of exploratory writing, that's ten to fifteen extra round-trips possible. In a 3 a.m. debugging session, it's tangible.

The supervision UI is also more modern — live log streaming, log search, dependency visualization. Not a structuring argument on its own, but cumulatively, it makes a real difference to operational comfort.

Keep in mind: in production, deployment stays disciplined by GitOps on both sides. The Kestra advantage shows up mainly during exploration and day-to-day operations.

Step 4 — The daily match

Beyond the deliverable and the iteration cycle, we reviewed the operations a data team performs every week. In summary:

• Re-running a job that failed overnight: more direct on the Kestra side (an explicit "Restart" button, where Airflow uses a less intuitive term for newcomers).
• Diagnosing a failure: advantage Kestra (live logs, native full-text search).
• Rotating an application secret: advantage Kestra (native alignment with the cluster's security mechanisms rather than an orchestrator-internal database).
• Notifying a team (Slack, Teams, email…): advantage Kestra (very broad catalog of native plugins).
• Onboarding a new team member: advantage Kestra on the first steps (YAML is more accessible than Python for beginners). Advantage Airflow as soon as the logic becomes complex (advanced transforms, ML).
• Testing and industrializing the code: advantage Airflow (Python = native unit testing, mature ecosystem).

In summary

Criterion	Verdict
Equivalence of produced data	Strictly identical
Developer experience, iteration cycle	Kestra
Community maturity and governance	Airflow
Number of components to operate	Kestra (very compact standalone edition)
Code testability and industrialization	Airflow
Native cloud integration catalog	Kestra
Junior team onboarding	Kestra
Reversibility and multi-vendor ecosystem	Airflow

What we take away about Kestra

On this build, Kestra delivered the same output as Airflow. That's the central argument, and it removes the uncertainty for anyone wondering.

Beyond that hard fact, the dominant impression after several days of exploration is that of a promising platform. More modern in its iteration cycle, more direct in its relationship with Kubernetes, more accessible for a team starting out. On certain day-to-day operations, the gap with Airflow is clearly in Kestra's favor.

But it's also a younger platform whose ecosystem, community, and operating patterns are still maturing. Today, for a client running a stable Airflow production with a trained team, the return on investment of a migration doesn't justify itself by default. The migration cost outweighs the short-term benefit.

Our conviction after this build: Kestra has everything it needs to become a reference player in open-source orchestration in the coming years. We're looking forward to seeing its ecosystem densify, its large-scale operating patterns stabilize, and its community grow. As these elements fall into place, the choice between the two will become less and less obvious — and more and more context-dependent.

Until then, Datakhi continues to operate Airflow in production for existing clients, and keeps Kestra in active watch for greenfield projects, teams in training, and contexts where its native plugin catalog brings real immediate value. We now know how to deploy it cleanly and steer clear of the usual frictions of a platform under active evolution.

Are you evaluating Kestra, Airflow, or another orchestrator? Do you want to test an orchestrator alongside your production without risk, and see what it gives on your real data? Datakhi has ported a real Airflow pipeline to Kestra and structured the method in reproducible steps. Let's talk.

Further reading

• Our previous field report on the same infrastructure: "From MinIO to Garage: benchmark, migration and production feedback"
• Kestra website: kestra.io
• Airflow documentation: airflow.apache.org