When to leave a legacy database, how to weigh a new platform, and what I think the next generation looks like.
Every few years a data team ends up in the same argument. The current platform feels old, a shinier one is on offer, somebody draws a slide with a single logo in the middle and an pointing at it, and before long the whole discussion has narrowed to which platform do we move to. I've sat through enough of these now to be fairly sure the problem is the question rather than any of the answers people give to it.
The question worth asking is rarely "where do we move everything." It's closer to "where does each thing actually belong, and what are we trying to become." Those are two quite different conversations, and running them together is how a team burns a year rebuilding what it already had, this time with a more expensive engine in the basement.
The premise most migrations never examine
Most of these debates carry an unstated assumption: that you pick one platform, move in, and lock the door behind you. That was a reasonable model in the days of monolithic appliances and warehouses, but it has aged badly.
There's no need to settle on a single home. You can let platforms complement each other and optimise per workload according to cost, capability, latency, governance, or plain preference. The centre of gravity has shifted from the database to the data itself: an open table format such as , sitting in cheap object storage like or Storage, with whatever engine you fancy reading from it. The compute comes and goes around the data depending on the job. The vendors are not subtle about this, either. has been pouring engineering into querying Iceberg on demand (it open-sourced its and, by February 2026, made querying its Iceberg tables from any external engine generally available) , and tells much the same open-format story, shipping UniForm so a single copy of Delta data reads as Iceberg , because both have worked out that they aren't going to be the final home for your data, and that holding it isn't where their money comes from.
Once that shift sinks in, the migration question turns into something more useful. You stop asking which platform to surrender to and start asking which engine suits which workload, over storage that you control.
Leaving the old system is easy, which is not the same as wise
The first thing to pin down about any "let's get off the old thing" project is whether you're looking at a like-for-like lift or an actual transformation, because people have a habit of selling the first while picturing the second.
A like-for-like lift (same dialect, same model, new managed home) is often easy and cheap, and I'm all for it when the numbers add up. You stop patching servers and minding your own backups, monitoring improves, spinning up a test environment becomes a button press, and a reservation usually brings the running cost down. Those are real and worth having.
What they are not is a strategy. They reduce the amount of tedious work you do, but they don't make you more capable on their own, and swapping the engine while redeploying the same model just gives you your last decade back with a fresher logo underneath it. The scalability and "future-readiness" everyone hopes a migration will deliver usually has nothing to do with the destination platform. It lives in the work that keeps getting deferred: incremental loading, micro-batching towards something nearer real-time, getting rid of redundant intermediate copies, tightening the modelling, clearing out the cruft that built up because the old environment had a generous batch window and only ever got optimised when something forced the issue. That work pays off whichever platform you end up on, which is precisely why it's worth doing and precisely why it keeps losing its slot in the calendar to the migration.
So the discipline I'd start with is honesty about which kind of project this is. Reducing operational cost and building new capability are both perfectly good reasons to act. The trouble starts when the first gets dressed up as the second.
Weighing a destination: pull the workloads apart from the hype
When a fashionable platform lands on the table, the useful move is to break it down into the specific things it would do for you and price each of them against the alternatives, including the alternative of doing nothing at all. A handful of considerations come up for me every time.
The most expensive engine in your stack (Databricks and Snowflake being the obvious examples) is the wrong place to run routine collection, prep, and load. Do the basal plumbing where compute is cheap and keep the premium engine for work that justifies the cost: advanced analytics, data science, sharing and distribution, on-demand acceleration. Shifting a workload that already runs perfectly well onto premium compute is the kind of thing that feels like progress on a roadmap and delivers none in production.
It's also worth interrogating the headline capabilities until they either hold up or fall apart. A surprising number of the reasons people cite for migrating turn out to be available without migrating at all. You can usually tap into sharing, marketplaces, notebooks, and services simply by making good datasets available to a platform, rather than relocating your entire estate into it. Once you strip out everything you could already have had, the benefit that genuinely depends on the migration is often small. If you can't name it clearly, you have your answer.
Then there's the cost-control burden, which rarely makes it into the business case. A consumption-billed platform can be excellent and still be the wrong call if living with it means constant budget anxiety and pressure to ration the very capacity your users are asking for. Policing spend is itself a cost.
Lock-in deserves a cold look too, especially the quiet operational sort. Parking analytical data somewhere is reversible. Weaving operational logic, transactional tables, and live applications into one vendor is not, and the deeper that integration goes the more it costs to leave and the worse it goes if you ever have to drag something back out.
I'd treat optionality as a feature and refuse to trade it away cheaply.
Finally, I'd discount the marketing, my own enthusiasm included. The major vendors all offer roughly the same spectrum and all sprint towards whatever customers will pay for, so a clever feature on one shows up on the rest within a release or two, and much of what passes for differentiation today is well on its way to being commodity. The economics of the vendor itself are game as well: a company that has spent years and a fortune buying customers without turning a profit will eventually either raise prices or shrink, and when its own stated route to profitability is a layer you can buy from anyone, that says something about how deep the moat really runs.
None of which adds up to "don't adopt the new platform." It adds up to adopting it for the things it's genuinely good at, on your own terms, with your data still portable.
Tooling: automate, or assemble?
Sitting underneath the platform debate is a build-versus-buy one that deserves more attention than it usually gets.
On one side are the integrated, automate-it-for-me tools. Data-warehouse-automation platforms like , and broader managed-pipeline suites like , , and Matillion, generate the boilerplate, work out the dependencies between objects, and orchestrate the load in the right order against whatever engine sits beneath, while still letting you drop in your own code wherever the standard machinery runs out. That dependency-resolution and orchestration is genuinely hard, and easy to take for granted once you've got it. The tools differ in temperament: a tool is essentially a good code generator that owns the dull, correct 80% for you, whereas something like Talend Cloud leans harder into and streaming and treats the cloud platforms as first-class citizens, at the price of being weaker for now on non-CDC loads, file and ingest, and conformed dimensions.
On the other side is the Modern Data Stack: throw five darts at a board and assemble best-of-breed components yourself. Roughly:
- Ingest: , Stitch, or
- Transformation: , , or
- Orchestration: or
- / catalog: or
- Platform: Snowflake or Databricks
It's powerful and fashionable, and it leaves you owning every seam between those five products while still having to write good code yourself.
Where I land: I lean towards the integrated, automated plays (Qlik Talend Cloud today, once it has matured) and away from assembling a stack by hand. The à la carte route is tempting, because picking the sharpest ingest tool, the sharpest transformation tool, and the sharpest and bolting them together sounds like the obvious way to a great result. The catch is that "best per layer" assumes the layers compose cleanly, and they don't. The trouble turns up at the seams: version skew, auth handoffs, a schema that drifts between two tools' assumptions, the failure that belongs to no single vendor because it lives in the gap between them. Five excellent components wired together don't make an excellent system; they make five vendors' worth of integration surface that you now own and operate.
My rule of thumb, then, is to pick one vendor that can supply most of what you need, get the orchestration and dependency-handling as a single coherent thing, and only then reach for one or two specialist extras where the platform genuinely falls short, deliberately and only when the seam each one adds is worth it.
This doesn't contradict staying open at the storage layer; it's the same instinct one floor down. Keep the data engine-agnostic in an open format so no compute platform can hold it hostage, and consolidate the build and orchestration tooling so you aren't the one operating the gaps between a fistful of vendors. Be open where lock-in is expensive and integrated where the seams are.
There's a team-size angle too. A small team that has to deliver safely and predictably has far more to gain from integration than a large team with the headcount to staff every seam, though even setting that aside I'd make the same call. Choose for the people actually doing the work, not for the set of logos that looks most current on a conference slide. And whichever way you go, don't hand-build what's visibly turning into commodity. The frameworks everyone's excited about at the moment are already drifting towards plug-and-play and showing up natively inside the integration tools. There's no medal for lovingly hand-rolling something you'll be able to buy for nothing in eighteen months.
The next generation, as I see it
If I had to compress the whole view into a single instruction, it would be this: don't do anything rash today that makes tomorrow expensive and inflexible. Keep cool. In more concrete terms, a few things matter.
Data should live in an open format in storage you control, independent of any one engine, so you stay rigged to switch engines as the economics and the capabilities move and you never let the data become a hostage. Most of the power a vendor has over you comes from holding your data, so this is the single property that does the most to keep you free.
Compute should run wherever it's cheapest and most appropriate, which sometimes means out of your own hands entirely. Some workloads belong in the centre, some at the edge, and some have to stay on the ground for cost, privacy, or regulatory reasons all at once, like running your own models on sensitive data. And some belong on hardware you don't even own; there's little sense in spending your compute when the user's phone or browser is sitting right there, free to you and very likely more responsive for them.
It's worth planning as though everything you pay a premium for today will be widely available and cheap before long, because most of it will be. Build so that when that happens you can take advantage of it rather than being stranded on a bet you made too early.
The most interesting developments, in my experience, tend not to be the loudest. Microsoft Fabric is the obvious sleeper. It's not production-ready as I write this, and I wouldn't put a workload on it yet, but Microsoft will eat a lot of lunches with it over the next three to five years, with its broad scope, legible pricing, cheap -backed storage in open Delta , and native cross-database queries . The platform players quietly putting together broad, open-storage-backed offerings deserve rather more attention than whoever has the biggest stand at the conference.
Alongside those I keep tabs on the lighter, composable, code-first tools that let you stay nimble without betting the company, the sort you can adopt for a single workload and drop just as easily. The ones on my radar at the moment, and why:
- : a very fast engine that's increasingly credible as a whole data platform rather than just an analytics accelerator, and worth watching as a cheap, open counterweight to the premium clouds.
- dbt: by now the default for version-controlled, testable, automatable transformation, and the piece most of the assemble-it-yourself crowd builds around.
- : in-process analytics that gets a startling amount of crunching done right where your code already runs, no cluster required. Very much in the spirit of using the hardware you already have.
- : code-first, -as-code dashboarding, with defined as files in a repo rather than clicked together in a , which is the right direction for anything you want versioned and reproducible.
- : spreadsheet-grade exploration, ad-hoc analysis, and what-if simulation straight over the warehouse, for the advanced users who'll never settle into a governed BI tool.
- Ollama: running open locally and on your own terms, which is the bit that makes "some compute has to stay on the ground for cost or privacy or regulation" actually workable.
- Code-first ML tooling: the lightweight, open machine-learning workflow tools that don't make you commit to one cloud's ML stack just to get going.
I'm not claiming any one of these wins. What they have in common is that you can try them cheaply and abandon them cheaply, which is the property you want while the ground is still moving.
And before any of it, decide what the goal actually is, because "migrate to platform X" is a task dressed up as an objective. Cutting operational cost, getting value out of an investment you've already made, standing up genuinely new services, and giving the team more interesting work all point in different directions, and conflating them is how you end up doing the expensive thing for a reason nobody has said out loud. Say the goal first and the platform decision tends to answer itself.
So, in practice
Strip the logos away and a sound data-platform decision comes down to a few habits. Take the easy operational wins when the numbers justify them, without pretending they're more than that. Put the effort you save into the modelling and loading discipline that compounds wherever you run. Keep premium compute for premium work and let the commodity work stay cheap. Hold your data in an open format you control, so the engine is always a choice you can revisit. Choose tooling that fits the team you actually have. And don't trade tomorrow's flexibility for a migration that merely looks tidy today.
The future isn't somewhere you migrate to. It's a set of habits you keep: open data, swappable compute, and the discipline to spend money only where it buys you something you couldn't have had otherwise.
Companion reading, the deliberately neutral survey and the numbers behind it:
References
- 1Snowflake, "Polaris Catalog Is Now Open Source" (vendor-neutral Iceberg REST catalog, donated to the Apache Software Foundation) and Snowflake Documentation, "External query engine support for Apache Iceberg tables with Snowflake Horizon Catalog (GA)", February 6, 2026. snowflake.com ↗ · docs.snowflake.com ↗
- 2Databricks, "Delta Lake Universal Format (UniForm) for Iceberg compatibility, now in GA" (one copy of Delta data readable by Iceberg clients). databricks.com ↗
- 3Microsoft Learn, "OneLake, the OneDrive for data" (tenant-wide lake; all tabular data stored as open Delta Parquet; cross-item querying). learn.microsoft.com ↗ Accessed 2026-06-13