AWS Acquires DuckLabs (ducklabs.com)

Yup, they are very serious about DynamoDB and it is a very good product. Wish they released more information about it, though.

Surely they are better than Google in this regard.

> Out of all the big orgs, Amazon is probably the one that has the least regard for keeping technically interesting projects alive

I'm curious what you mean by this. I would have said the exact opposite - AWS tends to keep projects around for a very long time. They haven't acquired very many open-source projects, but the few that they have are all still running as far as I know.

who currently has the most regard for keeping technically interesting projects alive?

What good is a foundation when it's core member is owned by a public corporation? Like we know what happens, the pattern isn't unique or rare. Corporate interests will always take priority over community interests, corporations care about profit not community. If Amazon cares about the community as they care about their own workforce, that does not bode well.

I think 'least regard' belongs to Oracle. For some reason I thought DuckDB was one of Gabe's ideas over at DuckDuckGo :-).

That said, the acquisition by Amazon is interesting in that it seems to overlap with their existing DB offerings? Which makes me wonder what exactly it is that they are getting out of this.

You are right.

AWS did not acquire the DuckDB technology itself, which is MIT-licensed open source and governed by the DuckDB Foundation which holds most of the related IP [1].

What they did acquire is DuckLabs, the Amsterdam-based services and development company behind the technology which is owned by and employs the creators and major contributors.

And MotherDuck is a US-based venture-funded commercial company, whose cloud-based data platform is centered around DuckDB but has been significantly expanded recently, including Python pipelines, an agentic context layer, and a visualisation layer.

[1]: https://duckdb.foundation/

[2] https://ducklabs.com/

[3] https://motherduck.com/

This is an important distinction, I hope the title can be changed.

I was really surprised when I first came across DuckDB at how good at it is for its target use cases. It is a game changer for me for the "local analytics" space, and its ability to scale up to a large degree helps a lot.

It is simply awesome to be able to point DuckDB at a mess of CSV and other files and have an instant database on top of it that I can run regular SQL over, and it is fast and just works.

you raised a good point but (et al) didn't touch on the meaningful questions: where does the "core team" of developers of DuckDB now reside? are they now "captured/cashing-in/getting paid" at AWS, or will meaningful MIT-licensed development of DuckDB continue in a robust way outside of commercial enterprises with proprietary needs that are not aligned with where the project was going?

Not sure why the submitter changed the title from the page: "DuckLabs to Join AWS, Projects to Remain Open Source"

It is not misleading at all. For all these softwares, the team that behind is the most important part rather than the software itself.

Fixed above. Thanks!

I started using DuckDB last year and I loved it. Small, focused, crackling with competence, no bullshit. I think what I liked about it had a lot to do with it being made by a small group of technical people who weren't interested in "maximizing share holder value" (that is, maximizing executive promotions and bonuses).

Maybe this will be the rare case where founders survive corporate shenanigans and keep on doing their thing. But I'm bummed, as I've heard that optimism many more times than I've seen it happen.

They built something amazing and now have a nice exit. The code remains open under a non profit foundation. I don't think it is that bad. If working conditions are too bad people will just quit and work for the foundation.

Honestly, there's just a huge amount of thrash across the cloud & SaaS industries as a whole. It's not a lot better at Google (way behind OAI & Anthropic on coding & local AI harnesses), Microsoft (no meaningful frontier model R&D to speak of, and a constantly eroding Windows business), the neoclouds (impossible to source adequate infrastructure and are existentially threatened by the hyperscalers (especially Google) if they can't keep up with demand, chipmakers (NVIDIA vs AMD vs Qualcomm vs Apple vs the long tail of specialty shops like Cerebras), data center pure plays (legislative & community pushback, natural resources, infrastructure availability), and the SaaS folks ("let's see if AI can do it").

It's both a great time to be gold mining in tech, and also a terrible time to be a bit employee.

> Place sounds like a total mess.

What big corp is not a total mess?

Came here to post the same thing. DataFusion is fantastic.

You can even use DuckLake with Datafusion: https://github.com/tobilg/datafusion-ducklake-provider

Is Datafusion related to Polars at all? I ask because you mentioned Rust integration.

Datafusion has all of the individual pieces, especially if you just want to data analysis, but is far from an embedded database.

Indexes, transactions, a first-class storage format are all things that come included with DuckDB, that you won't have with Datafusion.

If there's no price tag mentioned, it's likely not generational wealth.

Very much doubt it’s generational wealth. I’m not sure how much people think open source companies are worth, but the path to VC-scale profitability is incredibly rare.

My guess is that Amazon wants official hosted versions and doesn’t want to go through something like the Redis fiasco with licensing. In that case, they ended up having to support their own development anyway (with Valkey), so they might as well just buy the team.

DuckDB team probably gets a nice package and pay bump, but it’s really unlikely they’re getting hundreds of millions from this.

Agreed that DuckDB as a server feels far from their initial mission, but how does async IO divert from it?

agree - I was really uneasy with the lede in their v2.0 preview:

"DuckDB has been an in-process database since day one. But people have asked us – very persistently – for a client/server mode, and we have finally caved"

This was already satisfied by numerous projects and products, and feels like a "me too" attempt to capture AI-based workflows. DuckDB always felt like "SQLite for Analytic Data" but I fear these changes and now acquiring the org leading technical direction is where they deviate for good. AWS is so unnecessary for what DuckDB can (and should, IMO) be; MongoDB jumps to mind as a cautionary tale.

This gave me a shot of panic when I read the headline - thanks for correcting the record.

DuckDB is really great. It's definitely the best tool for working with any type of tabular laptop-sized analytical data. You can do some of what DuckDB does with SQLite, but DuckDB is much more versatile and performant.

There’s trade offs for performance, access patterns, throughput, latency, concurrency, workloads, rigor, type systems, extensibility… really every characteristic you could imagine.

It’s not a solved problem because each iteration of technology doesn’t just fix the mistakes of the past, it’s an evolution to solve the problems of the present.

This is an enormously wide question but the quickest way to give you an idea would be rephrasing as:

"Are cars not a solved problem? Why are there lots of different cars? Why is one faster than the other? What's different between them?"

I think the best way to approach the subject in an easy to grasp way is to ask Gemini or another frontier AI to teach you the basics, they will do a surprisingly good job and they'll be able to react to your questions with INFINITE patience.

It's all about tradeoffs. Same reason there are so many kinds of wheeled transport and so many kinds of bridges.

There's no single set of requirements and desired properties that people have for databases.

What queries does it accept? How does it persist data? How does it manage replication and partitioning across multiple servers? Are questions with many answers and the right one varies by application.

It's a fair question!

My attempt at an answer: no, databases are not solved.

Specifically, different databases are better or worse for different use-cases.

For example, Postgres is a great "all around database" - you can use it for a lot of different things. As a "relational" database, it's really good if you have a table full of users, a table full of order, and you want to see all orders made by a user with ID=123. You need to answer questions like that a lot (eg every time someone on a website loads a page) and you need the answer fast (hundreds of miliseconds at most)

However, say your use-case is more like... you've got 100 billion rows of billing data ("joe was charged $123.45 on 2026-03-07 for a shirt, blue, size 11, brand foobar") in one table. You don't care much about joe, but you want to be able to find out how much was billed, total, in 2026-03 for blue shirts (or all year for brand foobar, or all time, for size 11). Postgres would struggle with data of that volume - you'd need a really big expensive database. A "columnar" database like duckdb (or clickhouse) might be able to answer those questions better.

Anyway, different databases are better/worse for:

- Large piles of data that you need to query in seconds

- Huge (petabytes) of data that you need to query in minutes, but can query in parallel

- Many related piles (like a standard relational database)

- Cases where you're mostly getting or retrieving single items (key-value stores)

- Huge piles of data that represent a long stream of events in time (time-series datbases)

- Piles of data that look and act more like files (object stores)

- When you need strict transactions

- When your need is very write-heavy

- When your need is very read-heavy

- and probably many others - I'm not even a huge data guy :)

So it all depends on your use-case. There are still cases that are not served well by any existing database - eg "filtering billions of rows, in milliseconds, by an arbitrary portion of several dozen very-high-cardinality columns" (to use an example that came up recently for me IRL) :)

> Are databases not a solved problem?

What do you mean by solved problem? I don't own DuckDB (sqlite, postgresql, etc). If I think I can create something as good or better than DuckDB, should I give up doing so (and get filthy rich with an acquisition) because someone thinks databases are solved? Solved databases aren't mine.

My very high level take:

Every tool is a trade off between effort to create vs power of the solution.

Effort is generally expensive so most things settle on some general purpose local maximum. If you had infinite effort available, you could build bespoke hardware and software from the ground up to solve every problem. It would be faster and more power efficient than any solution available today.

CPUs win out over integrated circuits because the same CPU can be used for ~every software problem, so by using a CPU you benefit from everyone pooling their efforts to improve the general purpose CPU rather than their own specific niche. But when you reach a certain scale/requirements it makes sense to do something more specific. This is one reason why we have standardized GPUs. Still general purpose but more specialized than a CPU. Or think about how Bitcoin mining moved to ASICs, because they need to do one specific thing as fast and as power efficiently as possible.

So for databases, when you get to specific scale and requirements the same kind of specialization starts to make sense. DuckDB or Clickhouse for analytical loads, TigerBeetle for high scale transactional stuff, etc. And that scale is aggregated across ~all software users, i.e. scale of analytical workloads being big enough to support analytical DBs.

Also as time goes on and industries develop the cost to develop specific solutions can go down.

My understanding is that MotherDuck provides hosting for DuckDB.

And DuckLabs is the engineering behind DuckDB.

If I'm correct, this seems like a signal that AWS is coming after the segment of customers wanting to host DuckDB (MotherDucks customers).

Correct me if I'm wrong though, please.

Hey, MotherDucker here! We've been working closely with the DuckLabs team since day one, and the collaboration will continue! You can read more here: https://motherduck.com/blog/duckdb-amazon/

TL;DR AWS will either acquire MD next/soon, or (more likely) keep them around for the optics that they are totally not going to starve the community of anything other than DuckDB as a hosted AWS service.

From the MD "about page":

The idea for MotherDuck came after Jordan Tigani, MotherDuck co-founder and chief duck-herder, saw DuckDB in action and thought, “Wow, this is amazing! someone should really build a serverless version.”

...

Hannes and Mark, who founded DuckLabs to focus on the core technology and build the world’s best analytics database, were looking for partners who would build a commercial cloud offering.

That seems unlikely because AWS needs to build proprietary features within their services to lock in paying (enterprise) users. The problem now is that labs will inevitably focus on this work, which I view as opposed to local, in-process when your paycheque comes from selling hosted services.

And forced AWS to acquire them instead? Nice trick.

Wouldn't be the worst thing, when I last used it, I got some c++ specific errors.

Also repackaged Blazegraph as Neptune (the original project was shut down after acquisition).

This would be my bet and hope as well.

Athena already has Trino and Spark engines, it could totally have DuckDB one too.

Feel like AWS got a lot of the concepts right with S3 Tables* ("serverless" Iceberg catalog on top of S3) ... would love, love to see a version of S3 Tables that lets you choose between Ducklake or Iceberg as the catalog.

* Cloudflare also seems to have gotten the same idea right with R2 Data Catalog

DuckDB is for analytics, so OLAP instead of OLTP. If you're using the DB for normal transactions, stick with sqlite

I've already created Haybarn which is Query.Farm's distribution of DuckDB:

learn about it here: https://query.farm/haybarn/

DuckDB is still OSS btw. It didn't go proprietary like Redis or ElasticSearch.

MallarDB

Probably removed because DuckLabs is owned by Amazon now so it’s kind of confusing. Maybe Amazon said, “just take it down, our lawyers will approve some language.”