Announcing the Sundeck Query Engineering Platform for Snowflake

It’s an exciting day today. Sundeck is coming out of stealth. What’s more, we’re sharing fundraising news and announcing the public availability of our product.

Since we started Sundeck, our focus has been singular: expose a new creative frontier to data engineers, analysts and data scientists. Our specific target has been the space between data warehouses and the tools that use them. Our goal is to make this layer accessible to data engineers so they have more control and creativity about how queries are interpreted. We even think it should be possible to redefine what a query is. We call this Query Engineering. And today, we’re announcing our public preview: Sundeck, a Query Engineering Platform.

To help people better understand what a query engineering platform can help with, we provide a number of prepopulated recipes to enable common use cases including:

• Warehouse Optimization
• Adaptive dbt Routing
• User Help & Guardrails
• Granular Cost Control
• Adding New Query Syntax

I’m happy with the recipes we provide but I’m even more energized by the magic that comes with the interaction between Sundeck’s capabilities and the creativity of the data engineers and analysts using it. We’ve already seen a bunch of this in our private preview (and several of the recipes above are inspired by real customer examples).

The amazing team at Sundeck got us to this milestone. This small group of insanely smart and talented people make me feel lucky every day. Creating something entirely new and different isn’t easy and their tenacity and fortitude have been our backbone. I’m also super happy to share the news of our seed funding round of $20 million from our partners at NEA, Coatue and Factory. They’ve been great partners in our early journey.

No technology is entirely new and several key technologies were especially formative in the creation of Sundeck.

• dbt: The creative energy that dbt unleashed for analytics engineering is inspiring and we look to unleash similar energy for query engineering. Data engineers need tools that make things accessible, extensible and shareable. We look for Sundeck to satisfy those needs across all query use cases, just as dbt has done for etl and analytics engineering use cases.
• Apache Calcite: For those that don’t know, Calcite is a highly modular database with a very extensible optimizer. It is less well known than dbt but has been around much longer. I was lucky enough to one of the first serious users of it as a modular optimizer in a production system and have used it in multiple projects over the last decade. It’s awesome. Thinking about how to make a small fraction of those types of capabilities to Snowflake users inspired many of the features we built.
• Apache Arrow: When I co-founded Apache Arrow 9 years ago, so many people told me it was a pointless exercise. Why would a memory format + libraries be broadly useful. Today we find ourselves at more than 65 million downloads each month and industry-spanning prevalence (for example, Snowflake’s client protocol is built on Arrow). This is further proof that the more we can make powerful technologies available to broader communities, the more great things happen.
• Substrait: This is a relatively new open source project that I co-created eighteen months ago. It looks to do what Arrow did for data to query plans. It’s young but is also a key technology that underpins much of what we’ve built at Sundeck.

While Sundeck the company has existed for a little over a year, the ideas behind Sundeck are part of a career-spanning journey of making data more accessible. I hope you find it useful! It’s free to start and easy to sign-up. Reach out and let us know what you think!