Skip to main content

Spice.ai OSS

What is Spice?​

Spice is a portable runtime providing developers with a unified SQL interface to materialize, accelerate, and query data sourced from any database, data warehouse, or data lake.

📣 Read the Spice.ai OSS announcement blog post.

Spice connects, fuses, and delivers data to applications and AI, acting as an application-specific, tier-optimized Database CDN.

The Spice runtime is written in Rust and is built-with industry leading technologies like Apache DataFusion, Apache Arrow, Apache Arrow Flight, SQlite, and DuckDB.

OGP

Why Spice?​

Spice makes querying data by SQL across one or more data sources simple and fast. Easily co-locate a managed working set of data with your application or ML, accelerated with in-memory Arrow, with SQLite/DuckDB, or with attached PostgreSQL for high-performance, low-latency queries. Accelerated engines run tier-native in your infrastructure giving you flexibility and control over cost and performance.

How is Spice different?​

  1. Tier-optimized Acceleration with both OLAP (Arrow/DuckDB) and OLTP (SQLite/PostgreSQL) databases at dataset granularity compared to other OLAP only or OLTP only systems.

  2. Separation of materialization and storage/compute compared with monolith data systems and data lakes. Keep compute colocated with source data while bringing a materialized working set next to your application, dashboard, or data/ML pipeline.

  3. Edge to cloud native. Designed to be deployed standalone, as a container sidecar, as a microservice, in a cluster across laptops, the Edge, On-Prem, to a POP, and to all public clouds. Spice instances can also be chained, and deployed distributed across tiers of infrastructure.

Before Spice​

Before Spice

With Spice​

With Spice

Example Use-Cases​

1. Faster applications and frontends. Accelerate and co-locate datasets with applications and frontends, to serve more concurrent queries and users with faster page loads and data updates. Try the CQRS sample app

2. Faster dashboards, analytics, and BI. Faster, more responsive dashboards without massive compute costs. Watch the Apache Superset demo

3. Faster data pipelines, machine learning training and inferencing. Co-locate datasets in pipelines where the data is needed to minimize data-movement and improve query performance. Predict hard drive failure with the SMART data demo

4. Easily query many data sources. Federated SQL query across databases, data warehouses, and data lakes using Data Connectors.

FAQ​

  • Is Spice a cache? No, however you can think of Spice data materialization like an active cache or data prefetcher. A cache would fetch data on a cache-miss while Spice prefetches and materializes filtered data on an interval or as new data becomes available. In addition to materialization Spice supports results caching.

  • Is Spice a CDN for databases? Yes, you can think of Spice like a CDN for different data sources. Using CDN concepts, Spice enables you to ship (load) a working set of your database (or data lake, or data warehouse) where it's most frequently accessed, like from a data application or for AI-inference.

DEVELOPER PREVIEW

Spice is under active alpha stage development and is not intended to be used in production until its 1.0-stable release. If you are interested in running Spice in production, please get in touch below so we can support you.

Intelligent Applications​

Spice enables developers to build both data and AI-driven applications by co-locating data and ML models with applications. Read more about the vision to enable the development of intelligent AI-driven applications.

Connect with us​

We greatly appreciate and value your support! You can help Spice in a number of ways:

We’re also starting a community call series soon!

Thank you for sharing this journey with us. 🙏