2025 in Review: Databricks SQL, faster for every workload

by
0 comments
2025 in Review: Databricks SQL, faster for every workload

For most data teams, performance is no longer about one-time tuning — it is about keeping analytics fast as data, users, and governance scale, without costs growing in step. In a year-in-review post, Databricks reports that this expectation is now built into Databricks SQL (DBSQL): production workloads ran on average up to 40 percent faster in 2025, with no tuning, query rewrites, or manual intervention required.

The broader story spans the platform. From faster dashboard loads and more efficient pipelines to governance and queries that stay responsive even on shared data, performance improved across the board, while geospatial analytics and AI functions scaled without added complexity. With DBSQL Serverless, Unity Catalog managed tables, and predictive optimization, improvements apply automatically, so existing workloads benefit as the engine evolves. The company’s stated goal: make workloads faster and cheaper by default.

Faster query execution across workloads

Databricks measures performance using millions of real customer queries run repeatedly in production — tracking how those workloads change over time rather than relying on isolated benchmarks. In 2025, the reported gains applied by default through engine-level optimizations such as predictive query execution and Photon vectorized shuffle, with no configuration changes:

  • Exploratory workloads saw the biggest benefit, running up to 40 percent faster on average, letting analysts and data scientists iterate more quickly on large datasets.
  • Business intelligence workloads improved by roughly 20 percent, producing more responsive dashboards and smoother interactive analytics under concurrency.
  • ETL workloads ran about 10 percent faster, shortening pipeline runtimes without rework.
These measurements come from the Databricks Performance Index, which is derived from statistically repetitive workloads and calculated against billions of production queries.

In short: workloads evaluated on the platform a year earlier were already running faster without any customer action.

Governance that scales without slowing analytics

As data estates grow, governance can become a hidden source of latency — permission checks, metadata access, and lineage lookups all slow queries, especially in interactive, high-concurrency environments. In 2025, Unity Catalog end-to-end latency improved by up to 10x, driven by optimizations across the catalog service, networking stack, runtime client, and dependent services. The practical results: dashboards stay responsive under fine-grained access controls, high-concurrency workloads scale without metadata bottlenecks, and interactive exploration of governed data feels faster. Teams no longer need to trade strong governance for performance.

Delta Sharing: shared data that behaves like native data

Sharing data across teams or organizations has traditionally carried a performance penalty — queries against shared tables ran slower, and optimizations applied unevenly. In 2025, Databricks reports closing that gap: queries on tables shared through Delta Sharing improved by up to 30 percent through better query execution and statistics propagation, bringing shared-data performance in line with native tables.

Delta sharing and UC perf improvements
From 2024 to 2025, end-to-end Unity Catalog latency becomes 10x faster and delta sharing improves by 30%.

The change matters most where external data must behave like internal data: data marketplaces, cross-organization analytics, and partner-driven reporting can now run on shared datasets without sacrificing interactivity or predictability.

Lower storage costs by default

As data volumes grow, storage efficiency becomes a bigger slice of total cost. In 2025, Databricks made Zstandard — an open-source compression format — the default for all new Unity Catalog managed tables, delivering up to 40 percent storage savings versus older formats without degrading query performance.

zstd full fix
With Zstd, we have delivered cost savings of up to 40% compared to older storage formats.

The benefit applies automatically to new tables, with migration tooling for existing tables described as coming soon. Large fact tables, long-retention datasets, and fast-growing domains see immediate cost reductions with no changes to how queries are written.

Geospatial analysis without specialized systems

Spatial joins, range queries, and geometric calculations are computation-heavy, and at scale they have traditionally demanded specialized systems or careful tuning. In 2025, spatial SQL queries ran up to 17x faster on Databricks SQL, powered by engine-level work including R-tree indexing, optimized spatial joins in Photon, and intelligent range optimization.

local full correction
From 2024 to 2025, spatial connectivity for large-scale data accelerated by 17 times.

Teams can now work with location data in standard SQL while the engine handles execution complexity — making real-time location analytics, large-scale geofencing, and geographic enrichment practical inside the warehouse rather than in separate tooling.

AI Functions: scalable AI directly in SQL

Applying AI to data has usually meant leaving the warehouse: separate pipelines, model infrastructure, and glue code to bring results back. AI Functions put AI directly into SQL, and in 2025 their scale and performance expanded substantially. A new batch-optimized distributed infrastructure delivered up to 85x faster performance for tasks like ai_classify, ai_summarize, and ai_translate — letting batch jobs that took hours finish in minutes. Databricks also introduced ai_parse_document, using purpose-built document-understanding models hosted on Databricks Model Serving, reported at up to 30x faster than general-purpose alternatives for processing unstructured content at scale.

Complete revamp of AI functions
For large batch workloads, AI functions become up to 85x faster in 2025.

Together these enable intelligent document processing, insight extraction from unstructured data, and predictive analytics through familiar SQL — with AI workloads growing alongside analytics workloads rather than in separate systems.

Limitations and what to watch

  • All figures are self-reported by Databricks from its own telemetry; individual workloads will vary, and “up to” multipliers describe best cases, not averages (the workload averages cited are the exception).
  • Some capabilities, such as Zstandard migration tooling for existing tables, were still forthcoming at publication time.
  • Comparable engines (Snowflake, BigQuery, Fabric) publish similar year-over-year gains; buyers should benchmark on their own workloads rather than vendor numbers. Databricks’ broader platform trajectory is covered in this summary of its 2026 State of AI Agents report and this look at enterprise AI infrastructure.

Availability

All improvements described are live in Databricks SQL Serverless with nothing to enable and no configuration required; existing workloads benefit automatically as the platform evolves.

Related Articles