Databricks announces first-class support of the Iceberg format in Delta Sharding

by
0 comments
Databricks announces first-class support of the Iceberg format in Delta Sharding

With over 300% year-over-year usage growth for 2 consecutive years, Delta Sharding is the most widely adopted open protocol for data and AI sharing. Major data providers including SAP, Walmart, Atlassian, and LSEG use Delta Sharing to share data with their partners and customers across clouds and platforms. Today, we are pleased to announce that Databricks Delta Sharding has first-class support for the Apache Iceberg format.

Data providers can now share data securely and live with any client supporting the Apache Iceberg REST Catalog API from Databricks. Recipients can use this capability on platforms like Snowflake, Trino, Flink, and Spark across all clouds – connecting to Delta Sharing’s open ecosystem.

Additionally, we are launching a private preview that enables data providers to use Delta Sharing to share Iceberg tables managed by catalogs outside of Databricks – including AWS Glue, Hive Metastore, or Snowflake Horizon.

Figure 1: Integrate formats in sharing and collaboration for an open ecosystem

Additionally, you can share any new or existing tables (Delta or Iceberg, managed or foreign). It is moving towards full open interoperability. You can bring in Iceberg tables from any external catalog, manage them through Databricks and Unity Catalog, and then share them with any recipient – ​​whether they’re on Databricks, the Iceberg client, or the Delta client. This enables you to leverage Unity Catalog as your unified data governance layer, giving you one place to create and share data.

In this blog post, we will explain why open data sharing is important. We’ll also learn how Delta Sharing works for Iceberg customers through a demo and customer use example.

Why it matters: open vs. closed sharing

Most data sharing solutions aren’t actually sharing – they’re trapping. They are fundamentally closed and engineered to ensure vendor lock-in, so you can only share with people who are already inside their closed ecosystem. This limits your options, stifles innovation, and drives massive, redundant data replication.

Delta sharing is the most widely adopted open standard for secure data sharing. Used by class-leading data providers, it is built to work across a variety of clouds and platforms. Delta Sharing works on three main principles:

  • Share any property.
  • Share with anyone.
  • Share without any hassle.

The addition of Iceberg customer support strengthens this commitment. It lets you share a Delta table while recipients experience it as a native Iceberg table. Sharing happens over the Iceberg REST API, so recipients can connect from any Iceberg-compatible platform. This allows you to get the best of both worlds: data providers benefit from advanced Delta Data sharing features such as view sharing, while recipients receive regular Iceberg tables through the Iceberg REST API.

Figure 2: Share data directly on Iceberg-compatible tools
Figure 2: Share data directly on Iceberg-compatible tools

Recipients get secure, live access to the source data. It breaks down silos and lets you share data openly with anyone.

This feature is ideal for organizations that need to share data externally with partners and customers using the Iceberg client, such as those working on Snowflake or integrating with platforms like Trino, Flink or Spark. Unblocking seamless, bi-directional data exchange in a multi-cloud or hybrid environment also benefits companies with multiple business units operating on multiple platforms. Industries already taking advantage of these patterns include healthcare, retail, finance, ad-tech, and more.

Interoperability: both source and destination

Because we strongly believe in open data access, we don’t stop Iceberg customers from sharing data. We are now developing the next evolution: sharing foreign iceberg tables that exist in external catalogs such as AWS Glue or Snowflake Horizon. We’re excited to announce a private preview of Delta Sharing support for exotic Iceberg tables.

You may ask: why share iceberg tables? Through Delta sharing if it lives in AWS Glue or Snowflake? Why not share directly from within that platform?

First, by listing your external iceberg data in the Unity Catalog, you get a unified governance Layers in Unity Catalog, allowing you to gain full visibility and governance across your data estate. Additionally, using Delta Sharing you can achieve best of both worlds: You benefit from Delta Sharing’s best-in-class sharing functionalities, while keeping your data in iceberg format. This includes for example the ability to use Delta Share View for fine-grained access control, which is not natively supported by the Iceberg IRC API.

With this private preview, Databricks Lakehouse is open in both directions. Your lakehouse can provide data to and receive data from the growing iceberg ecosystem.

This dual capability lets you:

  • Simple Collaboration: No matter which open table format you use (Delta or Iceberg), work together.
  • Governed Sharing: Unity controls catalog access and provides audit logs.
  • Widest reach: Share data as both a sender and a recipient, breaking down platform walls.

How does this work?

Imagine your company, provider corporationUses Databricks and Delta Lake to manage customer data. All you need to securely share daily list of product sales Partner IncWhich uses Snowflake and prefers the Iceberg format.

Before this feature: Provider Corp must manually export the data, convert it to a Snowflake-readable format, upload it to the partner’s cloud storage and set up a complex synchronization task. It is slow, expensive, involves significant administrative overhead, and risks data becoming out of date.

For Iceberg customers with Delta Sharing:

  1. provider corporation Enables Iceberg to read sales data through Uniform (can include managed and external Delta tables, views, materialized views, and streaming tables), and share it via Delta Sharing. It provides live access without any duplication or re-engagement.
  2. Partner Inc Establishes a simple connection to Snowflake using the provided credentials for secure authentication via a short-term bearer token.
  3. Analysts at Partners Inc. can quickly query the shared table using standard SQL, treating it like a native Iceberg table in their Snowflake environment.
  4. The data they see is always stay (zero-copy), and Provider Corp maintains Complete security and governance With auditing and monitoring using Unity Catalog.

This makes data sharing quick, secure, and completely format-agnostic.

Related Articles

Leave a Comment