Managed tables in Lakeflow pipelines (Beta)

BetaSep 10, 2026VerifiedAdded Sep 22, 2026

You can now create managed tables in Lakeflow pipelines and write to them with one or more append flows. This feature is in Beta . You can also use import_checkpoint on an append flow to migrate an existing Structured Streaming workload into a pipeline without reprocessing the source. See create_table , CREATE TABLE ... FLOW (pipelines) , and append_flow .

Streaming Data integration

Read Databricks's release notes

https://docs.databricks.com/aws/en/release-notes/product/2026/september#managed-tables-in-lakeflow-pipelines-beta

Summaries of vendors' own notes. Product names and logos belong to their owners; logos via logo.dev.

More Lakeflow releases

LakeflowSep 10, 2026

Maintenance windows for continuous pipelines

You can now set a maintenance window on a continuous Lakeflow pipeline to control when Databricks applies platform updates that require a restart, such as new runtime versions and serverless infrastructure maintenance. Select a day of the week, a start hour, and a time zone, and Databricks starts maintenance only within that one-hour window.

UpdateData integrationDeveloper toolsSource

LakeflowSep 11, 2026

Standard performance mode is now available for one-time runs in the Jobs API

You can now set performance_target to STANDARD in the Jobs API. Apache Airflow's DatabricksSubmitRunOperator also supports this. See Create and trigger a one-time run and Select a performance mode .

UpdateDeveloper toolsPerformanceSource

LakeflowSep 11, 2026

Session restore for serverless jobs is in Public Preview

Restore the Python variables and Spark session from a serverless compute job run into a new interactive notebook, so you can debug a failure or explore results without rerunning the entire job. Databricks captures the snapshot automatically for eligible runs. See Session restore for serverless jobs .

PreviewNotebooksSource

LakeflowSep 9, 2026

Celigo connector (Beta)

The Celigo connector is now available in Lakeflow Connect in Beta . Workspace admins can control access to this feature from the Previews page. The connector ingests audit log events, such as create, update, and delete operations on resources, from Celigo (integrator.io) into Databricks. Authentication uses a Celigo integrator.io API token.

BetaData integrationObservabilityDeveloper toolsSource

LakeflowSep 9, 2026

Anysphere Organization connector (Beta)

The Anysphere Organization connector is now available in Lakeflow Connect in Beta . Workspace admins can control access to this feature from the Previews page. The connector uses the Anysphere (Cursor) Organization Admin API to ingest organization members, groups, and group members into Databricks as daily snapshots.

BetaData integrationDeveloper toolsCoding agentsSource

LakeflowSep 9, 2026

Anaplan connector (Beta)

The Anaplan connector is now available in Lakeflow Connect in Beta . Workspace admins can control access to this feature from the Previews page. The connector ingests audit trail events and user account records from your Anaplan tenant into Databricks. Authentication uses an Anaplan CA certificate and its paired private key. See Anaplan connector .

BetaPricingData integrationSource

LakeflowSep 8, 2026

Incremental ingestion for Salesforce formula fields in Lakeflow Connect will soon be generally available

Incremental ingestion for Salesforce formula fields in Lakeflow Connect will soon be generally available. Today, Lakeflow Connect ingests formula fields by taking a full snapshot on each pipeline run by default. When incremental formula field ingestion is generally available, full snapshots will no longer be the default.

GAPricingData integrationPerformanceSource

LakeflowSep 7, 2026

Zerobus Ingest into default storage is now in Public Preview

Ingesting into tables backed by default storage with Zerobus Ingest in Lakeflow Connect is now in Public Preview . See Create or identify the target table .

PreviewData integrationSource

Also shipped on Sep 10, 2026

Google Cloud BigQuerySep 10, 2026

BigQuery conversational analytics adds ML.CORRELATION in preview

Conversational analytics can now use ML.CORRELATION to calculate statistical correlations between a target column and one or more metric columns in a table.

PreviewSQLSource

Google Cloud BigQuerySep 10, 2026

BigQuery ML.METRICS evaluates predictions without a stored model

The preview ML.METRICS function computes classification or regression evaluation metrics on any table or query with actual and predicted values, with no need to create or reference a stored model.

PreviewSQLGovernanceObservabilitySource

Google Cloud BigQuerySep 10, 2026

You can use the AI.CAUSAL_EFFECT function to quantify the impact of specific interventions on time series data

You can use the AI.CAUSAL_EFFECT function to quantify the impact of specific interventions on time series data. This feature is in Preview .

PreviewSource

Google Cloud BigQuerySep 10, 2026

BigQuery Data Engineering Agent integrates with BigQuery Graph

The Data Engineering Agent now uses BigQuery Graph for extra context between source and destination schemas, improving schema mapping accuracy in data engineering pipelines. It is generally available.

GAAI agentsSQLData integrationSource

Snowflake SnowflakeSep 10, 2026

Using dbt artifacts for Slim CI and defer to production in dbt Projects on Snowflake (General availability)

Slim CI, defer to production, failed-execution recovery, and concurrent execution are now generally available for dbt Projects on Snowflake. Reuse dbt artifacts from recent production executions to accelerate development, streamline CI/CD workflows, and recover from failed production runs without maintaining a separate artifact store.

GAStreamingSource

Snowflake SnowflakeSep 10, 2026

External secret providers (Preview)

External secret providers are now in preview. If your organization already keeps API keys, database passwords, and similar values in AWS Secrets Manager, Azure Key Vault, or Google Cloud Secret Manager, Snowflake can read those values when a query needs them, so you don't have to maintain a second copy in a Snowflake secret .

PreviewSQLSecurityDeveloper toolsSource