What’s coming?

Learn about upcoming Databricks releases.

Updates to the Databricks SQL API for managing queries, alerts, and data sources

Databricks plans to release a new version of the Databricks SQL API for managing queries, alerts, data sources, and permissions. This upcoming release is designed to provide you with a more robust and user-friendly experience. When the new API version becomes available, the legacy version will continue to be supported for at least six months. This transition period is intended to give you sufficient time to migrate your applications and integrations to the new version before the older version is phased out.

Highlights of the new version of the Databricks SQL API

New versions for the Queries API and Alerts API include the following:

  • The new Queries API and Alerts API will be accessible at api/2.0/sql/..., replacing the current API path /api/2.0/preview/sql/....

  • The new Queries API and Alerts API use a new query and alert definition with more descriptive field names and types.

  • The update endpoints for the new Queries API and Alerts API support partial updates using PATCH instead of POST. Additionally, these endpoints allow transferring ownership of queries and alerts. Previously, this was only allowed using the transfer object ownership API.

  • The list endpoints for the new Queries API and Alerts API is paginated using token-based pagination.

  • The list endpoint for the new Queries API no longer supports filtering by name or custom ordering. Instead, all accessible queries are returned and sorted in ascending order by their creation time.

  • The restore endpoint for the Queries API will be deprecated. Trashed queries can continue to be restored through the Databricks UI. A tool for restoring workspace objects is planned as a replacement for this functionality and will be included as an update in the Workspace API.

  • The delete endpoint for the new Alerts API moves the alert to trash instead of permanently deleting the alert. Trashed alerts will be automatically cleaned up after 30 days. Within 30 days, trashed alerts can be restored through the Databricks UI. A tool for restoring workspace objects is planned as a replacement for this functionality and will be included as an update in the Workspace API.

Deprecation of the Data Sources API:

  • The Data Sources API will be deprecated. Currently, its functionality is limited to getting a list of SQL warehouses. The new Queries API will support passing in and returning SQL warehouse IDs instead of data source IDs. A separate API call to convert between data sources and SQL warehouses is no longer required.

Updates to Databricks SQL ACL/Permissions APIs:

  • The new version of the API no longer supports Get object ACL or Set object ACL. Those actions should be done using the Workspace API to manage permissions.

For more details on the new versions of the Queries API and Alerts API, including a precise list of field and format changes, contact your Databricks account team.

Legacy Git integration is EOL on January 31

After January 31st, 2024, Databricks will remove legacy notebook Git integrations. This feature has been in legacy status for more than two years, and a deprecation notice has been displayed in the product UI since November 2023.

For details on migrating to Databricks Git folders (formerly Repos) from legacy Git integration, see Switching to Databricks Repos from Legacy Git integration. If this removal impacts you and you need an extension, contact your Databricks account team.

External support ticket submission will soon be deprecated

Databricks is transitioning the support ticket submission experience from help.databricks.com to the help menu in the Databricks workspace. Support ticket submission via help.databricks.com will soon be deprecated. You’ll continue to view and triage your tickets at help.databricks.com.

The in-product experience, which is available if your organization has a Databricks Support contract, integrates with Databricks Assistant to help address your issues quickly without having to submit a ticket.

To access the in-product experience, click your user icon in the top bar of the workspace, and then click Contact Support or type “I need help” into the assistant.

The Contact support modal opens.

Contact support modal

If the in-product experience is down, send requests for support with detailed information about your issue to help@databricks.com. For more information, see Get help.

JDK8 and JDK11 will be unsupported

Databricks plans to remove JDK 8 support with the next major Databricks Runtime version, when Spark 4.0 releases. Databricks plans to remove JDK 11 support with the next LTS version of Databricks Runtime 14.x.

Automatic enablement of Unity Catalog for new workspaces

Databricks has begun to enable Unity Catalog automatically for new workspaces. This removes the need for account admins to configure Unity Catalog after a workspace is created. Rollout is proceeding gradually across accounts.

New charts and chart improvements

Databricks plans to add new charts to the SQL editor, SQL dashboards, and notebooks. This change will bring faster chart rendering performance, improved colors, and faster interactivity. See New chart visualizations in Databricks

Favorites functionality

Databricks plans to enable favorites functionality in the workspace. You’ll be able to save content such as notebooks, dashboards, experiments, and queries to a list of favorites, and then access your favorites from the homepage.

sqlite-jdbc upgrade

Databricks Runtime plans to upgrade the sqlite-jdbc version from 3.8.11.2 to 3.42.0.0 in all Databricks Runtime maintenance releases. The APIs of version 3.42.0.0 are not fully compatible with 3.8.11.2. Confirm your methods and return type use version 3.42.0.0.

If you are using sqlite-jdbc in your code, check the sqlite-jdbc compatibility report.