Databricks Serverless → Azure SQL Connectivity Test

A small Databricks notebook to check whether serverless compute can reach an Azure SQL Database through a private endpoint, using Lakehouse Federation (the supported path for reading SQL Server on serverless).

It's handy when you hit the classic error setting up a connection from serverless:

The TCP/IP connection to the host <server>.database.windows.net, port 1433 has failed.
Error: "... Temporary failure in name resolution ..."

What the notebook does

It climbs the network stack so that wherever it stops tells you which layer to look at:

1. DNS resolution of the SQL host (should resolve to a private IP if the private endpoint is live)
2. TCP connectivity on the SQL port
3. CREATE CONNECTION ... TYPE sqlserver — a Unity Catalog connection (this is the real serverless → Azure SQL test)
4. CREATE FOREIGN CATALOG mirroring the database
5. A query against the foreign catalog

There's a "how to read the results" table at the bottom that maps each failure point to a next step.

Why not spark.read.format("jdbc")?

Serverless notebooks don't allow custom JAR libraries, so you can't load the SQL Server JDBC driver yourself. The supported way to read SQL Server on serverless is Lakehouse Federation / the built-in sqlserver connector, which is what this notebook uses.

Prerequisites

• Azure Databricks workspace with Unity Catalog enabled and serverless turned on.
• A network path from serverless to your Azure SQL DB: a Network Connectivity Configuration (NCC) with a private endpoint rule for sub-resource sqlServer in status ESTABLISHED, and the workspace attached to that NCC. See Configure private connectivity to Azure resources.
• CREATE CONNECTION and CREATE CATALOG privileges on the metastore (workspace admins have these by default).
• A SQL Server login (username + password), ideally stored in a Databricks secret scope.

How to use

1. Import azure_sql_serverless_connectivity_test.py into your workspace (Workspace → Import, or via the CLI):

   databricks workspace import \
     "/Users/<you>/azure_sql_serverless_connectivity_test" \
     --file azure_sql_serverless_connectivity_test.py \
     --language PYTHON --format SOURCE

2. Open it and attach to serverless.
3. Fill in the widgets at the top: host, port, database, username, and either a secret scope + password key, or a plaintext password for a quick one-off.
4. Run all cells top to bottom.
5. Use the results table at the bottom to interpret any failure.
6. (Optional) Uncomment the last cell to drop the test connection and catalog when you're done.

About the "Temporary failure in name resolution" error

That's a DNS failure from the serverless compute plane. Common causes:

• The NCC private endpoint rule isn't ESTABLISHED yet, or the workspace isn't attached to the NCC.
• The serverless resource was already running when the rule was added and needs a restart to pick up the new private DNS.
• Propagation lag — rule changes usually apply within ~10 minutes but can take up to 24 hours.

DNS is managed for you on the serverless side once the rule is established; you don't configure private DNS zones yourself for serverless.

Alternative: Lakeflow Connect

If the goal is to ingest the data rather than query it in place, Lakeflow Connect's SQL Server connector is a managed option. Its ingestion gateway runs on classic compute in your own VNet (so it connects to SQL Server the normal private-endpoint way), and only the downstream ingestion pipeline runs on serverless.

Disclaimer

Provided as-is for testing and educational purposes. Not an official Databricks product. Verify behavior against the current Azure Databricks documentation.