Member-only story
How To Calculate the Real Cost of Azure Databricks SQL Warehouse Instances
In this story, we will learn how to calculate the cost of Azure Databricks SQL Warehouse instances.
A Databricks SQL Warehouse is a compute resource that lets us query and explore data on Azure Databricks.
At the moment of writing, Azure Databricks offers 3 types of SQL Warehouse:
- SQL Warehouse Classic: the SQL Warehouse Classic compute layer exists in our Azure subscription account and supports Photon but not Predictive IO or Intelligent Workload Management.
- SQL Warehouse Pro: the SQL Warehouse Pro compute layer exists in our Azure subscription account and supports Photon and Predictive IO but not Intelligent Workload Management.
- SQL Warehouse Serverless: using the Azure Databricks serverless architecture, a Databricks SQL Warehouse Serverless supports all of the performance features of Databricks SQL (Photon, Predictive IO, and Intelligent Workload Management) and exists in our Azure Databricks account.
As we can see in the list above, the most significant difference between SQL Warehouse Classic and SQL Warehouse Pro is that the compute layer is in our Azure subscription account, and the SQL Warehouse Serverless is in our Azure Databricks account.
Related Stories:
1. DIY Method
The first section is dedicated to developers or technical people who want to understand the logic behind the cost analysis and probably write their own tools or scripts to calculate the cost of Azure Databricks SQL Warehouse instances.
1.1. DIY Method — SQL Warehouse Classic
To calculate the cost of an Azure Databricks SQL Warehouse Classic or Azure Databricks SQL Warehouse Pro, we need to account for the following components:
- SQL Compute Hours: The time for which the SQL Warehouse was running.
- Databricks Unit (DBU) Hours: The compute units used by the SQL Warehouse are billed hourly.
- Storage Costs: Costs associated…
