Databricks cost: autotermination, Jobs compute and DBU rates
Short answer: a cluster bills DBUs for every hour it runs, busy or not, and the rate depends on the kind of compute. Most avoidable spend is scheduled work on all-purpose clusters, clusters nobody stopped, and long autotermination tails.
What it costs (illustrative)
All-purpose compute lists at roughly $0.55 per DBU; Jobs compute at roughly $0.15 (Premium, AWS pay-as-you-go; Azure, GCP and contracts differ).
VM cost is billed separately by your cloud and is the same on either kind of compute; this tool does not model it.
DBUs per hour is the sum over a cluster's nodes and is shown in the cluster UI.
Where the money is
Scheduled work on all-purpose clusters: a nightly job pointed at a shared cluster pays the all-purpose rate; a job cluster pays the Jobs rate.
Autotermination off: the cluster runs until someone stops it, 730 hours a month.
Long autotermination: every start idles for the whole setting before stopping.
Measure and fix
DBUs: SELECT usage_metadata.cluster_id, sku_name, SUM(usage_quantity) FROM system.billing.usage WHERE usage_date >= date_sub(current_date(), 30) GROUP BY 1, 2;
Autotermination and DBU per hour: the cluster configuration; starts from the cluster event log.
Paste rows into the advisor or run node cli.mjs analyze clusters.csv. It prints the setting to change per cluster.
Change one cluster at a time and watch for a week before rolling out.
Questions
What autotermination should a Databricks cluster use?
30 minutes is a common default for interactive clusters. Shorter saves more idle tail but restarts cold; longer pays for idle time after each session.
Why is my Databricks bill high when jobs are short?
Usually the compute kind and idle time: a job on a shared all-purpose cluster pays about 3.5 times the DBU rate of a job cluster, and an always-on cluster bills around the clock.
Does Jobs compute change what my job does?
The code is the same; the cluster is created for the run and terminated after, so there is a startup wait and no shared state between runs.