Technology & SoftwareAgentic AI
From Data Chaos to a Trust Score for Every Table
Databricks' own internal Data Platform team rebuilt data governance across its internal lakehouse of more than 100,000 tables spanning engineering, go-to-market, HR, finance and product telemetry, using Unity Catalog to move from inconsistent, team-by-team access patterns to automatic, measurable governance. The team built a universal classification model tagging every table by sensitivity and domain, an internal system called Fortress enforcing purpose-bound, time-limited access via Unity Catalog's security APIs, and a continuous Data Governance Score (0-100) per dataset covering documentation, reliability and governance, propagated automatically through data lineage. The resulting governance layer now underpins AI agents used across every function at Databricks, from engineering to finance to facilities, letting teams check a dataset's governance score before using it to train a model instead of running manual reviews.
Databricks
Unity Catalog · Lakeflow Connect · Databricks Apps +3