Data governance policies written as code and tested end to end
The first part of a series where the author writes data governance rules as code, builds a platform to test them, and has CI block any change that breaks a policy. It walks through how the data mesh principles map onto folders and checks.
- Governance rules live as files, and CI runs them on every pull request: a partner file that doesn't match its contract never reaches the warehouse, and a change that breaks another team's report fails and names that team.
- The project is organized around Zhamak Dehghani's four data mesh principles, mapped to folders (domains/, data_products/, platform/, governance/), with five domains and exactly one owner per dataset.
- The author favors end-to-end "proofs" over unit tests, since a proof checks that a policy still holds when all the pieces run together.
Jefferson Bonamim, a data engineer and governance coordinator, opens a series called "Governance as code" by arguing that most data governance lives in documents nobody reads. His alternative is to write the policies as code and build a working data platform to test them end to end, from a partner file landing in a bucket through to an answer from an AI assistant. The practical effect is that a bad change gets caught by CI, which fails the pull request and says why. He gives concrete examples: a partner file that doesn't match its contract never reaches the warehouse, a pull request that would break another team's report fails and names that team, column masking is defined once and enforced by the database per role, and the AI assistant can only answer from governed definitions using the least access it needs.
The middle of the piece maps Zhamak Dehghani's four data mesh principles onto actual folders: `domains/` for decentralized ownership, `data_products/` for data as a product, `platform/` for self-serve infrastructure, and `governance/` for federated computational governance. He describes five domains (Sales, Customer, Marketplace, Customer Experience and Reference), each owning its source files and contracts, with three also building data products for the others. Every dataset has exactly one owner. The governance folder holds the rules as files that run on every pull request, including checks that a partner file matches its contract before loading, that changes break no one's report without that team's sign-off, and that business terms like "customer" have one shared definition. Bonamim is candid that the defaults and override mechanisms are his own design choices, not something the mesh principles prescribe, and that he is not sure he drew the autonomy-versus-control line in the right place.
He explains why he favors "proofs" over unit tests: a unit test checks that one piece does what was intended, while a proof checks that a policy still holds once all the pieces run together. The article previews the rest of the series, covering ownership in part 2, selective use of data contracts in part 3 (including a case where a contract was wrong and the data was fine), and a partner file's journey through the landing zone in part 4. Code and a proof list are available in the linked GitHub repository, alongside references to Dehghani's 2020 Martin Fowler article and Stefan Negele's data product heuristics.