The Recipe: When the Business Stops Asking and Starts Building

Last week I told you the story of an audit that needed data yesterday and a central team that needed three months. And I ended by saying that this week we’d step into the kitchen.

Well, here we are. Welcome to the kitchen.

But before lighting any burners, let me tell you another story. One that explains why the kitchen needs a recipe before anyone touches a pan.

On a regulatory project, the business team knew exactly which report it needed. The outputs were perfectly defined: fields, formats, frequency. They knew exactly which dish they wanted to serve.

What they didn’t know was where the ingredients came from.

They didn’t know which data warehouse tables held the data they needed. Or what format it was in. Or what it was called. And worse: they didn’t know who to ask either, because nobody “owned” that data. That’s when IT turned detective: hunting for data by field names, by descriptions, by gut feeling. Asking in meetings whether anyone knew that table. Building a first version blind so the business could review it.

What followed was months of iterations. Accounting entries that shouldn’t have been there. Others that were missing and had to reconcile. Products that were called one thing in one system, something else in another, and something else again in a third. And every time a question came up, it was the same one: “Who knows what this field is? Who decided it gets calculated this way?” Questions that bounced from email to email without anyone answering them. Because the “recipe” didn’t exist: there was only a desired dish and an IT team guessing at the ingredients.

And meanwhile, the patches we kept putting in to limp along — a view here, a script there, a temp table we’d “clean up later” — stayed. Permanently. Every iteration left behind a little residue of code that nobody documented and nobody dared touch afterwards.

That project taught me something that sounds obvious but that almost nobody does: before you cook, you need a recipe.

Not a document with a photo of the finished dish. A real recipe: what ingredients you need, where they are, what they’re called, and how they come together.

The Most Expensive Game of Telephone in the World

What I just described isn’t an isolated case. It’s a pattern that repeats on every data project I’ve ever been part of. Let me formalize it so you can recognize it:

1. The head of a business area needs a new metric. She knows exactly what she wants: “Net premium adjusted for exchange rate, excluding reinsurance, with a monthly cutoff.” It’s crystal clear in her head.

2. She writes it up in an email. Or in a Word doc. Or in a Jira ticket. She does the best she can, but she’s an actuary, not a technical writer. The result is something like: “We need net premium minus reinsurance, converted to euros, by month.”

3. The ticket lands with the central data team. An engineer reads it. He knows Spark, Airflow and dbt. But he doesn’t know what “net premium” is or how it differs from “gross premium.” Or whether “excluding reinsurance” means subtracting a column or filtering out records.

4. The engineer makes his best interpretation, builds the pipeline and delivers it.

5. The business reviews it: “No, this isn’t what I asked for. The exchange-rate adjustment is applied before subtracting reinsurance, not after. And the line-of-business filter is missing.”

6. Back to step 3.

7. Repeat 4 to 7 times.

What should take a week ends up taking two months. Multiply that by the 120 requests in the queue and now you know why Time-to-Value in data is measured in quarters.

But the really interesting part isn’t that this process is slow. It’s why it’s slow.

It’s not a capacity problem. It’s not that the engineer is bad or that the actuary can’t explain herself. It’s a translation problem.

There’s a business concept that has to cross a border between two worlds. The world of the business (where concepts have meaning) and the world of IT (where concepts become code). And every time a concept crosses that border, it degrades. Like in the game of telephone: the signal loses fidelity with every hop.

And on my regulatory project, the problem was even worse: not only did we have to translate what the business wanted, we also had to guess where the data was. Because “Product” was called one thing in the contracts system, another in billing, and another in the data warehouse. Nobody had unified that identity.

The fascinating thing is that software engineering solved this problem years ago.

But before getting to the solution, let me tell you what really happens in each iteration. Because it’s not just that “it takes longer.” Each round of telephone carries a cost that compounds:

• The first iteration takes 2–3 weeks: the engineer builds the pipeline from scratch, based on his interpretation of the Word doc.
• The second iteration takes 1–2 weeks: the business reviews it, finds errors, the engineer fixes them. But now he has to understand why what he built isn’t what they asked for.
• The third iteration takes another week: the edge cases show up. “What about when the exchange rate is zero? When there’s no reinsurance? What about contracts canceled mid-month?”
• The fourth and fifth: by now the engineer has spent a month and a half on this table. He knows the data better than anyone in IT. But he has 119 other tables waiting for him.

And there’s something that usually doesn’t get counted: the cost of lost context. Every time the engineer switches tables and then comes back, he has to rebuild his mental context. What did we say about reinsurance? Was it before or after the exchange rate? Was the filter by line of business or by transaction type? All of that lives in his head or in a 47-message email thread nobody is ever going to reread.

Knowledge gets created, lost, re-created and lost again. With every iteration.

Software engineering solved this years ago. Why is data still stuck in 2005?

If you work in software development, the process I just described would seem absurd to you.

Imagine that, to build a feature, the product owner wrote a Word doc explaining what it should do. Then a developer read it, interpreted it and coded it by hand. Then another team wrote the documentation in a wiki. And then a third team created the tests in a different repository.

Four separate artifacts. Born from the same concept. Diverging from day one.

Nobody does this. It would be madness.

In modern software development, this problem was solved with a simple idea: a single artifact from which everything is generated automatically. The documentation, the code, the tests. If you change the artifact, everything changes with it. They can’t fall out of sync. They’re the same thing.

Now come back to the world of data. How many separate artifacts are born from a single business concept?

1. The business’s Word doc/email — what the user wants (in their language).
2. The engineer’s SQL — the technical interpretation (in another language).
3. The wiki documentation — whatever someone remembered to write down (out of date since day two).
4. The quality tests — if they even exist, and if anyone maintains them.
5. The business glossary PDF — the one nobody opens after the kickoff meeting.

Five artifacts. Five sources of truth. Five versions of the same concept, drifting a little further apart with every sprint.

What if you could have a single artifact that was at once the definition, the code, the documentation and the tests for your data?

The Recipe: much more than a Data Contract

This is where the kitchen metaphor fits like a glove.

In a professional restaurant, the recipe isn’t written by the oven manufacturer. It’s written by the chef. The one who knows flavors, textures and pairings. The one who understands the dish.

What the oven manufacturer does is guarantee that, given a recipe, its oven executes it correctly. 200 degrees for 45 minutes. The oven has no opinion on whether the dish takes rosemary or thyme. That’s the chef’s call.

Let’s bring this over to the world of data.

Who knows that a “Customer” is identified by NIF and that a Customer can have N Contracts? The business.

Who knows that “Net Premium = Gross Premium − Ceded Reinsurance” and that the exchange-rate adjustment is applied first? The business.

Who knows that the cancellation date can’t be earlier than the start date and that the amount can’t be negative? The business.

So why are we asking IT to write the recipe?

That’s the question that changes everything.

Grandma’s recipe

Think about Grandma’s recipe.

Grandma makes a paella that will change your life. You ask her for the recipe. She tells you: “A good glug of oil. A handful of rice. Salt to taste. And you leave it until it’s done.”

With those instructions, she makes a masterpiece. She’s been making it for 40 years. For her, “a good glug” is exactly 47 milliliters. “Until it’s done” is exactly 18 minutes. “Salt to taste” is exactly half a teaspoon. But none of that is written down anywhere. It lives in her hands.

Now imagine three grandchildren trying to follow those same instructions:

• The first reads “a good glug” as half a glass. Oil soup.
• The second reads “until it’s done” as half an hour. Charcoal.
• The third reads “salt to taste” as “to my taste.” Inedible.

Grandma tastes all three dishes and says: “But I explained it perfectly!”

Three grandchildren. Same recipe. Three different disasters.

Replace “Grandma” with “the business analyst who’s been around for 15 years and keeps it all in her head.” Replace “grandchildren” with “the three engineers trying to implement her requirement.” Replace “paella” with “data pipeline.”

Your data warehouse is Grandma’s recipe. It works as long as Grandma is around. The day she retires, moves to another project or simply goes on vacation… the dish disappears with her.

The recipe we need is something else. Not “a good glug.” But “200 ml of extra-virgin olive oil.” Not “until it’s done.” But “180°C for exactly 45 minutes.” Not ambiguous prose that a human interprets. But a specification that a machine executes. The same way every time. No matter who runs it.

The recipe — what in technical terms we’d call a Data Contract — should be an artifact with four properties:

1. It’s written by the business, in the business’s language. It doesn’t say CREATE TABLE or SELECT FROM. It says: “A Customer is identified by NIF and person type. A Customer has N Contracts. Net Premium is Gross Premium minus Ceded Reinsurance.”

2. It’s structured, not prose. It’s not a Word doc full of ambiguous paragraphs. It’s a structured file where every concept has its place. The definitions are precise. The rules are explicit. There’s no room for interpretation.

3. It’s executable. This is the key. It isn’t a document an engineer reads and interprets. It’s an artifact a system reads and executes. The system takes that recipe and automatically generates everything it needs: the tables, the loads, the validations, the documentation. No human translation. No game of telephone.

4. It’s technology-agnostic. The recipe doesn’t say “create a table in BigQuery.” It says what the business needs. If tomorrow you migrate from BigQuery to Snowflake, the recipe doesn’t change. The oven does. Exactly like in the kitchen: a Bolognese recipe is the same whether you cook it in a gas oven or an electric one.

What does a recipe look like in practice?

So this doesn’t sound abstract, let’s look at what a real recipe contains. I’m not going to show you code (that will come later in the series), but I will show you the conceptual structure.

Picture a table in an insurer’s policy system with 200 fields. Everything mixed into the same row: customer data, broker data, product data, branch office data and the policy itself.

The recipe for that table would say something like:

Source: Policy table from the core system. 200 fields.

Entities it contains: - Customer: identified by the ID_CLIENTE field. It’s a global entity (shared with other sources). - Broker: identified by COD_MEDIADOR. Global. - Product: identified by COD_RAMO + COD_MODALIDAD (composite key). Global. - Branch Office: identified by COD_TERRITORIAL + COD_OFICINA. Global. - Policy: identified by NUM_POLIZA + COD_CERTIFICADO (composite key). It’s the entity specific to this source.

Main relationship: - A Policy links a Customer, through a Broker, for a Product, at a Branch Office. That relationship captures the full transaction.

Attribute groups: - The name, NIF, address and scoring fields belong to Customer (and go into its satellite). - The commission and channel fields belong to Broker. - The premium, sum insured, deductible and terms fields are attributes of the Policy.

All of that information lives in a structured file. Not in a Word doc. Not in someone’s head. In an artifact the system can read, validate and execute.

Now think about the effort it would take to do this by hand. An engineer would have to go through the 200 fields one by one, decide which business entity each one represents, and write the SQL to create every table, every load, every validation.

With the recipe, the business analyst groups the fields by dragging them onto the entity they belong to. The system generates the rest.

See the difference from a traditional functional design?

A functional design is a document that a human reads, interprets and translates into code. It’s step 2 of the game of telephone. It’s subject to ambiguity, drift and obsolescence.

An executable recipe is an artifact a system consumes directly. There is no step 2. There’s no translation. There’s no game of telephone.

The specification AND the implementation are the same thing. That’s what software engineering achieved years ago. And that’s what the recipe brings to the world of data.

And most importantly: the person who writes it is the one who knows. The Risk actuary. The Marketing analyst. The Finance controller. In their own language. With their own concepts. Without needing to know SQL, or Python, or dbt, or how an Airflow pipeline works.

“Those who know finally can.”

An important nuance: when I say the business writes the recipe, I’m not saying they open a technical file and write code. The business uses a tool that shows them the source’s fields and lets them group them, assign them to entities and define rules — in their language, with their context. The tool guides them. The result is the executable recipe. But the one making the decisions about what each piece of data means is the business, not IT.

But there’s a fifth property almost nobody mentions

The recipe implements the Shared Kernel.

Remember the problem from Week 1? Each domain defined “Customer” its own way. Marketing with 23,000 records, Risk with 8,500, Sales with 12,000. Noodles with pizza sauce.

The recipe solves this at the root. When an analyst creates a recipe for a source table and says “this table contains the Product entity,” they aren’t inventing a new Product. They’re hooking their data up to the global Product concept that lives in the shared catalog. With its normalized identity. With its unique business key.

It doesn’t matter if it’s called COD_PRD in the contracts table, COD_PRODUCTO in the billing table, and PROD_BASE in the legacy system. Each source’s recipe says: “My COD_PRD field is the catalog’s Product.” And the system normalizes the identity automatically.

It’s the standard plug we talked about last week, but implemented in practice.

And there’s more. The recipe doesn’t just map entities. It lets you break a flat table down into its real business entities. Back to our 200-field policy table. With a recipe, the analyst can say: “These name and NIF fields belong to the Customer entity (and go into its satellite). These commission fields belong to Broker. These line and coverage-type fields belong to Product. And the rest are attributes of the Policy.” Each attribute group is assigned to the entity it semantically belongs to. Without writing SQL. Declaratively.

The flat table breaks down into its business entities. And each entity connects to the global catalog.

That’s what makes the recipe far more powerful than a conventional Data Contract. It doesn’t just document what the business wants. It also implements the common language between domains. It also normalizes identities. It also generates the code. It’s the glue between Week 1 (the Shared Kernel) and the weeks ahead (the pantry and the mise en place of the architecture).

And there’s a sixth property: the recipe is governed

This may sound obvious, but in practice almost nobody gets it right.

The recipe has an owner. A responsible team. Someone to go to when something doesn’t add up.

Sounds trivial, right? Well, think about how many data warehouses you know where that’s actually true. Where, if you need to know what a field means, or why a transformation rule is the way it is, or who to ask when a number doesn’t add up… there’s a clear answer.

In most companies I’ve seen, one of two things happens:

Scenario 1: There’s no governance. Nobody owns anything. The tables exist because someone created them three years ago and they’re still there. If you need to understand a piece of data, you ask on Slack, send an email, and with luck someone remembers. Knowledge lives in the heads of people who might switch teams or leave the company. When that happens, the data becomes a black box.

Scenario 2: There’s governance… made of cardboard. There’s a formal process. To push something to production, someone has to fill in a governance form: field name, description, type, sensitivity. The problem is that the form is filled in by technical people who don’t know the data. They do it to tick the box, not to document reality. The result is documentation that says things like “IMP_SALDO: balance amount” (thanks, I could tell that from the field name) or “COD_PROD: product code” without explaining which product, from which system, or how it differs from COD_PRD in the table next door.

It’s governance just to get by. It ticks the checkbox. But it’s no use to anyone.

And there’s a third scenario, the most treacherous one: the one that almost works. On one project, someone did things right. They documented every table, every field, every transformation rule. They built a complete catalog in Confluence. It was flawless work. But they didn’t assign owners. They didn’t declare who was responsible for maintaining each definition. Six months later, that person moved to another project. Nobody knew who maintained the catalog. The definitions slowly went stale. A year later, the catalog was just another dead PDF — only prettier.

The problem wasn’t the documentation. It was that the documentation had no owner.

The recipe changes this at the root, for three reasons:

First, the recipe is written by whoever knows the data. Not a technician filling in a form, but the business analyst who knows why that field exists and what it means. The documentation is useful because it’s written by the person with the context.

Second, the recipe has an explicit owner: a team, a domain, a responsible person. If something doesn’t add up, you know who to ask. No more “this was built by someone who isn’t here anymore.” Data ownership is declared in the artifact itself.

Third, and this is the most important: governance isn’t an extra step. It isn’t a form you fill in after building the pipeline. It is the pipeline. The recipe IS the documentation, IS the mapping, IS the quality contract. You can’t create a pipeline without a recipe, and the recipe comes with governance built in. There’s no way to “skip the paperwork” because the paperwork is the artifact.

Governance stops being a toll booth that slows development down and becomes the infrastructure that enables it.

Hold on: isn’t this the same as a functional design?

If you’re a data architect, you’re probably thinking: “Okay, but this sounds like what we’ve always done. A functional design, a spec, a mapping document. What’s new about it?”

The difference is subtle but it changes everything: a functional design is read by a human. The recipe is read by a machine.

A functional design is a document a human interprets and translates into code. If they get it wrong, another iteration. If the design changes and nobody updates the SQL, drift. An executable recipe is an artifact a system consumes directly. There’s no interpretation. There’s no middle step. It’s like the difference between a road map and turn-by-turn GPS: the map informs you, the GPS takes you there.

The recipe is the first artifact in the history of data that is simultaneously specification, documentation, code and contract. They’re the same thing.

Back to my regulatory project

Remember the regulatory report where IT played detective for months?

Now that we’ve seen the six properties of the recipe, think about what would have changed:

Product would already have existed as a global entity in the catalog. Every source containing product data (contracts, billing, the legacy system) would have had its mapping declared in its own recipe: “My COD_PRD field in this system = the catalog’s Product.”

IT wouldn’t have played detective. When the regulatory report needs “Product,” the system already knows where it is in each source, what it’s called in each place, and how to normalize it. The report would have been assembled by crossing recipes, not by guessing.

And when something didn’t add up, we’d have known who to ask. The extra accounting entries? The accounting recipe has an owner. You ask them directly. The product that’s called something different in every system? The owner of the Product entity in the catalog is the one who resolves the ambiguity. Not an engineer searching Slack to see if anyone remembers.

On our project, half the iterations weren’t due to technical errors. They happened because we didn’t know who to ask. “Is this the field we need or is it the other one?” “Who knows why this account shows up twice?” “Who decided this filter should be applied this way?” Questions that bounced from email to email, meeting to meeting, for weeks. With an owner declared in every recipe, those questions get answered in minutes.

The extra or missing accounting entries would have been caught on day one. Because the recipe explicitly states which fields participate in the report. If one is missing, the system knows before generating anything. Not after someone builds a pipeline and the business says “this doesn’t add up.”

And the months of iterations over “this isn’t the product I need, it’s the other one” would have been minutes: you change the mapping in the recipe, the system regenerates, the business validates. Without waiting for IT to reinterpret a Word doc.

“Those who know finally can. And when they don’t know, they know who to ask.”

“So, are data engineers out of a job?”

If you’re a data engineer and you’ve made it this far, you’re probably thinking: “Hold on. If the business writes the recipe and the system generates everything automatically… what do I do?”

It’s the elephant in the room. And the answer is exactly the opposite of what it seems.

Back to the kitchen. When I say “the chef writes the recipe,” I’m not saying the restaurant doesn’t need engineers. Quite the opposite. Who designs the kitchen? Who installs the ovens, the walk-in coolers, the exhaust systems? Who makes sure the gas reaches every station, the ventilation works, the safety systems are in order?

The engineers. Without them, there’s no kitchen.

But — and this is the key — their job isn’t to cook. Their job is to make the kitchen work. So that any chef can walk in, read their recipe and execute it without worrying about whether the oven is calibrated.

In data, the story is exactly the same. The data engineer stops being the one who translates Word docs into SQL (a job that, let’s be honest, nobody enjoys and that’s beneath their level of expertise). And becomes the owner of the platform:

They build the kitchen. They design the infrastructure: how data is ingested, how it’s stored, how it’s processed.

They maintain the standards. They make sure one domain’s recipes are compatible with another’s. That the Shared Kernel is respected. That when Risk says “Customer” and Marketing says “Customer,” they’re talking about the same person.

They guarantee scalability. When the kitchen goes from serving 50 dishes a day to 500, the engineer is the one who expands capacity.

They evolve the platform. A domain needs a type of transformation the platform doesn’t support? The engineer builds it and adds it as a new capability all domains can use. Once. Not once per domain.

In other words: the data team goes from being the bottleneck (everyone depends on them for everything) to being the enabler (they build the platform, the domains build the products).

And there’s something that radically changes job satisfaction: it’s far more interesting work. Instead of spending your life translating ambiguous requirements (which, let’s be honest, is like being the interpreter at a never-ending trial where neither the prosecution nor the defense speaks clearly), you now design platforms, solve scalability problems, create reusable patterns and build systems that multiply the capacity of an entire organization.

For the CDO / CTO: the change that scales

If you have a team of 8 data engineers and today all 8 are hand-cranking SQL to serve the 120 requests in the queue, your capacity is 8. Linearly capped by team size.

If those same 8 engineers build and maintain the platform, and the 15 business domains create their own data products using that platform… your capacity no longer depends on the size of the data team.

It depends on how many domains get on the platform. And that scales in a completely different way.

As I said last week: manually modeling a source table in Data Vault 2.0 takes between 16 and 32 hours. With an executable recipe, it drops to 1–2 hours. But what matters isn’t the time saved: it’s that those 1–2 hours can be done by the business user. The dependency disappears.

The actuary doesn’t wait 3 months in a queue. She defines her recipe, the system executes it. No middlemen. No game of telephone. And the data team spends its time making the platform better every day.

Let’s run the numbers.

Say your company has 200 source tables that need modeling. With the traditional model (Word doc → interpretation → hand-written SQL → iterations → documentation): 200 tables × 24 hours on average = 4,800 hours. With a team of 8 engineers at 60% capacity (the other 40% goes to putting out fires, as Monte Carlo Data found): 4,800 ÷ (8 × 0.6 × 1,700 hours/year) ≈ 7 months. Just for the Raw Vault (the raw ingredients, organized exactly as they arrive from the source). Not counting the Business Vault (the mise en place: where they’re turned into business information), or quality, or iterations.

With executable recipes: 200 tables × 1.5 hours = 300 hours. And those hours can be done by the analysts in the 15 domains themselves, in parallel. The bottleneck disappears. The horizon goes from 7 months to weeks.

But the key isn’t the numbers for the first load. It’s what happens afterwards. Every new pattern the platform team implements, every new capability they add, multiplies the autonomy of every domain at once. It’s compound growth, not linear.

And there’s something else the numbers don’t capture: speed of reaction. When an urgent regulatory requirement lands (and one always does), the question is no longer “how long will IT take to do this?” but “how long will the business take to define its recipe?” And the answer is measured in hours, not months.

A benefit that isn’t obvious: knowledge survives

There’s a consequence of this model that isn’t visible at first glance but that, in the long run, is the most valuable of all.

Because the recipe is in business language and technology-agnostic, business knowledge survives technology migrations.

Think about how many times you’ve lost business knowledge in a migration. Those rules hardcoded into an Oracle stored procedure that nobody documented. Those transformations that only existed in an Informatica job the vendor set up 8 years ago. That solvency calculation buried on line 847 of a Python script that only one person understood, and that person no longer works at the company.

Every technology migration means starting from scratch.

Not because you change technology, but because the business knowledge was trapped inside the technology. Tangled up with the code. Indistinguishable from the implementation.

Here’s a concrete example. During an Oracle-to-cloud migration, someone discovers a 2,000-line stored procedure that calculates the technical provisions for a product. Nobody knows exactly what it does because the person who wrote it left 4 years ago. The code mixes business logic (how the provision is calculated) with infrastructure logic (how it’s optimized in Oracle, how it manages memory, how it accesses the tables). Which part is business and which part is technology? Nobody knows. And migrating that to Snowflake means rewriting it from scratch, with the risk of losing business rules that were implicit in the code.

If the logic had lived in an agnostic recipe, the migration would be: switch the generation target from Oracle to Snowflake. The recipe (with the definition of the provision, its rules, its validations) would remain intact. The engine would generate the new SQL for Snowflake. No rewriting. No guessing what each line of the procedure did.

When knowledge lives in the recipe and the recipe is agnostic, migrating from BigQuery to Snowflake (or from dbt to something else, or from a data warehouse to a lakehouse) is just swapping the oven. The recipe — with all the definitions, rules, mappings and identities the business and the analyst have built up over the years — is still there. Intact. Versioned. Auditable.

And this ties directly back to the Big Ball of Mud we talked about in Week 1. Why does technical debt pile up without anyone being able to clean it up? Because the knowledge is tangled up with the code, and touching the code is scary because you don’t know which business rule you’re breaking. When knowledge lives in the recipe and the code is generated from it, paying down technical debt is just swapping the oven. The knowledge stays intact.

Grandma’s recipe dies with her. The executable recipe is immortal.

What’s next: the pantry

Today we’ve put the first piece of the kitchen on the table: the Recipe. Who writes it (the business), in what language (theirs), how it implements the common language between domains (the Shared Kernel), and what changes for IT (from artisans to platform owners).

But a good recipe without ingredients is useless. And the ingredients — the raw data coming in from the source systems — need a place where they’re stored in an orderly, labeled and traceable way. In a professional kitchen, that place is the pantry.

Next week we’ll look at how a data warehouse’s pantry works. How data is stored exactly as it arrives, untouched, but organized so you can know exactly where each ingredient came from, when it arrived and who brought it. And why that’s the foundation of everything that comes after: auditing, reprocessing and automation.

But before we open the pantry: