Does Your dbt Project Have More Models Than Anyone Can Explain?

dbt, now one company with Fivetran, made creating a model trivial. select * from ref('...') and a new node exists. That is the feature, and over two or three years it is also how a project grows into hundreds of models with lineage nobody can hold in their head. When creating code has no friction, accumulation is the default outcome.

Why projects sprawl

• Forking is cheaper than changing. dbt has macros and packages for the standard patterns, but any developer can edit them, and changing a shared one properly means regression tests and migration scripts. So a pattern that does not quite fit gets a second version instead.
• Structure and logic live in the same files. A model that stages a source, one that handles history, and one that computes margin all look the same in the DAG.
• Dependencies are declared, not designed. ref() records that a model reads another; it does not say why, or what breaks if the upstream key changes.
• Governance arrives after the fact. Naming conventions, folders and tests are applied to a project that was already large by the time anyone wrote them down.

The tooling is not at fault. A framework that makes SQL modular will be used to write a lot of modular SQL.

Where the structure belongs

• Most models are structural. Staging, deduplication, key generation, history tracking: the same pattern per source, differing only in column names.
• Structural code should be generated, not accumulated. Declare the business key, the relationships and the attributes once, and derive the loading code from a pattern that no project edits.
• Logic is the part worth writing by hand. Business rules, metrics and mart shapes are a small share of the project and the part that deserves a code review.

What changes with Datavault Builder

Datavault Builder holds the structure in a visual model and generates hubs, links, satellites and their loads from it, so the repeated part of the project stops being files.

• One definition per concept. A customer hub with five sources is one hub and five mappings in the model, not fifteen models in a folder.
• Dependencies are derived. The load order and lineage follow from the model, so an upstream key change is visible before it is deployed, not after.
• Metadata travels through the layers. Types, keys and descriptions are declared once and carried from staging to the marts, instead of being repeated in every model file.
• Refactoring is a model edit. Change the mapping, regenerate, and every dependent structure is rebuilt consistently.
• The documentation is the model. Lineage, definitions and history rules are readable in one place, without a docs build.
• Business rules are managed in the model, with versions. The delivery layer is built by drag and drop on the semantic layer, so rules and data products have one home and a history.
• dbt can stay, generated. If operations remain in dbt, the dbt models are generated from the same model and regenerated on change, which is a much simpler way to update them than editing hundreds of files.

What to decide

Audit the project and sort each model by what it does: stages, keys, historizes, or computes. If the first three are most of the list, that is the structural share, and it is the share a generator should own.