Is Your Fivetran Schema Forcing You to Rebuild the Model After Every Load?
Fivetran lands each source in its own standardized schema. Business keys, custom fields and legacy structures then have to be reshaped in SQL that breaks when the connector changes. The fix is not a better script. It is a model that owns the shape.
The data warehouse automation solution trusted by data teams across industries
Does this sound familiar?
- Every connector lands its own schema, and the joins between them are written by hand, per source, in post-load SQL.
- A column added or renamed on the source side breaks a transformation a week later, in production.
- Customer, product and account exist three times with three different keys, and every developer reconciled them in their own style.
- The real business logic is scattered across dbt models, views and notebooks, and nobody can point to where a rule is defined.
A standardized schema per connector is what makes a managed pipeline quick to set up. It is also what makes the second month harder than the first. The shape of the data is decided by the connector, and everything your business actually needs, conformed keys, custom fields, rules that span three sources, has to be built on top, in dbt or by hand.
Why the fixed schema fights you
- The schema is the vendor’s, not yours. It reflects the source system’s API, not the way the business talks about customers, orders or accounts.
- Post-load SQL is the real model, and it is unmanaged. Key harmonization, deduplication, history handling and rules end up in scripts nobody designed as a whole, each in the style of whoever wrote it.
- Source changes arrive as production incidents. A renamed column upstream is a broken transformation downstream, discovered only when an executive report fails or shows the wrong numbers.
- The same entity lands in silos. Customer from the CRM, customer from the ERP and customer from the web shop are three tables with three keys until somebody writes the join.
Nothing here is a connector fault. It is what happens when the tool that lands the data is also expected to give it a shape the business can use.
Where the shape belongs
- Shape is a modeling decision, not a scripting one. Business keys, relationships and attributes should be declared once, in a model, and the loading code derived from it.
- The raw layer should absorb change. Data Vault 2.0 separates keys, relationships and context precisely so that a new source attribute is a new satellite, not a rewrite.
- Rules belong above the raw layer. Conformed dimensions and business logic sit in the business vault and the marts, where a source schema change cannot reach them.
What changes with Datavault Builder
Datavault Builder replaces the fixed target schema with a model-driven Data Vault 2.0 layer, and generates the loading code from the model.
- Landing tables stay as the connector delivers them. Nothing is reshaped by hand. The mapping from landing table to hub, link and satellite is done visually.
- Business keys are harmonized in the model. Customer from three systems forms one hub, with each source’s context in its own satellite. The join is declared once.
- A source change is a mapping change. Update the mapping, regenerate, and the business vault and marts above it are untouched.
- Small changes heal themselves. A column that goes missing is loaded as null with a warning, not a failed run. A varchar that grows is widened, not truncated.
- Every pipeline has the same shape. The loads are generated, so there is no per developer style to review, and no script that only its author understands.
- Documentation and lineage are generated too. Every rule sits in the business vault or the mart layer, is visible in the model, and can be traced from report to source without anyone writing it up.
- The output is still a star schema. Dimensional marts are generated on top, so the BI layer sees conformed dimensions, not vault internals.
What to decide
Audit the post-load scripts that exist only to reshape connector output: key harmonization, dedupe, history, cross source joins. That list is the model you have been maintaining by hand. It is the first thing to move.
See It Running on One of Your Sources
Book a free demo and bring the connector that costs you the most, in money or in time.
Three Steps to a Pipeline You Control
-
Keep the landing schema as it is
Whatever the connector produces stays in staging untouched. Nothing is reshaped by hand.
-
Map it to the model you own
Business keys, relationships and attributes are mapped visually into hubs, links and satellites.
-
Put rules in the business vault
Conformed dimensions and custom logic sit above the raw vault, where a source change cannot reach them.
How Datavault Builder Takes the Friction Out of Ingestion
-
Ingestion is built in
Batch, delta and CDC loads from databases, files, REST APIs, NoSQL and Python sources, with streams such as Kafka arriving as micro-batches. Same platform that generates the warehouse, no second invoice.
-
Your schema, not the vendor's
Source tables are mapped to a Data Vault 2.0 model you designed. A new column or a renamed table changes a mapping, not a chain of post-load scripts.
-
Only deltas move
Hubs, links and satellites load what changed. Full reloads stay in staging instead of being reprocessed downstream every night.
-
History is kept by design
Every change is retained as it arrives, so as-was reporting works even where the source overwrites its own rows.
-
Code you never hand-write
Loading, historization and lineage are generated from the model in real time and run natively on Snowflake, Databricks, BigQuery, SQL Server, Fabric, Oracle or PostgreSQL.
-
One platform, up to nine tools fewer
Modeling, ETL, CI/CD, documentation and lineage in one place. That is what makes 14.7 minutes from requirement to production possible.
Recognized by BARC in The Data Fabric Survey 26
Meet Our Expert
Twenty minutes with our Sales Director, and an honest answer on whether this fits your stack.
Matt Collett
Sales Director
Great, pick a time that works for you:
Other Problems This Series Covers
-
Fivetran Cost and Schema Headaches? How Automated Modeling Fixes Ingestion Friction
Plug and play ELT is fast to start and slow to control. Two things erode over time: what the pipeline costs, and who decides the shape of the data. An automated Data Vault layer gives both back without giving up the connectors that work.
-
Does Your Fivetran Bill Jump Every Time a Source Table Gets Busy?
Fivetran charges by Monthly Active Rows. A bulk update or a schema migration upstream touches every row again, and the invoice follows. The rows were never the problem. Paying per row for data you then reprocess yourself is.
Questions and Answers
- It replaces the hand-written part: the staging models, key harmonization and historization that every team ends up writing. The generated vault and marts are ordinary tables and views in your warehouse, so anything you keep in dbt can read from them.
- The landing table changes, the mapping is updated in the model, and the loading code is regenerated. Nothing downstream of the raw vault is touched, because the business vault and marts read the model, not the source.
- The raw layer is Data Vault 2.0, which is what makes it absorb source changes. On top of it Datavault Builder generates dimensional or 3NF marts, so what the business sees is a star schema.