Fivetran Cost and Schema Headaches? How Automated Modeling Fixes Ingestion Friction
Plug and play ELT is fast to start and slow to control. Two things erode over time: what the pipeline costs, and who decides the shape of the data. An automated Data Vault layer gives both back without giving up the connectors that work.
The data warehouse automation solution trusted by data teams across industries
The first Fivetran month is a good month. Connectors are live in an afternoon, the warehouse fills up, and the team gets on with building. The invoice that lands after the first ERP migration is when the conversation changes, and the schema conversation usually starts in the same quarter.
The trade that plug and play ELT makes
Managed ingestion trades two things for convenience, and neither shows up on day one.
- Cost predictability. Monthly Active Rows meter what changes, and what changes is decided by source system teams running migrations and bulk updates. The bill tracks their project calendar, not your budget. The detail is in the article on MAR pricing.
- Schema control. Each connector lands its own standardized schema. Conformed keys, custom fields and cross source rules have to be added on top, and break when the source changes. The detail is in the article on fixed schemas.
Both are the same underlying issue. The tool that moves the rows has also been made responsible for what they cost and what shape they take, and it was only ever designed for the first job.
Two ways out, and you can take both
Take ingestion in house for the tables that hurt. Datavault Builder includes batch, delta and CDC loads from databases, files, REST APIs, NoSQL and Python sources, with streams such as Kafka arriving as micro batches. The handful of tables that carry most of the MAR land through the platform, on compute you already own. The long tail of SaaS sources stays on the connector where it is cheap.
Put a model you own behind everything. Whatever lands in staging, by connector or by direct load, is mapped visually into a Data Vault 2.0 model: hubs for business keys, links for relationships, satellites for context and history. The loading code is generated from that model. A source change is a mapping change, and the business vault and marts above it do not move.
What that changes in practice
- Reloads stay in staging. Only the delta loads into the vault, so a full reload upstream is not a full reload of every layer.
- Customer is one hub, not three tables. Cross source keys are harmonized in the model, once, with lineage.
- Rules have an address. Business logic lives in the business vault or the mart layer, visible in the model, not scattered across scripts and notebooks.
- The BI layer sees a star schema. Dimensional marts are generated on top, so nothing about the vault reaches a report author.
- Cost is a sizing decision. Warehouse compute is measurable and reservable. Row activity in someone else’s system is not.
Where to start
Two lists, one afternoon. The MAR report sorted by table tells you which sources to move to direct ingestion. The inventory of post-load scripts that exist only to reshape connector output tells you what the model has to contain. Both lists are usually shorter than expected, and both are the whole project.
See It Running on One of Your Sources
Book a free demo and bring the connector that costs you the most, in money or in time.
How Datavault Builder Takes the Friction Out of Ingestion
-
Ingestion is built in
Batch, delta and CDC loads from databases, files, REST APIs, NoSQL and Python sources, with streams such as Kafka arriving as micro-batches. Same platform that generates the warehouse, no second invoice.
-
Your schema, not the vendor's
Source tables are mapped to a Data Vault 2.0 model you designed. A new column or a renamed table changes a mapping, not a chain of post-load scripts.
-
Only deltas move
Hubs, links and satellites load what changed. Full reloads stay in staging instead of being reprocessed downstream every night.
-
History is kept by design
Every change is retained as it arrives, so as-was reporting works even where the source overwrites its own rows.
-
Code you never hand-write
Loading, historization and lineage are generated from the model in real time and run natively on Snowflake, Databricks, BigQuery, SQL Server, Fabric, Oracle or PostgreSQL.
-
One platform, up to nine tools fewer
Modeling, ETL, CI/CD, documentation and lineage in one place. That is what makes 14.7 minutes from requirement to production possible.
Recognized by BARC in The Data Fabric Survey 26
Meet Our Expert
Twenty minutes with our Sales Director, and an honest answer on whether this fits your stack.
Matt Collett
Sales Director
Great, pick a time that works for you:
Other Problems This Series Covers
-
Is Your Fivetran Schema Forcing You to Rebuild the Model After Every Load?
Fivetran lands each source in its own standardized schema. Business keys, custom fields and legacy structures then have to be reshaped in SQL that breaks when the connector changes. The fix is not a better script. It is a model that owns the shape.
-
Does Your Fivetran Bill Jump Every Time a Source Table Gets Busy?
Fivetran charges by Monthly Active Rows. A bulk update or a schema migration upstream touches every row again, and the invoice follows. The rows were never the problem. Paying per row for data you then reprocess yourself is.
Questions and Answers
- No. It is an argument to stop letting the connector decide your cost and your schema. Keep it where it is cheap and convenient, move the expensive tables to direct ingestion, and put a model you own behind both.
- The landing tables already exist, so the work is modeling, not extraction. The 14.7 minutes from requirement to production figure refers to adding a change once the model is in place; the initial mapping of a source is measured in days, not quarters.