Your dbt Core implementation may be working exactly as intended.
But as your organisation grows, the way you use and manage dbt can change significantly.
More users, models, jobs and business dependencies can turn a relatively simple dbt Core environment into one that requires considerable effort to operate and maintain.
At this point, organisations may start considering whether moving from dbt Core to dbt Cloud*, now referred to as self-hosted dbt and the dbt platform, could better support their growing requirements.
So how do you know when it is time to reconsider your current approach?
7 Signs You’ve Outgrown Your dbt Setup
1.
Your dbt environment has grown beyond its original design
Many dbt implementations start relatively simply: a small team, a manageable number of models and straightforward workflows.
As adoption grows, so does the environment around it. More data sources, models, jobs, environments and dependencies can introduce complexity that the original setup was never designed to accommodate.
2.
More people are contributing to dbt
Growth often means more engineers and analysts working across the same dbt environment.
That increases the importance of consistent development practices, permissions, testing, documentation and clear separation between development and production.
Processes that worked well for a small team can become harder to coordinate as more people contribute.
3.
Jobs and dependencies are becoming more complex
As the number of models and workflows increases, orchestration becomes more important.
Dependencies need to run in the right order, failures need to be identified quickly and teams need visibility into how changes affect downstream data.
If managing these dependencies is becoming increasingly complicated, it may indicate that the operating model needs to evolve.
4.
Deployments require more coordination
Deploying a change becomes more consequential when more systems and people rely on the resulting data.
Teams may need stronger CI/CD processes, testing and controls to ensure changes can move into production reliably without disrupting downstream workloads.
5.
More of the business depends on dbt
What began as a transformation tool for the data team may now underpin dashboards, executive reporting, operational processes, data products or AI applications.
As dbt becomes more business-critical, reliability, governance and visibility become increasingly important.
6.
Your team spends more time running dbt
Consider how much engineering effort now goes into maintaining everything around dbt.
Orchestration, infrastructure, deployment workflows, access, monitoring, upgrades and troubleshooting can all consume engineering capacity.
At some point, it is worth asking whether your team should be spending this time running dbt or working with dbt.
7.
Every new requirement adds more complexity
Perhaps the clearest warning sign is when scaling means continually adding another script, integration, process or piece of infrastructure.
Each addition may solve an immediate problem, but collectively they can increase maintenance requirements and create dependencies that become harder to manage over time.
Has Your Operating Model Kept Pace?
Outgrowing your existing approach does not mean your dbt implementation has failed. It can be a consequence of successful adoption.
The important question is whether the way you operate dbt still reflects what your organisation needs today.
If your environment is becoming more complex, consuming more engineering capacity or supporting increasingly critical workloads, it may be time to assess whether your current approach is still the right fit.
Helping Organisations Get More From dbt
Data Army is a dbt Visionary Consulting & Services Partner, the highest tier of partnership within the dbt Partner Program. As a dbt-certified organisation, we bring deep expertise in analytics engineering, implementing and optimising dbt for our clients, and helping organisations advance their modern data stack with scalable, trusted data practices.
We also use dbt extensively within Data Army’s own data environment, so our team has first-hand experience of the platform and the same practical considerations our clients encounter when implementing, managing and scaling dbt.
Considering the Move to dbt Cloud?
See how we’ve helped others make the move, or learn how to manage your own migration to dbt Cloud.
See a dbt Cloud migration in action
See how Data Army helped Australian digital home loan platform and financial technology company, Lendi, migrate to dbt Cloud, creating a trusted data foundation for its AI-native future.
How to migrate to dbt Cloud
Our practical guide steps through how to move to dbt Cloud while managing dependencies, reducing disruption and taking the opportunity to build a more scalable, easier-to-manage environment.
*dbt Labs now refers to what many teams know as dbt Core as self-hosted dbt, while dbt Cloud is now the dbt platform.
You can read more about the distinction between self-hosted dbt and the dbt platform here.