Data6 min read

dbt (Data Build Tool)

Transform raw warehouse data into reliable, tested, documented analytics tables - using SQL models, tests, and version control with dbt.

dbtData TransformationSQLData WarehouseAnalytics Engineering

skill: dbt (Data Build Tool) category: Data tags: [dbt, Data Transformation, SQL, Data Warehouse, Analytics Engineering] readingTime: 6

dbt is the standard analytics engineering tool - it brings software engineering practices (version control, testing, documentation) to SQL transformations. If your data team writes transformation SQL outside of dbt, they're creating untested, undocumented technical debt. dbt works with any SQL warehouse: BigQuery, Snowflake, Redshift, PostgreSQL, DuckDB.

Workflow

Set up dbt with a data warehousedbt connects to your warehouse and runs SQL models there - it's not a data movement tool, just a transformation orchestrator.
Write SQL models with ref()ref() creates a DAG of dependencies between models - dbt runs them in the correct order and handles schema references automatically.
Add data quality testsTests run after each model build and fail loudly when data doesn't meet expectations - catching issues before they reach dashboards.
Use incremental models for large tablesRe-processing billions of rows on every run is slow and expensive. Incremental models only process new or changed rows.
Document and generate docsdbt auto-generates a documentation site from model descriptions and column comments - the first time a data team has automatically maintained docs.

Output Format

dbt project: staging models → mart models, schema tests, incremental materialization, and auto-generated docs.

models/
├── staging/           # Light cleanup of source tables
│   ├── stg_orders.sql
│   └── stg_users.sql
├── intermediate/      # Business logic joins
│   └── int_order_items.sql
└── marts/             # Final fact + dim tables for BI
    ├── fct_orders.sql
    └── dim_users.sql

schema.yml: descriptions + tests for all models

Examples

Input

Output

Edge Cases & Gotchas

Model naming collisions - Two models with the same name in different subdirectories produce a collision. Use a naming convention: stg_ for staging, int_ for intermediate, fct_/dim_ for marts.
Incremental full refresh - Incremental models can drift from full-rebuild results over time. Run dbt run --full-refresh on a schedule (weekly) to rebuild from scratch and validate.
Long-running tests - relationship tests on billion-row tables run for hours. Use where clauses in tests or run tests only on recent data: dbt test --select fct_orders --vars '{start_date: 2026-08-01}'.
Seeds for reference data - Small static CSVs (country codes, currency symbols) can be loaded as dbt seeds: dbt seed → creates a table from the CSV in the warehouse.

When to Break These Rules

Use Python dbt models (dbt 1.3+) for transformations that require pandas, ML inference, or operations not expressible in SQL - run Python directly in Snowflake/BigQuery.
Use Spark/PySpark for transformations that need to run on data too large for a single warehouse node or require complex ML pipelines within the transformation step.

Reference Files

https://docs.getdbt.com/
https://docs.getdbt.com/docs/build/incremental-models