Statline: Local NFL Lakehouse

2025 · GitHub

Local medallion pipeline for historical NFL data. Python ingest from nflreadpy lands in DuckLake bronze; dbt builds silver staging and gold star-schema marts (player-game and team-game facts); Dagster orchestrates bronze → silver → gold as local assets.

I kept rebuilding the same public NFL sources for hobby models. The sources were fine; the glue wasn't. Statline is the pipeline so the data is already in a clean, queryable place when I need it.

Stack (and the pivot)

Originally scoped for Databricks + Unity Catalog + Delta. One sport and ~25 seasons of box scores does not need distributed compute, so it stayed on a single-machine stack:

• Extract: Python + nflreadpy
• Storage: DuckLake (SQL catalog + Parquet)
• Transform: dbt + dbt-duckdb (7 silver staging views, 5 gold dimension and fact tables)
• Orchestration: Dagster local assets (dagster-dbt)

Data model (gold)

• fact_player_game: one row per player per game
• fact_team_game: one row per team per game
• dim_player, dim_team, dim_game

Bronze stays source-shaped. Silver is clean/rename/filter. Gold is the star schema for analysis and a sports app.

Done: bronze ingest, silver staging, gold marts, local Dagster. Not built yet: live game feeds, multi-sport.