Rocky

Rocky is a trust system for data pipelines — a Rust-based control plane that brings branches, replay, column-level lineage, compile-time safety, and per-model cost attribution to your existing data warehouse. It works alongside Databricks or Snowflake, handling the DAG orchestration layer without requiring you to replace your warehouse. Rocky ships as a CLI binary, a Dagster integration, and a VS Code extension, and runs locally on DuckDB for zero-credential playground exploration.

• Schema drift detection: Rocky diffs source vs. target on every run, automatically dropping and recreating targets when upstream column types change — no silent data corruption.
• Compile-time data contracts: Missing required columns, protected column removals, or unsafe type changes surface as diagnostic codes (E010, E013) before a single row is written.
• Named branches: Create isolated schema branches for risk-free experiments, inspect results, then drop or promote — with column-level lineage showing downstream blast radius before shipping.
• Column-level lineage: Trace a single column from a downstream fact table all the way back to its seed, enabling precise blast-radius analysis without reading every model.
• AI model generation: Describe a transformation in plain English; Rocky generates a Rocky DSL model, compiles it, and retries automatically on parse failure.
• PR-time lineage diff: rocky lineage-diff compares two git refs and outputs per-changed-column downstream consumer readouts as Markdown, ready to drop into a GitHub PR comment.
• Classification, masking, and compliance: Tag PII columns in model sidecars, bind tags to mask strategies per environment, and gate CI with rocky compliance --fail-on exception.
• Incremental loads with watermark state: Use strategy = "incremental" with a timestamp_column to persist high-water marks and only process deltas on subsequent runs.
• Dagster integration: The dagster-rocky PyPI wheel wraps the Rocky CLI as a Dagster resource and component for orchestration workflows.
• VS Code extension: An LSP client and command palette for AI-assisted model generation and pipeline navigation directly in the editor.
• Adapter SDK: Build custom warehouse adapters (ClickHouse, Trino, Redshift, etc.) using the documented Rust-native adapter skeleton.