HN CompanionHN Companion new | best | ask | show | jobs
How do you validate AI-generated data transformations before prod? (yorph.ai)
1 point by areddyfd 5 hours ago | 1 comment


The hard part about analyzing data isn’t generating a a series of analysis steps, it’s proving it’s correct.

Our current approach: - Sandboxed / sample runs on smaller datasets before full execution - Step-level transparency: summaries, intermediate tables, and generated code are all visible - Parallel and sequential test-time execution to surface inconsistencies - dbt-style pipelines for reproducibility and explicit dependencies - Decomposing analyses into small, verifiable steps to avoid error compounding (similar to MAKER-style approaches) - Online validation checks on intermediate and final outputs that trigger re-analysis when assumptions are violated - A gradually evolving semantic layer to improve consistency and governance over time

Curious how others think about this: what would make you trust an AI-driven data platform?