Pipeline Explorer
Click a step
Extract
Validate
Load
Transform
Extract: read CSV from data/raw
# ingestion/load_to_bigquery.py (excerpt)
df = pd.read_csv(csv_path)
df.columns = (
df.columns.astype(str).str.strip().str.lower()
.str.replace(r"[^a-z0-9]+", "_", regex=True).str.strip("_")
)
df = df.dropna(how="all")