Gaby wasn't built for Kaggle's BigQuery AI hackathon, but his objective lined up with most of the "inspiration" projects, so in he went. My real goal was to build more robustness into how I design projects.
Robust data-cleaning stages that adapt to any dataset, while letting reasoning agents steer themselves, balancing curious exploration (explore) with goal-driven strategies (greedy/exploit).
The demo had Gaby self-orchestrate cleaning a dirty café sales dataset, picked because real data is never as perfect as public data. livekit stood in for the client interface he doesn't have yet.
Checklist
- ✅ At least 3 BigQuery AI/ML methods
- ❌ A runnable Jupyter notebook. My Google Cloud and app development account was banned the day before submission.
- ✅ Linked blog, but no how-to video
I spent a week refactoring to meet the requirements, didn't set up my Google Cloud account correctly, and that probably provoked the ban. It took my Firebase-served frontend down with it.
The BigQuery AI/ML methods
Starting point: Google's Llama inference with BigQuery ML notebook.
| Function | Input | Output | How Gaby uses it | Action |
|---|---|---|---|---|
AI.GENERATE_TABLE | Text inputs containing data of any form, a data table, data fields to append | Data table | A text data summary can be reinserted and a schema built from it, to verify consistency over text / object data during cleaning | data cleaning |
AI.GENERATE_BOOL | Any prompt | Boolean | Checks whether something exists in a column field, handy for outlier detection | data cleaning, data analytics |
AI.GENERATE_DOUBLE | Any prompt | A double / float extracted from the prompt (and data column? to self-check) | Outlier / anomaly detection | data cleaning |
AI.GENERATE_INT | Any prompt | An integer extracted from the prompt (and data column? to self-check) | Outlier / anomaly detection | data cleaning |
Resources used
Self-feedback
- Setting up any tech stack needs far more time than I give it.
- I didn't plan enough.
- My sub-objectives were still too big to test in the timeframe. With prompt chains, the test cases that matter are the ones that still hold on a larger sample or similar situations.
- The good: I documented every step.
The original lifecycle had six stages, profiling through to a "trajectory path paver" for data science teams, which was way too broad. What I took away is that data wrangling is the hard part. Once that skeleton pipeline is solid, the other stages come much more easily, and whatever breaks after that usually comes from how the external integrations are built.
Next round: the AWS AI Agent Global Hackathon.