What I built

  • Built the whole data pipeline on my own. It scrapes about 48 official sources, reads scanned PDFs with OCR, and has nearly 2,900 tests.
  • Made search hybrid: Japanese full-text search plus vector search, both in Postgres.
  • Wrote the original AI server that answers from those sources (retrieval-augmented generation, or RAG). It first ran a self-hosted Gemma model on a GPU; I moved it to a hosted Qwen model.
  • Tracked down two production outages caused by the pipeline's own database load, and fixed them by pacing its jobs.

More projects