WhereDidISpend

Most expense trackers fail because they require manual input for every transaction. In India, where UPI dominates payments, I realized I could eliminate that friction entirely: just forward your UPI screenshot to WhatsApp or Telegram, and the app handles the rest.

The Hard Problem

UPI screenshots aren’t standardized -- different banks, different layouts, different languages. Extracting structured data (amount, recipient, date) from these images reliably was the core challenge. On top of that, the system needed to process messages in near real-time without blocking or losing data.

How I Built It

I designed this as a microservice system in Nest.js. One service listens to WhatsApp Business and Telegram webhooks, downloads the screenshot, and feeds it to Google Gemini with carefully engineered prompts to extract transaction data. Parsed results go into a Redis-backed queue, where a second service picks them up and updates the user’s account. This decoupled design means a spike in incoming messages won’t crash the processing pipeline.

The Flutter mobile app and React web app both consume the same REST API. The web version adds spending visualizations with charts and graphs so users can spot trends over time.

Impact

Built the entire system end-to-end as a solo developer. The AI extraction pipeline handles varied screenshot formats with high accuracy, and the queue-based architecture keeps processing resilient under load. This project deepened my understanding of prompt engineering, event-driven architecture, and building products that solve a real daily annoyance.