Web Data Collection & Analysis
A robust data collection and analysis ecosystem designed to support a long-term academic study.
Developed as part of an ongoing research role at the University of Canterbury. The system gathers, stores, and processes large volumes of online data with a focus on automation, reliability, and scalability.
Built on a Linux foundation and leveraging Python, MySQL, and a range of REST APIs, the platform efficiently ingests structured web data while handling rate limiting, proxy rotation, and failure recovery.
This work has deepened my expertise in backend systems, API-driven development, and data pipeline design, all within the context of meaningful real-world research.