PythonSeleniumMySQLAutomation

Annual Procurement Plan Parser

A data-enrichment service that hunts down missing identifiers for 1,000+ procurement records against a live portal — achieving a 99%+ success rate through two-tier search and confidence-gated matching.

Overview

Procurement records often arrive incomplete, missing the identifiers needed to link and analyse them — and filling those gaps by hand across thousands of rows is infeasible.

This service automates the enrichment: it reads incomplete records, searches an external portal to locate the missing data, disambiguates the matches, and writes verified records back — with an audit trail for anything uncertain. It is a backend-only microservice of EZBid, a procurement-intelligence platform.

What I Built

Built in Python with headless Selenium and a dual-database MySQL setup, containerised with Docker. The architecture strictly separates a read-only source from a write-only destination.

Two-tier search with fallback — precise filters first, then broader filters, with exponential backoff.

Intelligent disambiguation — progressive narrowing, confidence scoring, and quality gates that block uncertain inserts.

Dual-database architecture — strict read-only source and write-only destination separation for safety.

Portal navigation resilience — multi-level menus, dynamic content detection, and selector resilience.

Production reliability — headless execution, process locking, comprehensive logging, and audit trails.