Finished · 100%
Product Feed Toolbox
A local web tool I built to help validate, convert, compare, prettify, and publish large JD Sports product feeds for testing.
Overview
- Status
- Finished
- Progress
- 100%
- Difficulty
- Challenging
- Started
- Jul 26, 2026
Stack & tools
01 / MOTIVATION
Why I built this
I started this because I wanted to help test the backend team’s product feed output. The feed can contain more than 80k products, so checking products one by one was never realistic. I needed a way to quickly spot potential issues against the expected requirements and documentation. At first, the task sounded simple: rename and prefix a few variant attributes. But once I started using the feed for QA, the tool naturally grew into something broader.
02 / QUESTION TO EXPLORE
What I wanted to understand or solve
Large product feeds are difficult to review manually. Even small formatting or naming differences can create indexing issues, frontend bugs, or confusing QA results. The feed also has multiple modes. A full feed behaves differently from a delta feed, and delta feeds can use add, update, or delete operations. That means the tool needed to understand both structure and intent. I also needed reports that were readable. A raw JSON diff is technically useful, but not friendly when the actual question is “which SKU changed, what changed, and should I care?”
03 / APPROACH
How I turned the idea into something testable
I began with a small Python script that transforms variant attributes. It keeps variantId unprefixed, changes variantID into variantId, adds v_ to other variant-level attributes, and removes known unnecessary parent or variant fields. Then I turned it into a local web app so I could upload a JSON file, choose the feed type, select export format, and download the converted output without running a command manually. The app later grew into separate tabs: Convert, Prettify, Validate, and Compare. That separation made the tool easier to use depending on the task. For Compare, I moved away from a generic “before vs after” idea and made it more flexible: File 1 and File 2 can represent different systems or snapshots. The report uses uniqueId like a lookup key, then groups differences by product, variant, price, availability, categoryPath, and schema-level changes. The tool also includes UNBXD publishing support with feed type routing and status polling, because the upload response alone does not confirm indexing success.
04 / FIELD NOTES
It started as feed cleanup
The first version was focused on one specific requirement: attributes under variants needed a v_ prefix, except variantId. There was also a case-sensitive naming issue where variantID had to become variantId.
That was the kind of detail that is easy to miss when looking at a huge feed manually, but important enough to break downstream processing.
Validation became just as important as conversion
Once the converter worked, I realized that changing the feed was only part of the problem. I also needed to know whether the feed was already valid before exporting or publishing it.
The validation flow grew to check things like wrong variant key prefixes, incorrect variantId casing, invalid variants containers, missing or blank values, and collisions between uniqueId and variantId.
Compare needed to be readable
A normal JSON diff is not enough for product feed QA. I needed something closer to a QA report: product counts, variant counts, product added or missing, attribute changes, datatype differences, price changes, availability changes, and categoryPath differences.
The Compare tab became more structured over time. Price-related fields were grouped into one row per SKU. Availability also became its own section, showing product-level availability and variant-level availability in a way that is easier to scan.
Publishing required patience
Publishing to UNBXD added another workflow problem. The upload endpoint can accept a file, but the real indexing result comes later from the status endpoint.
The tool therefore includes polling and local logs, so uploads can be tracked with timestamps instead of relying on a single immediate response.
05 / LEARNING
This project reminded me that QA tools do not always need to start as big systems. Sometimes the useful version begins as a small script for one annoying format rule.
The more I used it, the more I understood that the real value was not only automation. It was making a large feed reviewable by turning noisy JSON into grouped, human-readable reports.
I also learned that publish workflows need defensive design. A successful upload response is not the same as a successful indexing result, especially when only one indexing job can run at a time.
LET'S TALK
Something worth talking about?
Product, systems, something you're building, or just an idea worth comparing notes on—feel free to reach out.