AI-Powered Product Catalog Matching for an International Food Distributor | BWT

AI-Powered Product Catalog Matching for an International Food Distributor

Technologies
Client
Confidential
Platform
Web
Duration
6 months
Industry
AI-Powered Product Catalog Matching for an International Food Distributor
87%
Top-1 Matching Accuracy
82%
Attribute Extraction Accuracy
10x
Less Time Spent on Manual Matching

Development of an AI-powered product catalog matching system that automatically matches items from a company's internal product database with external supplier catalogs. The solution combines LLMs, semantic search, product attribute extraction, and machine learning.

Services
  • AI System Development
  • Machine Learning Development
  • LLM Integration
Team
  • 1 Project Manager / AI Consultant
  • 3 AI Engineers
  • 1 QA Engineer
  • 1 Account Manager
Target Audience
  • Retailers and Distributors
  • Wholesale Trading Companies
  • Procurement and Supply Chain Teams

Challenge

Our client is a large international food distributor specializing in sourcing, importing, and distributing products from different countries.

The company manages an extensive product assortment and regularly analyzes external supplier and competitor catalogs to compare prices, monitor assortment changes, and identify product categories not yet covered by its portfolio.

Its internal product database contains approximately 10,000 SKUs. To support market analysis, the company needs to match these items against external catalogs and identify identical products or comparable alternatives.

The main challenge was the lack of standardized product descriptions across different catalogs. Even identical products could have significantly different names depending on the supplier.

Common inconsistencies included:

  • Different word orders, abbreviations, and additional product descriptors.
  • Product names written in different languages or transliterated.
  • Inconsistent product category structures.
  • Missing product attributes and specifications.
  • Different formats for brands, weights, volumes, and other characteristics.

For example, one supplier might list a product as "EVO Olive oil", while another uses "Olive Oil Extra Virgin." The system needed to recognize potential matches while distinguishing between products with different specifications, packaging sizes, or brands.

Product matching rules also varied by category. Some products required exact brand and attribute matches, while others could be matched with alternatives sharing similar characteristics.

Another challenge was the continuous updating of supplier catalogs. New suppliers could also introduce unfamiliar naming conventions and data formats.

The client needed an automated product catalog matching system capable of handling inconsistent product data, applying category-specific matching logic, and identifying uncertain matches for manual review.

Solution

We developed a hybrid AI product matching system combining LLMs, lexical and semantic search, product attribute extraction, and ML-based match classification.

Data Preprocessing and Normalization

The first step was to normalize product data from the internal database and external supplier catalogs.

The system cleans product names, removes unnecessary characters, standardizes formatting, and expands abbreviations. An LLM handles context-dependent abbreviations based on the product category.

For example, the Russian abbreviation "ПС" in a wine description may stand for "semi-dry," while "ПСЛ" refers to "semi-sweet."

Both original and normalized product names are retained for subsequent matching to avoid losing potentially relevant information.

Hybrid Product Search

For each item in the internal product database, the system identifies potential matches from external supplier catalogs.

The search combines two approaches:

  • Lexical Search identifies candidates based on text similarity, matching keywords, and product name patterns.
  • Semantic Search uses a locally deployed embedding model and the Qdrant vector database to identify products with similar meanings, even when their descriptions differ.

Combining these methods allows the system to find potential matches despite differences in wording, language, and naming conventions.

The resulting candidate list is passed to the next stage for attribute-based comparison.

LLM-Based Product Attribute Extraction

To improve matching accuracy, we implemented automated product attribute extraction using an LLM.

The system analyzes product descriptions and extracts structured attributes, including:

  • Product category
  • Brand
  • Weight and volume
  • Fat content
  • Alcohol content and beverage type
  • Country of origin
  • Other category-specific characteristics

The solution uses a set of 28 attributes covering different product categories.

These extracted attributes support facet-based product matching, allowing the system to compare products based on their characteristics rather than relying solely on name similarity.

For example, two products may have similar names but differ in fat content, country of origin, or packaging volume. Attribute-based comparison helps identify such differences and distinguish valid matches from unsuitable alternatives.

ML-Based Product Matching and Confidence Scoring

We developed a machine learning classifier based on gradient boosting to evaluate potential product matches.

The model considers multiple matching signals:

  • Lexical similarity between product names.
  • Semantic similarity.
  • Brand and category matches.
  • Product attribute consistency.
  • Conflicting attributes between compared products.

Based on these features, the classifier calculates a matching confidence score for each product pair.

The system then categorizes matches into three groups:

  • Match: High-confidence product matches.
  • Manual Review: Uncertain matches requiring human validation.
  • No Match: Products that do not meet the matching criteria.

This approach enables automated processing of high-confidence matches while directing ambiguous cases to specialists for review.

Incremental Catalog Updates

To optimize the processing of frequently updated supplier catalogs, we implemented an incremental catalog update mechanism.

Instead of reprocessing the entire product database whenever a supplier updates its catalog, the system detects changes and processes only the affected products and their existing matches.

When an existing product is modified, the corresponding matching scores are recalculated. Newly added products are matched against relevant items in the internal database, while removed products are excluded from the matching results.

This approach is particularly useful for large catalogs where only a small percentage of products change each month.

Incremental processing reduces the number of computations and LLM API calls, lowering token consumption and processing costs while accelerating catalog updates.

User Interface and Manual Validation

We also developed a web-based interface for managing supplier catalog matching and reviewing the results.

The interface allows users to:

  • Upload internal and external product catalogs.
  • Select columns containing product names.
  • Run product data normalization and attribute extraction.
  • Start automated product matching.
  • Review suggested matches and confidence scores.
  • Validate or correct uncertain matches.
  • Export matching results.

Catalog processing runs in the background, allowing users to work with large datasets without continuously monitoring execution.

Result

We developed an AI-powered product matching system that automates the identification of corresponding products across the client's internal database and external supplier catalogs.

During testing, the solution achieved the following results:

  • 87% Top-1 Matching Accuracy: The correct match ranked first in approximately 80% of evaluated test cases.
  • 82% Attribute Extraction Accuracy: LLM-extracted product attributes matched the manually annotated reference data in approximately 82% of cases.

By combining hybrid search, LLM-based attribute extraction, and ML classification, the system can process product catalogs with inconsistent naming conventions, languages, and levels of detail.

Incremental catalog updates optimize recurring processing by focusing on modified products rather than reprocessing entire catalogs, reducing LLM API calls and associated costs.

The web interface provides a centralized workflow for catalog uploads, automated product matching, and manual validation of uncertain results.

The solution provides a foundation for further automation of assortment analysis, supplier product comparisons, and the identification of new product categories.

Contact Us

Let's Work Together!

Do you want to know the total cost of development and realization of the project? Tell us about your requirements, our specialists will contact you as soon as possible.

Please fill in the 'Name'
Please fill in the 'Phone'
Please fill in the 'Email'
Please fill in the 'Message'