Development of an AI-powered product catalog matching system that automatically matches items from a company's internal product database with external supplier catalogs. The solution combines LLMs, semantic search, product attribute extraction, and machine learning.
Our client is a large international food distributor specializing in sourcing, importing, and distributing products from different countries.
The company manages an extensive product assortment and regularly analyzes external supplier and competitor catalogs to compare prices, monitor assortment changes, and identify product categories not yet covered by its portfolio.
Its internal product database contains approximately 10,000 SKUs. To support market analysis, the company needs to match these items against external catalogs and identify identical products or comparable alternatives.
The main challenge was the lack of standardized product descriptions across different catalogs. Even identical products could have significantly different names depending on the supplier.
Common inconsistencies included:
For example, one supplier might list a product as "EVO Olive oil", while another uses "Olive Oil Extra Virgin." The system needed to recognize potential matches while distinguishing between products with different specifications, packaging sizes, or brands.

Product matching rules also varied by category. Some products required exact brand and attribute matches, while others could be matched with alternatives sharing similar characteristics.
Another challenge was the continuous updating of supplier catalogs. New suppliers could also introduce unfamiliar naming conventions and data formats.
The client needed an automated product catalog matching system capable of handling inconsistent product data, applying category-specific matching logic, and identifying uncertain matches for manual review.
We developed a hybrid AI product matching system combining LLMs, lexical and semantic search, product attribute extraction, and ML-based match classification.

The first step was to normalize product data from the internal database and external supplier catalogs.
The system cleans product names, removes unnecessary characters, standardizes formatting, and expands abbreviations. An LLM handles context-dependent abbreviations based on the product category.
For example, the Russian abbreviation "ПС" in a wine description may stand for "semi-dry," while "ПСЛ" refers to "semi-sweet."
Both original and normalized product names are retained for subsequent matching to avoid losing potentially relevant information.
For each item in the internal product database, the system identifies potential matches from external supplier catalogs.
The search combines two approaches:

Combining these methods allows the system to find potential matches despite differences in wording, language, and naming conventions.
The resulting candidate list is passed to the next stage for attribute-based comparison.
To improve matching accuracy, we implemented automated product attribute extraction using an LLM.
The system analyzes product descriptions and extracts structured attributes, including:
The solution uses a set of 28 attributes covering different product categories.

These extracted attributes support facet-based product matching, allowing the system to compare products based on their characteristics rather than relying solely on name similarity.
For example, two products may have similar names but differ in fat content, country of origin, or packaging volume. Attribute-based comparison helps identify such differences and distinguish valid matches from unsuitable alternatives.
We developed a machine learning classifier based on gradient boosting to evaluate potential product matches.
The model considers multiple matching signals:
Based on these features, the classifier calculates a matching confidence score for each product pair.
The system then categorizes matches into three groups:

This approach enables automated processing of high-confidence matches while directing ambiguous cases to specialists for review.
To optimize the processing of frequently updated supplier catalogs, we implemented an incremental catalog update mechanism.
Instead of reprocessing the entire product database whenever a supplier updates its catalog, the system detects changes and processes only the affected products and their existing matches.
When an existing product is modified, the corresponding matching scores are recalculated. Newly added products are matched against relevant items in the internal database, while removed products are excluded from the matching results.
This approach is particularly useful for large catalogs where only a small percentage of products change each month.
Incremental processing reduces the number of computations and LLM API calls, lowering token consumption and processing costs while accelerating catalog updates.
We also developed a web-based interface for managing supplier catalog matching and reviewing the results.
The interface allows users to:
Catalog processing runs in the background, allowing users to work with large datasets without continuously monitoring execution.
We developed an AI-powered product matching system that automates the identification of corresponding products across the client's internal database and external supplier catalogs.
During testing, the solution achieved the following results:
By combining hybrid search, LLM-based attribute extraction, and ML classification, the system can process product catalogs with inconsistent naming conventions, languages, and levels of detail.
Incremental catalog updates optimize recurring processing by focusing on modified products rather than reprocessing entire catalogs, reducing LLM API calls and associated costs.
The web interface provides a centralized workflow for catalog uploads, automated product matching, and manual validation of uncertain results.
The solution provides a foundation for further automation of assortment analysis, supplier product comparisons, and the identification of new product categories.
Do you want to know the total cost of development and realization of the project? Tell us about your requirements, our specialists will contact you as soon as possible.