The challenge

The B2B and B2C e-commerce market is growing rapidly. The increase in products sold directly to consumers via globally operating platforms represents both an opportunity in terms of sourcing and a growing challenge in terms of market analysis for European manufacturers and retailers. In this environment, it is becoming increasingly complex to decide on the right assortment and pricing of products and to identify innovations, trends, and counterfeits. This is because it has become particularly challenging to compare the product portfolios and individual products of manufacturers and retailers in the EU with those of international companies in order to identify competitors early on or to find alternative sources of supply with better availability.

In the past this comparison was done using unique identifiers like the European Article Number (EAN) or Manufacturer Part Number (MPN), for example. However, since many platforms have since disabled the EAN or MPN search functions, a new product-matching method had to be found. For this purpose, 

image or textual data from product descriptions could be used. While Price Intelligence had used both approaches in isolation, their combination in a single search approach would offer the possibility to create a more reliable and powerful model for product identification and comparison. Unfortunately, this method would be highly compute-intensive since no pre-trained networks capable of integrating image and textual data at the same time were previously available to Price Intelligence.

The solution

The most important step in creating a product matching algorithm for market analysis is pre-processing, where product data (like image and text data) are transformed into so-called embedding vectors, trying to encode product attributes into numeric values suitable for machine learning. Product similarity is then determined by comparing these vectors such that similar vectors come from similar products.

The key tool for computing these embedding vectors is neural networks; and pre-trained networks based on generic data are an important and valuable building block, which can be customized later on.

Prior to the experiment, Price Intelligence had created such pre-trained networks for both image and textual data in isolation (employing machine learning approaches that differed according to data source). However, in order to generate larger embedding vectors combining both data sources to obtain better product search results, HPC resources were needed and, thanks to FF4EuroHPC, could be employed to implement a synthesized algorithmic approach to build networks pre-trained with a combination of both images and text data.

For each specific customer, these pre-built networks will subsequently be customised using the customer's product data and also related product data (e.g. found on online marketplaces) to generate better embedding vectors, fitted to the specific use cases, and thereby producing better matching results.

Business impact

The newly created matching service decisively expands the existing software of Price Intelligence and will be available to over 100 European manufacturers and retailers to help them optimize their pricing and, as an example of a new feature, even optimize their portfolio. Because of new software products using the matching service, Price Intelligence expects an increase in sales of up to 30%.

In addition, the matching service is already being used productively by two Parsionate retail customers within a web store and call centre to optimize their suggestion system for alternative products. Parsionate has already identified many other potential customers from various industries and is in discussions with them about implementing the matching service.

In the future, the knowledge gathered in the experiment can be used to create other new services such as assortment analysis, detection of product trends, or detection of counterfeit products using similar techniques.

Due to the wide range of projects in which the two partners Price Intelligence and Parsionate are involved, many large companies as well as SMEs in Europe will benefit from the capabilities developed in the experiment.

Benefits

• End customers from retail and manufacturing use over 70% less time when searching and comparing items on the internet.
• With the new product matching service in the Price Intelligence software, an increase in sales of up to 30% is expected.
• The newly developed matching service for finding similar products in an assortment has already led to new sales at Parsionate. They have implemented a recommendation engine for online shopping customers.

Organisations involved: 

AI Expert: Parsionate GmbH 
Domain Expert: Price Intelligence  
HPC Providers: German Top Secure – Data Processing GmbH, High Performance Computing Center - University of Stuttgart
Scientific collaborator: SICOS BW GmbH

Partners High Performance Computing Center and Sicos are part of the NCC Germany.