Why do people buy something different from what they’re looking for when they shop? This is a question worth thinking about. Liane Lewin-Eytan, senior manager for search for the search function on Alexa’s shopping team, explains in her blog:
For example, people issue search instructions for Alexa, and through the shopping search algorithm, Alexa ends up listing products for users to choose from, which can be interpreted as “relevant products” that meet the user’s needs. The final presentation of these “relevant products” depends on the judgment of the human noteman.
In short, the manual noteper marks the relevant products that meet the user’s expectations, while the others are unrelated. Interestingly, Amazon recently found that users often associate with products marked as irrelevant by commenters.
For example, an artificial notewill would associate the term “buy a burger” with a hamburger product, rather than the term “hamburger”, but Amazon found that users who issued the “buy burger” directive might buy a seemingly unrelated burger and not a real burger.
Amazon categorizes user behavior, first by buying unrelated products directly, as in the example of the burger machine senumerated earlier, and by “interacting” with unrelated products, such as adding unrelated products to their shopping carts or sharing them with other users.
Amazon’s research shows that users are more likely to buy unrelated products when faced with higher-selling or cheaper products, and that people are more likely to buy unrelated products in categories such as toys and digital products than in beauty and grocery categories.
To learn more about this behavior of users, Amazon researchers have made many efforts.
First, the team used statistical methods to identify search results for keywords of varying length. The study found that users searching through short/relatively lengthy keywords were more flexible in their purchasing decisions than medium-length search terms. Therefore, they believe that short keywords represent the uncertainty of the user and the willingness to explore, and that long keywords reduce the likelihood of an accurate match;
In addition, the researchers considered the indirect relationship between the relevant and unrelated products in the search results. “For example, if two products are of the same style, brand, or category, or they are often purchased in conjunction with each other, there is an indirect relationship between them.”
The researchers used two different indirect relationship measurements, one based on the meaning of descriptive terms and the other based on purchase history. Both of these factors affect the likelihood that users will buy unrelated products.
Much of this is statistical analysis, and Amazon has conducted two experiments to assess the value of seemingly irrelevant products in search results.
First, the researchers conducted 1,500 search queries, each recording a related product and an unrelated product, and considered the results of applying five different selection strategies to those products.
The first strategy, the “best strategy”, always selects products that lead to higher levels of purchase or participation. (The engagement level/purchase level is the ratio of the engagement/purchase behavior to all interactions in the data sample.) The other four are “relevant strategies”, i.e. select related products, “unrelated strategies”, i.e. select unrelated products, “random strategies”, i.e., random selection in both products, “worst strategy”, i.e. select products that reduce participation/purchase levels.
pRatio is the purchase level, eRatio is the level of participation
As can be seen from the table above, the remaining strategies have a significant gap in terms of purchase level and participation level compared to the goods selected by optimal strategy and related strategies (without error).
In another experiment, the researchers used the same 1500 queries to train three different machine learning models: one to maximize correlation, the second to maximize purchase levels, and the third to maximize learning engagement levels. On this basis, Amazon has built two fusion models, one combining the association model and the participation model, the other combining the association model and the purchase model, and the ability to tune each fusion model to assign different weights to the output of the two models that make up the model.
For example, in an association purchase fusion model, when the values of the association and purchase levels are set to 1 and 0, respectively, the fusion model will produce only the associated model output; For these two fusion models, Amazon set a series of weights and plotted the results.
As the figure above shows, there is a trade-off between correlation and purchase level/participation level: increasing or decreasing correlation will affect the performance of the level of purchase/participation.
Amazon says customers may understand and forgive the deficiency if the search results do not meet the customer’s needs but appear to be relevant. “At the same time, the level of purchase/participation represents a more subjective type of correlation, which cannot be assessed by a human commenter, which may also lead to the inability to recommend a satisfactory product.”
Currently, amazon’s model of assessing the trade-off between relevance and buy/participation levels is rather rough, and a more complex and powerful machine learning model can yield better results, especially if it is explicitly trained to evaluate certain factors such as keyword length, price, and indirect relationships.