“Well, you can’t recommend that! Why sometimes the best recommendations aren’t right.”

My number one question for the Amazon recommendations team is: “exactly how many vacuum cleaners do you think I need?” Just because I bought the fancy Dyson to clean up after my dog does not mean that I’m soon to buy a second one, and yet my Amazon homepage seems hell-bent on recommending it. The reality is, there will always be business logic–or basic human logic–that you want to include in the flow of your recommendation system to prevent silliness. Whether it’s contextually inappropriate recommendations, business infeasible recommendations, or simply the necessity to keep the set of recommendations a bit less monomaniacal, the last-step ordering can crucially improve recommendations.

But hold your horses! Don’t think the ordering step is all switch-cases and manually overriding your recommendation system. There needs to be a synergy between your ranking and your serving. One of the authors, Bryan, has a story about a certain query-based recommender he built for clothes: he wanted to implement a super-simple diversity filter on his recommendations – checking that the clothes recommended were of different merchandise classes. He made the output of his scoring model stack-rank the recs by merchandise class, so he could pick a few from each to serve. Low-and-behold the first week in prod he was recommended 3, 4, even 5 backpacks out of 10 recommendations. Studious as users may be, this seemed erroneous, and required a bit of QA. His error? Backpacks can be a member of up to 3 different merch classes, so they were sneaking into several diversity classes!

Tricky problems where the theory meet production recommendations are the subject of this section. We’ll talk about diverse recommendations as in this example, but we’ll also discuss other important business priorities that factor into the serving part of a recommendation pipeline.