Recommender engines are affected mainly by the following two issues:
- The sparsity problem: Recommender engines work upon user preferences (or ratings for different items, depending upon the application) to predict or recommend products. Usually the ratings are given on some chosen scale but the user may choose not to rate certain items which he/she hasn't bought or looked at. For such cases, the rating is blank or zero. Hence, the ratings matrix R has elements of the form:
For any real world application, such as an e-commerce platform, the size of such a ratings matrix is huge due to the large number of users and items available on the platform. Even though a lot of user related information is gathered on such a platform, the ratings matrix itself might still be pretty sparse, that is the matrix might have a many elements as blanks (or zeroes). This problem in general is termed the sparsity problem. The sparsity problem renders the recommender engine's predictions ineffective as the algorithms are not able to infer the correlations correctly due to blanks or missing ratings. In the worst cases, the algorithm may term two users as un-correlated when actually they have highly similar preferences. The sparsity problem usually affects collaborative filtering algorithms.
- The cold start problem: A special case of the sparsity problem is the cold start issue. As mentioned previously, when the ratings matrix contains sparsely populated elements (or ratings), the recommender engine fails to return valid recommendations. The cold start problem occurs in two particular cases. Firstly, assume a user has newly been added to the system. In this case, the row representing the user would contain zeroes (mostly). Recommending items to such a user is virtually impossible due to unavailability of information related to his/her preferences. The second scenario is when an item is newly added to the system. Since the newly added item will not have any ratings by the users, recommending such an item would be difficult for the recommender system. Hence, these two scenarios represent what is termed the cold start problem. Very much like the sparsity problem, the cold start problem also plagues collaborative filters.