Model-based recommendation with Spark

To make a preference prediction for any user, collaborative filtering uses a preference by other users of similar interests and predicts movies of your interests, that are unknown to you. Spark MLlib uses Alternate Least Squares (ALS) to make a recommendation. Here is a glimpse of a collaborative filtering method used in the ALS algorithm:

Table 1 – User-movie matrix

Users M1 M2 M3 M4
U1 2 4 3 1
U2 0 0 4 4
U3 3 2 2 3
U4 2 ? 3 ?

 

In the preceding table, user ratings on movies are represented as a matrix (that is, a user-item matrix), where a cell represents ratings for a particular movie by a user. The cell with ? represents the movies user U4 is not aware of or hasn't seen. Based on the current preference of U4, the cell with ? can be filled in with an approximate rating of users who have similar interests as U4. So at this point, ALS cannot do it alone, but the LFs are then used to predict the missing entries.

The Spark API provides the implementation of the ALS algorithm, which is used to learn these LFs based on the following six parameters:

  • numBlocks: This is the number of blocks used to parallelize computation (set to -1 to auto-configure).
  • rank: This is the number of LFs in the model.
  • iterations: This is the number of iterations of ALS to run. ALS typically converges to a reasonable solution in 20 iterations or less.
  • lambda: This specifies the regularization parameter in ALS.
  • implicitPrefs: This specifies whether to use the explicit feedback from the ALS variant (or one user defined) for implicit feedback data.
  • alpha: This is a parameter applicable to the implicit feedback variant of ALS that governs the baseline confidence in preference observations.

Note that to construct an ALS instance with default parameters, you can set the value based on your requirements. The default values are as follows: numBlocks: -1, rank: 10, iterations: 10, lambda: 0.01, implicitPrefs: false, and alpha: 1.0.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset