As you conduct clustering analysis in real life, one of the things that can quickly become apparent is the fact that neither hierarchical nor k-means are specifically designed to handle mixed datasets. By mixed data, I mean both quantitative and qualitative or, more specifically, nominal, ordinal, and interval/ratio data. The reality of most datasets that you will use is that they will probably contain mixed data. There are a number of ways to handle this, such as doing Principal Components Analysis (PCA) first in order to create latent variables, then using them as input in clustering or using different dissimilarity calculations. We will discuss PCA in the next chapter.
With the power and simplicity of R, I prefer to use the Gower dissimilarity coefficient to turn mixed data to the proper feature space. In R, you can even include factors as input variables to cluster. Additionally, instead of k-means, I recommend using the PAM clustering algorithm. PAM is very similar to k-means but offers a couple of advantages. First, PAM accepts a dissimilarity matrix, which allows the inclusion of mixed data. Second, it is more robust to outliers and skewed data because it minimizes a sum of dissimilarities instead of a sum of squared Euclidean distances (Reynolds, 1992). This is not to say that you must use Gower and PAM together. If you choose, you can use the Gower coefficients with hierarchical and I've seen arguments for and against using it in the context of k-means. Additionally, PAM can accept other linkages. However, they make an effective method together to handle the mixed data. Let's take a quick look at both of these concepts before moving on.
The Gower coefficient compares cases pairwise and calculates a dissimilarity between them, which is essentially the weighted mean of the contributions of each variable. It is defined for two cases called i and j as follows:
Here, Sijk is the contribution provided by the kth variable and Wijk is 1 if the kth variable is valid, or else 0.
For ordinal and continuous variables, Sijk = 1 – (absolute value of xij – xik) / rk', where rk is the range of values for the kth variable.
For nominal variables, Sijk = 1 if xij = xjk', or else 0.
For binary variables, Sijk is calculated based on whether an attribute is present (+) or not present (-), as shown in the following table:
|
Variables |
Value of attribute | |||
|---|---|---|---|---|
|
Case |
+ |
+ |
- |
- |
|
Case |
+ |
- |
+ |
- |
|
|
1 |
0 |
0 |
0 |
|
|
1 |
1 |
1 |
0 |
For Partitioning Around Medoids, let's first define a medoid. A medoid is an observation of a cluster that minimizes the dissimilarity (in our case, calculated using the Gower metric) between the other observations in that cluster. So, similar to k-means, if you specify five clusters, you will have five partitions of the data.
With the objective of minimizing the dissimilarity of all the observations to the nearest medoid, the PAM algorithm iterates over the following steps:
- Randomly select k observations as the initial medoid.
- Assign each observation to the closest medoid.
- Swap each medoid and non-medoid observation, computing the dissimilarity cost.
- Select the configuration that minimizes the total dissimilarity.
- Repeat steps 2 through 4 until there is no change in the medoids.
Both Gower and PAM can be called using the cluster package in R. For Gower, we will use the daisy() function in order to calculate the dissimilarity matrix and the pam() function for the actual partitioning. With this, let's get started with putting these methods to the test.
Until a couple of weeks ago, I was unaware that there were less than 300 certified Master Sommeliers in the entire world. The exam, administered by the Court of Master Sommeliers, is notorious for its demands and high failure rate. The trials, tribulations, and rewards of several individuals pursuing the certification are detailed in the critically-acclaimed documentary, Somm. So, for this exercise, we will try and help a hypothetical individual struggling to become a Master Sommelier find a latent structure in Italian wines.