We will now put everything together and demonstrate our system for the following new post that we assign to the new_post variable:

new_post = '''
Disk drive problems. Hi, I have a problem with my hard disk.
After 1 year it is working only sporadically now.
I tried to format it, but now it doesn't boot any more.
Any ideas? Thanks. '''

As you learned earlier, you will first have to vectorize this post before you predict
its label:

>>> new_post_vec = vectorizer.transform([new_post])
>>> new_post_label = km.predict(new_post_vec)[0]

Now that we have the clustering, we do not need to compare new_post_vec to all post vectors. Instead, we can focus only on the posts of the same cluster. Let's fetch their indices in the original data set:

>>> similar_indices = (km.labels_ == new_post_label).nonzero()[0]

The comparison in the bracket results in a Boolean array, and nonzero converts that array into a smaller array containing the indices of the True elements.

Using similar_indices, we then simply have to build a list of posts together with their similarity scores:

>>> similar = []
>>> for i in similar_indices:
...    dist = scipy.linalg.norm((new_post_vec - vectorized[i]).toarray())
...    similar.append((dist, train_data.data[i]))
>>> similar = sorted(similar)
>>> print("Count similar: %i" % len(similar))
Count similar: 56

We found 56 posts in the cluster of our post. To give the user a quick idea of what kind of similar posts are available, we can now present the most similar post (show_at_1), and two less similar but still related ones, all from the same cluster:

>>> show_at_1 = similar[0]
>>> show_at_2 = similar[len(similar) // 10]
>>> show_at_3 = similar[len(similar) // 2]

The following table shows the posts together with their similarity values:

It is interesting that the posts reflect the similarity measurement score. The first post contains all the salient words from our new post. The second also revolves around booting problems but is about floppy disks and not hard disks. Finally, the third is neither about hard disks nor about booting problems. Still, all the posts, we would say, belong to the same domain as the new post.