8.2.1 Selecting the segmentation population: the mobile telephony core segments

Mobile telephony customers are typically categorized in core segments according to their rate plans and the type of the relationship with the operator. The first segmentation level differentiates residential (consumer) from business customers. Residential customers are further divided into postpaid and prepaid:

• Postpaid—contractual customers: Customers with mobile phone contracts. Usually, they comprise the majority of the customer base. These customers have a contract and a long-term billing arrangement with the network operator for the received services. They are billed on a monthly basis and according to the traffic of the past month; hence, they have unlimited credit.
• Prepaid customers: They do not have a contract-based relationship with the operator, and they are buying credit in advance. They do not have ongoing billing and they pay for the services before actually using them.

Additionally, business customers are further differentiated according to their size into corporate, SME, and SOHO customers.

The typical core segments in mobile telephony are depicted in Figure 8.1.

The objective of the marketers was to enrich the core segmentation scheme with refined subsegments. Therefore, they decided to focus their initial segmentation attempts exclusively on residential postpaid customers. Prepaid customers need a special approach in which attributes like the intensity and frequency of top-ups (recharging of their credits) should also be taken into account. Business customers also need a different handling since they comprise a completely different market. It is much safer to analyze those customers separately, with segmentation approaches such as value based, size based, industry based, etc.

Moreover, only MSIDNs (telephone numbers) with current status active or in suspension (due to payment delays) were included in the analysis. Churned (voluntary and involuntary churners) MSISDNs have been excluded from the start. Their “contribution” is crucial in the building of a churn model but trivial in a segmentation scheme mainly involving phone usage. In addition, segmentation population was narrowed down even further by excluding users with no incoming or outgoing usage within the past 6 months. Those users have been flagged as inactive, and they have been selected for further examination and profiling. They could also form a target list for an upcoming reactivation campaign, but they do not have much to contribute to a behavioral analysis since, unfortunately, inactivity is their only behavior at the moment.

8.2.2 Deciding the segmentation level

Customers may own more than one MSISDN which may use in a different manner to cover different needs. In order to capture all the potentially different usage behaviors of each customer, it has been decided to implement the behavioral segmentation at MSISDN level. Therefore, relevant input data have been preaggregated accordingly, and the derived cluster model assigned each MSISDN to a distinct behavioral segment.

8.2.3 Selecting the segmentation dimensions

Once again, the mining datamart tables, comprised the main sources of the input data. Table 8.1 outlines the main usage aspects that were selected as segmentation criteria.

The modeling phase was followed by extensive profiling of the revealed segments. In this phase, all available information, including demographic data and contract information details, were cross-examined with the identified customer groups.

8.2.4 Time frames and historical information analyzed

A 6-month observation period was analyzed, a time span that in general ensures the capturing of stable, nonvolatile behavioral patterns instead of random or outdated ones. Summary fields (sums, counts, percentages, averages, etc.) covering the 6-month observation period were used as inputs in the clustering model.

8.5.1 Preparing data for clustering: combining fields into data components

The number of the original segmentation inputs exceeded 30. Using all those fields as direct input in a clustering algorithm would have produced a complicated solution. Therefore, the approach followed was to incorporate a data reduction technique in order to reveal the underlying data dimensions prior to clustering.

This approach was adopted to eliminate the risk of deriving a biased solution due to correlated attributes. Moreover, it also ensures a balanced solution, in which all data dimensions contribute equally, and it simplifies the tedious procedure of segmentation understanding by providing conceptual clarity.

Specifically, a Principal Components Analysis (PCA) model with varimax rotation was applied to the original segmentation fields. A Type node was used to set as Inputs (Direction In) the designated attributes of Table 8.2. The rest of the attributes were assigned with a None role and didn’t contribute to the subsequent model. The modeling data were then fed into a PCA/Factor modeling node, and a PCA model was developed which grouped original inputs into components. The parameter settings for the PCA model are listed in Table 8.3.

Initially, based on the eigenvalues over 1 criterion, the algorithm suggested the extraction of nine components. However, the ninth component was mainly related with a single original attribute, the GPRS traffic. Moreover, it accounted for a small percentage of the information of the original inputs, about 3%. Having considered that, the analysts decided to go for a simpler solution of eight components, selecting to sacrifice a small part of the original information for simplicity. In the eight-component solution, the GPRS traffic was combined with Events usage to form a single component.

Before using the derived components and substituting more than 30 fields with a handful of new ones, the data miners of the organization wanted to be sure that:

• The PCA solution carried over most of the original information.
• The derived components, which would substitute the original attributes, were interpretable and had a business meaning.

Therefore, they started the examination of the model results by looking at the table of “Explained Variance.” Table 8.4 presents these results.

The resulted eight components retained more than 73% of the variance of the original fields. This percentage was considered satisfactory and thus the only task left before accepting the components was their interpretation. In practice, only a solution comprised of meaningful components should be retained.

So what do those new composite fields represent? What business meaning do they convey? As these new fields are constructed in order to substitute the original fields in the next stages of the segmentation procedure, it is necessary to be thoroughly decoded before being used in upcoming models.

The component interpretation phase included the examination of the “rotated component matrix” (Table 8.5), a table that summarizes the correlations (loadings) between the components and the original fields.

The “interpretation” results are summarized in Table 8.6.

The explained and labeled components and the respective component scores were subsequently used as inputs in the clustering model. This brings us to the next phase of the application: the identification of useful groupings through clustering.

8.5.2 Identifying the segments with a cluster model

The generated components represented effectively all the usage dimensions of interest, in a concise and comprehensive way, leaving no room for misunderstandings about their business meaning. The next step of the segmentation project included the usage of the derived component scores as inputs in a cluster model. Through a new Type node, the generated components were set as Inputs for the training of the subsequent cluster model.

The clustering process involved the application of an Auto Cluster node for the training and evaluation of 2 TwoStep models with different parameter settings in terms of outlier handling. Figure 8.4 presents an initial comparison of the two cluster models provided by the generated Auto Cluster model.

The cluster viewer presents the number of identified clusters as well as the Silhouette coefficient and the size of smallest and largest clusters for each model. In our case, the first TwoStep model was selected for deployment due to its higher Silhouette value and, more importantly, due to its transparency and interpretability.

The parameter settings of the selected cluster model are presented in Table 8.7.

As shown in Figure 8.5, the model yielded a five-cluster solution with a fair Silhouette measure of 0.29.

The distribution of the revealed clusters is depicted in Figure 8.6.

As a reminder, we outline that these clusters were not known in advance, neither imposed by users, but uncovered after analyzing the actual behavioral patterns of the usage data. The largest cluster, Cluster 1, which as we’ll see corresponds to “typical” usage, contained about 30% of customers. The smallest one included about 11.5% of total users.

8.5.3 Profiling and understanding the clusters

Each revealed cluster corresponds to a distinct behavioral typology. This typology had to be understood, named, and communicated to all the people of the organization in a simple and concise way before being used for tailored interactions and targeted marketing activities. Therefore, the next phase of the project included the profiling of the clusters through simple reporting techniques. The “recognize and label” procedure started with the examination of the clusters in respect to the component scores, providing a valuable first insight on their structure, before moving to profiling in terms of the original usage fields. The table of centroids is shown in Figure 8.7.

Since the component scores are standardized, their overall means are 0 and the mean for each cluster denotes the signed deviation from the overall mean. By studying these deviations, we can see the relatively large mean values of Factors 1 and 8 among cases of Cluster 5. Since Factors 1 and 8 represent voice usage and average call duration, it seems that Cluster 5 was comprised of high voice users. This conclusion is further supported by studying the distribution of the factors for Cluster 5 with the series of boxplots presented in Figure 8.8.

The background boxplot summarizes the entire population, while the overlaid boxplot refers to the selected cluster. In respect to Factors 1 and 8, the boxplots for Cluster 5 are at the right side of those for the entire population, indicating high values and consequently high voice usage.

After studying the distributions of the factors, the profile of each cluster had started to take shape. In the final profiling stage, analysts returned to the primary inputs, seeking for more straightforward differentiations in terms of the original attributes. Table 8.8 summarizes the five clusters in terms of some important usage attributes. It presents the mean of each attribute over the members of each cluster. The last column denotes the overall mean.

Large deviations from the marginal mean characterize the respective cluster and denote a behavior that differentiates the cluster from the typical behavior.

Although inferential statistics have not been applied to flag statistically significant differences from the overall population mean, we can see some large observed differences which seem to characterize the clusters. Based on the information summarized so far, the project team started to outline a first rough profile of each cluster:

• Cluster 1 contains average voice users.
• Cluster 2 seems to include users with basic usage. They also seem to be characterized by increased average call duration and very small communities.
• Cluster 3 includes roamers and users with increased communication with international destinations. They are also accustomed to Internet and event calls.
• Cluster 4 is mainly consisted of SMS users that seem to have an additional inclination to tech services like MMS, Internet, and event calls.
• Cluster 5 contains heavy voice users. They also have the largest outgoing communities. Perhaps, this segment included residential customers who use their phones for business purposes (self-professionals).

Although not presented here, the profiling of the clusters went on with demographical exploration and by assessing their revenue contribution. Additionally, the cohesion of the clusters was also examined mainly with boxplots and with dispersion measures, such as the standard deviations of the clustering fields for each cluster. Finally, a series of summarizing charts were built to graphically illustrate in an intuitive manner the cluster “structures.” The bars in the graphs represent the means of selected, important attributes, after standardization (in z-scores) over the members of the cluster. The vertical line at 0 denotes the overall mean. Hence, a bar facing to the right indicates a cluster mean higher than the overall mean, while a bar facing to the left indicates a lower mean. The bar charts are followed by a more detailed profiling of each cluster which wrap ups all their defining characteristics. The clusters were labeled according to these profiles (Figures 8.9, 8.10, 8.11, 8.12, and 8.13).

8.5.4 Segmentation deployment

The final stage of the segmentation process involved the integration of the derived scheme in the company’s daily business procedures. From that moment, all customers were characterized by the segment to which they were assigned. The deployment procedure involved a regular (on a monthly basis) segment update. More importantly, the marketers of the operator decided to further study the revealed segments and enrich their profiles with attitudinal data collected through market research surveys. Finally, the organization used all the gained insight to design and deploy customized marketing strategies which improved the overall customer experience.

8.6.1 Clustering with the K-means algorithm

A two-step approach was followed for clustering with RapidMiner. Initially, a PCA algorithm was applied for reducing the data dimensionality and for replacing the original inputs with fewer combined measures. Subsequently, the generated component scores were used as the clustering fields in a K-means algorithm which identified the distinct customer groupings. The RapidMiner process is presented in Figure 8.14.

More specifically, a Retrieve operator was used to retrieve the modeling data from the RapidMiner repository. Then, through a Select Attributes operator, only the attributes designated as Inputs in Table 8.2 were retained for clustering. Since the PCA model in RapidMiner requires normalized inputs, the selected attributes were normalized with a Normalize operator. A z-normalization method was applied so eventually all attributes ended with a mean value of 0 and a standard deviation of 1. The z-scores were then fed into a PCA model operator with the parameter settings shown in Figure 8.15.

After trial and experimentation, a variance threshold of 85% was selected as the criterion for the extraction of components. This setting led to the extraction of 12 components which cumulatively accounted for about 85% of the total variance/information of the 31 original attributes, as shown in the rightmost column of the table in Figure 8.16. Initially, eight components were extracted which explained about 75% of the information of the original inputs. However, it turned out that the extraction of a larger number of components improved the K-means clustering, leading to a richer and more useful clustering solution. Therefore, the final choice was to proceed with the 12 components.

The 12 components were then used as clustering inputs in a K-means model. The parameter settings are shown in Figure 8.17.

The Euclidean distance was selected for measuring the similarities of the records. The add-cluster attribute option generated the cluster membership field and assigned each instance to a cluster. After many trials and evaluations of many different clustering solutions, a 5-cluster solution was finally adopted for deployment. Hence, a “k” of 5 was specified, guiding the algorithm to form five clusters. The cluster distribution is presented in Figure 8.18. Clusters 3 and 4 are dominant since they comprise almost 80% of the total customer base.

The generated K-means model was connected with a Cluster Distance Performance operator to evaluate the average distance between the instances and the centroid of their cluster. The respective results are shown in Figure 8.19.

The segmentation procedure was concluded with the profiling of the revealed clusters and with the identification of their defining characteristics. The first step was to study the centroids table, available in the results tab of the cluster model. To facilitate additional profiling in terms of the original attributes, the cluster membership field was cross-examined with the original inputs. Figure 8.20 presents a profiling chart which depicts the averages of some important (normalized) attributes over the clusters. The vertical line at 0 corresponds to the marginal means of the normalized attributes. Hence, bars and dots to the right of this reference line designate cluster means above the marginal mean, while bars and dots to the left designate lower cluster means.

A brief profile of the clusters is presented below. The clusters were named according to their identified characteristics:

• Cluster_0 SMS users: Highest SMS usage. Highest SMS to voice ratio. Increased usage of MMS, Events, and Internet
• Cluster_1 International users: Highest roaming and international usage. Also increased usage of voice and MMS
• Cluster_2 Professional users: High voice usage. Increased community, frequency (days), and duration of voice calls
• Cluster_3 Basic users: Lowest usage of all services and lowest community
• Cluster_4 Typical voice users: Average usage, almost exclusively voice

The revealed clusters seem indeed similar to the ones identified by the TwoStep algorithm. Tempted to compare the two solutions? After joining and cross-tabulating the cluster membership fields, it appears that the two models present an “agreement” of about 70%. Specifically, after excluding the TwoStep “outlier” cluster, the two models seem to assign approximately 70% of the instances in analogous groups. Where do they not agree? Their main difference is in the “international” segment which in the K-means solution is smaller, including only those roamers with increased general usage. The similarity of the clusters nevertheless is a good sign for the validity of the solutions.