Cluster and Factor Analysis: Grouping Data and Latent Structure
What is Cluster and Factor Analysis?
Cluster analysis groups samples by feature similarity — similar within groups, different between groups — and is an unsupervised learning method. K-means takes a preset number k and iteratively assigns samples to the nearest centroid before updating the centroids; hierarchical clustering merges samples from singletons upward without a preset k and outputs a dendrogram where the cut is chosen by eye. Exploratory factor analysis (EFA) assumes a few latent factors behind the observed variables, extracts them by principal-component or maximum-likelihood methods, and rotates them (for example Varimax) so each variable loads cleanly on one factor.
When to Use It
Use clustering for customer segmentation, defect-pattern classification, supplier grading and equipment-state grouping; use EFA for questionnaire scale structure validation and grouping multi-dimensional quality indicators. Standardize the variables before clustering when units differ, choose the distance metric (Euclidean, Manhattan), and determine k in K-means from the silhouette coefficient or the elbow plot of within-group sums of squares.
How to Use It (Step by Step)
Paste the variable data and choose clustering (K-means with k, or hierarchical with a distance metric) or factor analysis (extraction method, number of factors, rotation). Read the cluster table, centroids and dendrogram, or the KMO and Bartlett tests, rotated loadings, communalities and factor scores; then name each factor from the variables with loadings above 0.4–0.5 and use the factor scores in further analysis.
Key Formulas / Example
K-means minimizes the within-cluster sum of squares Σᵢ‖xᵢ − μ_{c(i)}‖²; the silhouette coefficient (b − a)/max(a, b) measures cluster quality. For EFA, sample size should be at least 5–10 times the number of variables and at least 100; communality is the proportion of a variable's variance explained by the factors, and Varimax maximizes the variance of the squared loadings without changing model fit.