Unsupervised Learning

Part 1: Clustering — Finding Hidden Groups 🧑‍🤝‍🧑

Clustering is the task of automatically grouping similar data points together. The objective is to create clusters where the items within a single cluster are very similar, and the items in different clusters are very different. It's a powerful tool for discovering natural groupings you might not have known existed.

Common applications include:

• Customer Segmentation: Grouping customers with similar purchasing behaviors for targeted marketing campaigns.
• Document Analysis: Grouping news articles by topic without any prior knowledge of the topics.
• Image Segmentation: Grouping pixels of similar color to identify objects in an image.

Algorithm: K-Means Clustering

Intuition: K-Means is the most popular and intuitive clustering algorithm. It works by trying to find a user-defined number of cluster centers (the 'K' in K-Means) and assigning each data point to the nearest center. Here's the process:

1. Choose K: First, you decide how many clusters you want to find (e.g., K=3).
2. Initialize Centroids: The algorithm randomly places K "centroids" (the center of a cluster) onto your data plot.
3. Assign Points: Each data point is assigned to its closest centroid. This forms K initial clusters.
4. Update Centroids: The center of each cluster is recalculated by finding the mean of all the points assigned to it. The centroid moves to this new center.
5. Repeat: Steps 3 and 4 are repeated until the centroids no longer move significantly. At this point, the clusters are stable.

Challenge: How to Choose K? The Elbow Method

The biggest challenge with K-Means is choosing the right value for K. One popular technique is the **Elbow Method**. We run the K-Means algorithm for a range of K values (e.g., 1 to 10) and for each run, we calculate the model's "inertia" (the sum of squared distances of samples to their closest cluster center). When we plot inertia against K, the plot often looks like an arm. The "elbow" of the arm—the point where the rate of decrease sharply changes—is a good estimate for the best value of K.

Python Implementation: Let's find clusters in a synthetic dataset.

# --- 1. Import libraries ---
import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans

# --- 2. Generate synthetic data ---
# Create a dataset with 4 distinct clusters for our algorithm to find
X, y = make_blobs(n_samples=300, centers=4, cluster_std=0.8, random_state=42)

# --- 3. Use the Elbow Method to find the optimal K ---
inertia = []
for k in range(1, 11):
    kmeans = KMeans(n_clusters=k, n_init=10, random_state=42)
    kmeans.fit(X)
    inertia.append(kmeans.inertia_)

plt.figure(figsize=(8, 5))
plt.plot(range(1, 11), inertia, marker='o')
plt.title('Elbow Method')
plt.xlabel('Number of clusters (K)')
plt.ylabel('Inertia')
plt.show() # The elbow is clearly at K=4!

# --- 4. Apply K-Means with the optimal K ---
kmeans = KMeans(n_clusters=4, n_init=10, random_state=42)
kmeans.fit(X)
cluster_labels = kmeans.labels_
centroids = kmeans.cluster_centers_

# --- 5. Visualize the results ---
plt.figure(figsize=(10, 7))
plt.scatter(X[:, 0], X[:, 1], c=cluster_labels, cmap='viridis', s=50, alpha=0.7)
plt.scatter(centroids[:, 0], centroids[:, 1], c='red', s=200, marker='X', label='Centroids')
plt.title('K-Means Clustering Results')
plt.xlabel('Feature 1')
plt.ylabel('Feature 2')
plt.legend()
plt.show()

Part 2: Dimensionality Reduction — Taming Complexity 🌪️

Many modern datasets are "high-dimensional," meaning they have a large number of features (columns). This can lead to the **"curse of dimensionality,"** where data becomes very sparse, models become computationally expensive to train, and performance can degrade due to noise. **Dimensionality Reduction** is the process of reducing the number of features while retaining as much of the original information as possible.

Think of it like creating a summary of a long book. You lose some of the fine details, but you preserve the main plot points. The two main benefits are:

1. Data Visualization: It's impossible to plot data with 50 dimensions. By reducing it to 2 or 3, we can create scatter plots to visually inspect its structure.
2. Performance Improvement: It can sometimes lead to faster and more accurate models by removing noise and redundant features.

Technique: Principal Component Analysis (PCA)

Intuition: PCA is the most popular dimensionality reduction technique. It works by transforming your original features into a new set of artificial features called **Principal Components**. These new components have two special properties:

• They are ordered by the amount of **variance** they capture in the data. The first principal component (PC1) is the direction that explains the most "spread" in the data. PC2 explains the next most, and so on.
• They are **uncorrelated** with each other.

By keeping only the first few principal components, we can reduce the number of features while retaining most of the data's original variance (i.e., its information).

Python Implementation: Let's use the Iris dataset, which has 4 features. We'll use PCA to reduce it to 2 features so we can visualize it.

# --- 1. Import libraries ---
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
import pandas as pd

# --- 2. Load and scale the data ---
iris = load_iris()
X, y = iris.data, iris.target

# Standardize the features
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# --- 3. Apply PCA ---
# We want to reduce the 4 dimensions to 2
pca = PCA(n_components=2)
principal_components = pca.fit_transform(X_scaled)

# Create a DataFrame with the new principal components
pca_df = pd.DataFrame(data=principal_components, columns=['PC1', 'PC2'])

# --- 4. Check the explained variance ---
print("Explained variance by component:", pca.explained_variance_ratio_)
print("Total variance explained:", sum(pca.explained_variance_ratio_))
# This tells us how much information was retained. Often ~95% is great!

# --- 5. Visualize the 2D representation ---
plt.figure(figsize=(10, 7))
# Color the points by their original flower species (y)
plt.scatter(pca_df['PC1'], pca_df['PC2'], c=y, cmap='viridis')
plt.title('PCA of Iris Dataset')
plt.xlabel('First Principal Component (PC1)')
plt.ylabel('Second Principal Component (PC2)')
plt.show()
# Notice how well the three species are separated in just 2 dimensions!