Implement Soft Voting Classifier

#306 · Machine Learning · Medium

Problem

Implement a soft voting classifier. Instead of counting class votes, average the predicted class probabilities from each classifier and pick the class with the highest average probability.

Solution

import numpy as np

def soft_voting(probabilities: list[np.ndarray],
                weights: list[float] = None) -> np.ndarray:
    """
    probabilities: list of arrays of shape (n_samples, n_classes),
                   one per classifier.
    weights: optional list of weights for each classifier.
    Returns: array of final class predictions.
    """
    n_classifiers = len(probabilities)

    if weights is None:
        weights = [1.0 / n_classifiers] * n_classifiers
    else:
        total = sum(weights)
        weights = [w / total for w in weights]

    weighted_sum = np.zeros_like(probabilities[0], dtype=float)
    for i, probs in enumerate(probabilities):
        weighted_sum += weights[i] * np.array(probs)

    return np.argmax(weighted_sum, axis=1)

Explanation

Each classifier provides a probability distribution over classes for each sample.
Compute the weighted average of these probability distributions. If no weights are given, use uniform weights.
The final prediction is the class with the highest weighted average probability.
Soft voting typically outperforms hard voting because it considers the confidence of each classifier, not just its top prediction.

Complexity

Time: O(n c k) where n is samples, c is classes, k is classifiers
Space: O(n * c) for the weighted sum array

← #305 #307 →