Softmax Activation Function Implementation

Problem

Implement the softmax activation function. Given a list of values, compute the softmax probabilities so they sum to 1.

Solution

import math

def softmax(scores: list[float]) -> list[float]:
    # Subtract max for numerical stability
    max_score = max(scores)
    exp_scores = [math.exp(s - max_score) for s in scores]
    total = sum(exp_scores)
    return [round(e / total, 4) for e in exp_scores]

Explanation

1. Numerical stability: Subtract the maximum value from all scores before exponentiating. This prevents overflow without changing the result (since e^(a-c) / sum(e^(b-c)) = e^a / sum(e^b)).
2. Exponentiate each adjusted score.
3. Divide each exponentiated score by the sum to normalize to a probability distribution.

Complexity

• Time: O(n) where n is the number of scores
• Space: O(n) for the exponentiated scores