Implement the Softplus Activation Function

Problem

Implement the Softplus activation function. Softplus is a smooth approximation of ReLU defined as f(x) = ln(1 + exp(x)).

Solution

import numpy as np

def softplus(x: np.ndarray) -> np.ndarray:
    # Numerically stable: for large x, softplus(x) ≈ x
    return np.where(x > 20, x, np.log1p(np.exp(x)))

Explanation

1. Softplus computes ln(1 + exp(x)), which smoothly approximates max(0, x) (ReLU).
2. For large positive x, exp(x) overflows, but ln(1 + exp(x)) ≈ x, so we return x directly for numerical stability.
3. np.log1p computes ln(1 + z) with better numerical precision for small z.
4. Unlike ReLU, Softplus is differentiable everywhere, and its derivative is the sigmoid function.

Complexity

• Time: O(n) where n is the number of elements
• Space: O(n) for the output array