Implement the Hard Sigmoid Activation Function

Problem

Implement the Hard Sigmoid activation function. Hard Sigmoid is a piecewise linear approximation of the sigmoid function that is computationally cheaper: max(0, min(1, 0.2x + 0.5)).

Solution

import numpy as np

def hard_sigmoid(x: np.ndarray) -> np.ndarray:
    return np.clip(0.2 * x + 0.5, 0, 1)

Explanation

1. The Hard Sigmoid approximates the smooth sigmoid curve with three linear segments:
2. - For x <= -2.5: output is 0
3. - For x >= 2.5: output is 1
4. - For -2.5 < x < 2.5: output is 0.2x + 0.5 (linear interpolation)
5. np.clip efficiently handles the clamping to [0, 1].
6. This is much faster than computing 1 / (1 + exp(-x)) since it avoids the exponential function.

Complexity

• Time: O(n) where n is the number of elements
• Space: O(n) for the output array