Implement the Hardtanh Activation Function

Problem

Implement the Hardtanh activation function. Hardtanh clamps its input to the range [min_val, max_val], typically [-1, 1].

Solution

Clip each value to the specified range.

def hardtanh(x: float, min_val: float = -1.0, max_val: float = 1.0) -> float:
    if x < min_val:
        return min_val
    elif x > max_val:
        return max_val
    return x


def hardtanh_list(
    values: list[float], min_val: float = -1.0, max_val: float = 1.0
) -> list[float]:
    return [round(hardtanh(v, min_val, max_val), 6) for v in values]

Explanation

1. If x < min_val, return min_val.
2. If x > max_val, return max_val.
3. Otherwise, return x unchanged.
4. Hardtanh is a piecewise linear approximation of tanh. It is computationally cheaper and has constant gradients in the linear region.
5. The gradient is 1 in [min_val, max_val] and 0 outside, similar to a clipped ReLU.

Complexity

• Time: O(n) for n elements
• Space: O(n) for output