9 lines
400 B
Plaintext
9 lines
400 B
Plaintext
|
|
|||
|
|
Gelu takes one input data (Tensor<T>) and produces one
|
|||
|
|
output data (Tensor<T>) where the gaussian error linear units function,
|
|||
|
|
$y = 0.5 * x * (1 + erf(x/sqrt(2)))$ is applied to the tensor elementwise.
|
|||
|
|
If the attribute "approximate" is set to "tanh", the function estimation,
|
|||
|
|
$y = 0.5 * x * (1 + Tanh(sqrt(2/\pi) * (x + 0.044715 * x^3)))$ is used and applied
|
|||
|
|
to the tensor elementwise.
|
|||
|
|
|
|||
|
|
is `0`. <00><>
|