ActivationFunction

Function used to activate neuron.

Values of this enumeration:

• Sigmoidal - \( f(x) = \frac{1}{1+e^{-x} }\)
• Linear - Saturated linear function. \( f(x) = \begin{cases} 1 & \text{if } x > 1 \\ 0 & \text{if } x < 0 \\ x & \text{otherwise} \end{cases}\)
• Tanh - \(f(x) = \tanh(x) = \frac{\exp(2x)-1}{\exp(2x)+1}\)
• Log - Logarithmic function.
• ArcTan - \( f(x) = \arctan(x)\)
• Gauss - \( f(x) = e^{-x^{2} }\)
• Elliot - \( f(x) = \frac{x}{1+|x|}+0.5\)
• Step - \( f(x) = \begin{cases} 1 & \text{if } x > 0 \\ 0 & \text{otherwise} \end{cases}\)
• FastSigmoidal - The same as Sigmoidal, but uses optimized exponent function.
• FastTanh - The same as Tanh, but uses optimized exponent function.

namespace ActivationFunction
{
	enum Type
	{
		Sigmoidal,
		Linear,
		Tanh,
		Log,
		ArcTan,
		Gauss,
		Elliot,
		Step,
		FastSigmoidal,
		FastTanh
	};
}