MULTILAYER PERCEPTRON WITH ADAPTIVE ACTIVATION FUNCTIONS
View/ Open
Date
2016-05-26Author
Rane, Chinmay Appa
0000-0002-1373-819X
Metadata
Show full item recordAbstract
A Multilayer perceptron typically has a fixed nonlinear activation function for each hidden unit. In this thesis, an adaptive activation function for individual hidden unit is designed, where the network learns these activation functions at every iteration using a modern second order algorithm. Methods and algorithms for these adaptive activation functions along with several other techniques for training a multilayer perceptron’s weights are discussed.
Comparisons between a multilayer perceptron with sigmoidal activation functions and a multilayer perceptron with piecewise linear activation functions are also discussed. The common activation function used is the sigmoidal activations, but it is still not proven that the sigmoidal activations works best for all the applications. Hence the adaptive activation technique described in this thesis can be used, which learns independently as it passes through the data.