The Self-Organizing Restricted Boltzmann Machine for Deep Representation with the Application on Classification Problems
Recently, deep learning is proliferating in the field of representation learning. A deep belief network (DBN) consists of a deep network architecture that can generate multiple features of input patterns, using restricted Boltzmann machines (RBMs) as a building block of DBN. A deep learning model can achieve extremely high accuracy in many applications that depend on the model structure. However, specifying various parameters of deep network architecture like the number of hidden layers and neurons is a difficult task even for expert designers. Besides, the number of hidden layers and neurons is typically set manually, while this method is costly in terms of time and computational cost, especially in big data. In this paper, we introduce an approach to determine the number of hidden layers and neurons of the deep network automatically during the learning process. To this end, the input vector is transformed from the feature space with a low dimension into the new feature space with a high dimension in a hidden layer of RBM. In the following, new features are ranked according to their discrimination power between classes in the new space, using the Separability-correlation measure for feature importance ranking algorithm. The algorithm uses the mean of weights as a threshold, so the neurons whose weights exceed the threshold are retained, and the others are removed in the hidden layer. The number of retained neurons is presented as a reasonabl…