Bruges, Belgium, April 23-24-25
Content of the proceedings
-
Links between neural networks and webs
Self-organization and topology representation
Mathematical aspects of neural networks
ANN models and learning I
New directions in support vector machines and kernel based learning
ANN models and learning II
Biologically plausible learning
Mixtures and ensemble learning
Classification
Dynamical systems and recurrent networks
Industrial and agronomical applications of neural networks
ANN models and learning III
Digital image processing with neural networks
Links between neural networks and webs
ES2003-600
An introduction to learning in web domains
M. Diligenti, M. Gori, M. Maggini, F. Scarselli, A.C. Tsoi
An introduction to learning in web domains
M. Diligenti, M. Gori, M. Maggini, F. Scarselli, A.C. Tsoi
ES2003-22
Subject Categorization for Web Educational Resources using MLP
M. Nakayama, Y. Shimizu
Subject Categorization for Web Educational Resources using MLP
M. Nakayama, Y. Shimizu
Abstract:
The purpose of this study is to develop subject categorization methods for educational resources using multilayer perceptron (MLP) and to examine the performance of the test documents as an application system. To examine the performance two methods are examined: Latent Semantic Indexing method (LSI) and a three layer feedforward network as a simple MLP. The document vectors were estimated by the term feature vectors which were extracted from the teaching guidelines based on the singular value decomposition method (SVD). Comparing recall and precision rates and F1 measure for the subject categorization, the categorization performance using MLP showed better than using LSI.
The purpose of this study is to develop subject categorization methods for educational resources using multilayer perceptron (MLP) and to examine the performance of the test documents as an application system. To examine the performance two methods are examined: Latent Semantic Indexing method (LSI) and a three layer feedforward network as a simple MLP. The document vectors were estimated by the term feature vectors which were extracted from the teaching guidelines based on the singular value decomposition method (SVD). Comparing recall and precision rates and F1 measure for the subject categorization, the categorization performance using MLP showed better than using LSI.
ES2003-32
Modeling of growing networks with directional attachment and communities
M. Kimura, K. Saito, N. Ueda
Modeling of growing networks with directional attachment and communities
M. Kimura, K. Saito, N. Ueda
Abstract:
With the aim of acquiring a more precise probabilistic model for the future graph structure of such a real-world growing network as the Web, we propose a new network growth model and its learning algorithm. Unlike the conventional models, we have incorporated directional attachment and community structure for this purpose. We formally show that the proposed model also exhibits a degree distribution with a power-law tail. Using the real data of Web pages on the topic “mp3”, we experimentally show that the proposed method can more precisely predict the probability of a new link creation in the future.
With the aim of acquiring a more precise probabilistic model for the future graph structure of such a real-world growing network as the Web, we propose a new network growth model and its learning algorithm. Unlike the conventional models, we have incorporated directional attachment and community structure for this purpose. We formally show that the proposed model also exhibits a degree distribution with a power-law tail. Using the real data of Web pages on the topic “mp3”, we experimentally show that the proposed method can more precisely predict the probability of a new link creation in the future.
Self-organization and topology representation
ES2003-97
High-dimensional labeled data analysis with Gabriel graphs
M. Aupetit
High-dimensional labeled data analysis with Gabriel graphs
M. Aupetit
Abstract:
We propose the use of the Gabriel graph for the exploratory analysis of potentially high dimensional labeled data. Gabriel graph is a subgraph of the Delaunay triangulation, which connects two data points vi and vj for which there is no other point vk inside the open ball with diameter [vivj ]. If all the Gabriel neighbors of a datum have a di.erent class than its own, this datum is said to be ”isolated”. While if some of its Gabriel neighbors have the same class as its own and some others have not, then this datum is said to be ”border”. Isolated and border data together with Gabriel graph, allow to get informations about the topology of the di.erent classes in the data space. It is complementary with “classical” and “neural” projection techniques.
We propose the use of the Gabriel graph for the exploratory analysis of potentially high dimensional labeled data. Gabriel graph is a subgraph of the Delaunay triangulation, which connects two data points vi and vj for which there is no other point vk inside the open ball with diameter [vivj ]. If all the Gabriel neighbors of a datum have a di.erent class than its own, this datum is said to be ”isolated”. While if some of its Gabriel neighbors have the same class as its own and some others have not, then this datum is said to be ”border”. Isolated and border data together with Gabriel graph, allow to get informations about the topology of the di.erent classes in the data space. It is complementary with “classical” and “neural” projection techniques.
ES2003-27
Unsupervised Recursive Sequence Processing
M. Strickert, B. Hammer
Unsupervised Recursive Sequence Processing
M. Strickert, B. Hammer
Abstract:
We propose a self organizing map (SOM) for sequences by extending standard SOM by two features, the recursive update of Sperduti [7] and the hyperbolic neighborhood of Ritter [5]. While the former integrates the currently presented item and recent map activations, the latter allows representation of temporally possibly exponentially growing sequence diversification. Discrete and real-valued sequences can be processed efficiently with this method as demonstrated in three experiments.
We propose a self organizing map (SOM) for sequences by extending standard SOM by two features, the recursive update of Sperduti [7] and the hyperbolic neighborhood of Ritter [5]. While the former integrates the currently presented item and recent map activations, the latter allows representation of temporally possibly exponentially growing sequence diversification. Discrete and real-valued sequences can be processed efficiently with this method as demonstrated in three experiments.
ES2003-62
Neural networks organizations to learn complex robotic functions
G. Hermann, P. Wira, J-P. Urban
Neural networks organizations to learn complex robotic functions
G. Hermann, P. Wira, J-P. Urban
Abstract:
This paper considers the general problem of function estimation with a modular approach of neural computing. We propose to use functionally independent subnetworks to learn complex functions. Thus, function approximation is decomposed and amounts to estimate different elementary sub-functions rather than the whole function with a single network. This modular decomposition is a way to introduce some a priori knowledge in neural estimation. Functionally independent subnetworks are obtained with a bidirectional learning scheme. Implemented with self-organizing maps, the modular approach has been applied to a robot control problem, a robot positioning task.
This paper considers the general problem of function estimation with a modular approach of neural computing. We propose to use functionally independent subnetworks to learn complex functions. Thus, function approximation is decomposed and amounts to estimate different elementary sub-functions rather than the whole function with a single network. This modular decomposition is a way to introduce some a priori knowledge in neural estimation. Functionally independent subnetworks are obtained with a bidirectional learning scheme. Implemented with self-organizing maps, the modular approach has been applied to a robot control problem, a robot positioning task.
ES2003-80
Self-organizing maps and functional networks for local dynamic modeling
N. Sánchez-Maroño, O. Fontenla-Romero, A. Alonso-Betanzos, B. Guijarro-Berdiñas
Self-organizing maps and functional networks for local dynamic modeling
N. Sánchez-Maroño, O. Fontenla-Romero, A. Alonso-Betanzos, B. Guijarro-Berdiñas
Abstract:
The paper presents a method for times series prediction using a local dynamic modeling based on a three step process. In the first step the input data is embedded in a reconstruction space using a memory structure. The second step, implemented by a self-organizing map (SOM), derives a set of local models from data. The third step is accomplished by a set of functional networks. The goal of the last network is to fit a local model from the winning neuron and a set of neighbors of the SOM map. Finally, the performance of the proposed method was validated using two chaotic time series.
The paper presents a method for times series prediction using a local dynamic modeling based on a three step process. In the first step the input data is embedded in a reconstruction space using a memory structure. The second step, implemented by a self-organizing map (SOM), derives a set of local models from data. The third step is accomplished by a set of functional networks. The goal of the last network is to fit a local model from the winning neuron and a set of neighbors of the SOM map. Finally, the performance of the proposed method was validated using two chaotic time series.
ES2003-44
Robust Topology Representing Networks
M. Aupetit
Robust Topology Representing Networks
M. Aupetit
Abstract:
Martinetz and Schulten proposed the use of a Competitive Hebbian Learning (CHL) rule to build Topology Representing Networks. From a set of units and a data distribution, a link is created between the first and second closest units to each datum, creating a graph which preserves the topology of the data set. However, one has to deal with finite data distributions generally corrupted with noise, for which CHL may be unefficient. We propose a more robust approach to create a topology representing graph, by considering the density of the data distribution.
Martinetz and Schulten proposed the use of a Competitive Hebbian Learning (CHL) rule to build Topology Representing Networks. From a set of units and a data distribution, a link is created between the first and second closest units to each datum, creating a graph which preserves the topology of the data set. However, one has to deal with finite data distributions generally corrupted with noise, for which CHL may be unefficient. We propose a more robust approach to create a topology representing graph, by considering the density of the data distribution.
ES2003-48
Visualizing asymmetric proximities with MDS models
A. Muñoz, M. Martin-Merino
Visualizing asymmetric proximities with MDS models
A. Muñoz, M. Martin-Merino
Abstract:
\N
\N
Mathematical aspects of neural networks
ES2003-601
Mathematical Aspects of Neural Networks
B. Hammer, T. Villmann
Mathematical Aspects of Neural Networks
B. Hammer, T. Villmann
Abstract:
In this tutorial paper about mathematical aspects of neural networks, we will focus on two directions: on the one hand, we will motivate standard mathematical questions and well studied theory of classical neural models used in machine learning. On the other hand, we collect some recent theoretical results (as of beginning of 2003) in the respective areas. Thereby, we follow the dichotomy offered by the overall network structure and restrict ourselves to feedforward networks, recurrent networks, and self-organizing neural systems, respectively.
In this tutorial paper about mathematical aspects of neural networks, we will focus on two directions: on the one hand, we will motivate standard mathematical questions and well studied theory of classical neural models used in machine learning. On the other hand, we collect some recent theoretical results (as of beginning of 2003) in the respective areas. Thereby, we follow the dichotomy offered by the overall network structure and restrict ourselves to feedforward networks, recurrent networks, and self-organizing neural systems, respectively.
ES2003-58
On the weight dynamics of recurrent learning
U. D. Schiller, J. J. Steil
On the weight dynamics of recurrent learning
U. D. Schiller, J. J. Steil
Abstract:
We derive continuous-time batch and online versions of the recently introduced efficient O(N2) training algorithm of Atiya and Parlos [2000] for fully recurrent networks. A mathematical analysis of the respective weight dynamics yields that efficient learning is achieved although relative rates of weight change remain constant due to the way errors are backpropagated. The result is a highly structured network where an unspecific internal dynamical reservoir can be distinguished from the output layer, which learns faster and changes at much higher rates. We discuss this result with respect to the recently introduced “echo state” and “liquid state” networks, which have similar structure.
We derive continuous-time batch and online versions of the recently introduced efficient O(N2) training algorithm of Atiya and Parlos [2000] for fully recurrent networks. A mathematical analysis of the respective weight dynamics yields that efficient learning is achieved although relative rates of weight change remain constant due to the way errors are backpropagated. The result is a highly structured network where an unspecific internal dynamical reservoir can be distinguished from the output layer, which learns faster and changes at much higher rates. We discuss this result with respect to the recently introduced “echo state” and “liquid state” networks, which have similar structure.
ES2003-66
A Neural Graph Isomorphism Algorithm based on local Invariants
B.J. Jain, F. Wysotzki
A Neural Graph Isomorphism Algorithm based on local Invariants
B.J. Jain, F. Wysotzki
ES2003-112
Analyzing surveys using the Kohonen algorithm
M. Cottrell, P. Letremy
Analyzing surveys using the Kohonen algorithm
M. Cottrell, P. Letremy
Abstract:
The Kohonen algorithm (SOM, Kohonen 1995) is a very powerful tool for data analysis. Most of the time, each observation is a p-vector of numerical values. But in many cases, for survey analysis for example, the observations are described by qualitative variables with a finite number of modalities. In that case, we define a specific algorithm (KDISJ) which provides a simultaneous classification of the observations and of the modalities.
The Kohonen algorithm (SOM, Kohonen 1995) is a very powerful tool for data analysis. Most of the time, each observation is a p-vector of numerical values. But in many cases, for survey analysis for example, the observations are described by qualitative variables with a finite number of modalities. In that case, we define a specific algorithm (KDISJ) which provides a simultaneous classification of the observations and of the modalities.
ES2003-56
Magnification Control in Winner Relaxing Neural Gas
J. C. Claussen, T. Villmann
Magnification Control in Winner Relaxing Neural Gas
J. C. Claussen, T. Villmann
Abstract:
We transfer the idea of winner relaxing learning from the self-organizing map to the neural gas to enable magnification control independently of the shape of the data distribution.
We transfer the idea of winner relaxing learning from the self-organizing map to the neural gas to enable magnification control independently of the shape of the data distribution.
ES2003-109
On Convergence Problems of the EM Algorithm for Finite Gaussian Mixtures
C. Archambeau, J. A. Lee, M. Verleysen
On Convergence Problems of the EM Algorithm for Finite Gaussian Mixtures
C. Archambeau, J. A. Lee, M. Verleysen
Abstract:
Efficient probability density function estimation is of primary interest in statistics. A popular approach for achieving this is the use of finite Gaussian mixture models. Based on the expectation-maximization algorithm, the maximum likelihood estimates of the model parameters can be iteratively computed in an elegant way. Unfortunately, in some cases the algorithm is not converging properly because of numerical difficulties. They are of two kinds: they can be associated to outliers or to repeated data samples. In this paper, we trace and discuss their origin while providing some theoretical evidence. As a matter of fact, both can be explained by the concept of isolation, which is leading to the width of the collapsing mixture component to become zero.
Efficient probability density function estimation is of primary interest in statistics. A popular approach for achieving this is the use of finite Gaussian mixture models. Based on the expectation-maximization algorithm, the maximum likelihood estimates of the model parameters can be iteratively computed in an elegant way. Unfortunately, in some cases the algorithm is not converging properly because of numerical difficulties. They are of two kinds: they can be associated to outliers or to repeated data samples. In this paper, we trace and discuss their origin while providing some theoretical evidence. As a matter of fact, both can be explained by the concept of isolation, which is leading to the width of the collapsing mixture component to become zero.
ES2003-64
An Associative Memory for the Automorphism Group of Structures
B.J. Jain, F. Wysotzki
An Associative Memory for the Automorphism Group of Structures
B.J. Jain, F. Wysotzki
ES2003-30
Approximation of Function by Adaptively Growing Radial Basis Function Neural Networks
L. Jianyu, L. Siwei, Q. Yingjian
Approximation of Function by Adaptively Growing Radial Basis Function Neural Networks
L. Jianyu, L. Siwei, Q. Yingjian
Abstract:
In this paper a neural network for approximating function is described. The activation functions of the hidden nodes are the Radial Basis Functions (RBF) whose parameters are learnt by a two-stage gradient descent strategy. A new growing radial basis functions node insertion strategy with different radial basis functions is used in order to improve the net performances. The learning strategy is able to save computational time and memory space because of the selective growing of nodes whose activation functions consist of different radial basis functions. An analysis of the learning capabilities and a comparison of the net performances with other approaches have been performed. It is shown that the resulting network improves the approximation results.
In this paper a neural network for approximating function is described. The activation functions of the hidden nodes are the Radial Basis Functions (RBF) whose parameters are learnt by a two-stage gradient descent strategy. A new growing radial basis functions node insertion strategy with different radial basis functions is used in order to improve the net performances. The learning strategy is able to save computational time and memory space because of the selective growing of nodes whose activation functions consist of different radial basis functions. An analysis of the learning capabilities and a comparison of the net performances with other approaches have been performed. It is shown that the resulting network improves the approximation results.
ANN models and learning I
ES2003-8
Cellular topographic self-organization under correlational learning
S. Sakamoto, S. Seki, Y. Kobuchi
Cellular topographic self-organization under correlational learning
S. Sakamoto, S. Seki, Y. Kobuchi
Abstract:
We consider two layered binary state neural networks in which cellular topographic self-organization occurs under correlational learning. The main result is that for separable input relations, a mapping is topographic if it is stable and vice versa.
We consider two layered binary state neural networks in which cellular topographic self-organization occurs under correlational learning. The main result is that for separable input relations, a mapping is topographic if it is stable and vice versa.
ES2003-47
Self-Organization by Optimizing Free-Energy
J.J. Verbeek, N. Vlassis, B.J.A. Kröse
Self-Organization by Optimizing Free-Energy
J.J. Verbeek, N. Vlassis, B.J.A. Kröse
Abstract:
We present a variational Expectation-Maximization algorithm to learn probabilistic mixture models. The algorithm is similar to Kohonen's Self-Organizing Map algorithm and not limited to Gaussian mixtures. We maximize the variational free-energy that sums data log-likelihood and Kullback-Leibler divergence between a normalized neighborhood function and the posterior distribution on the components, given data. We illustrate the algorithm with an application on word clustering.
We present a variational Expectation-Maximization algorithm to learn probabilistic mixture models. The algorithm is similar to Kohonen's Self-Organizing Map algorithm and not limited to Gaussian mixtures. We maximize the variational free-energy that sums data log-likelihood and Kullback-Leibler divergence between a normalized neighborhood function and the posterior distribution on the components, given data. We illustrate the algorithm with an application on word clustering.
ES2003-37
Comparison of neural algorithms for blind source separation in sensor array applications
G. Bedoya, S. Bermejo, J. Cabestany
Comparison of neural algorithms for blind source separation in sensor array applications
G. Bedoya, S. Bermejo, J. Cabestany
Abstract:
A test bed of experiments with real and artificially generated data has been designed to compare the performance of three well-known algorithms for BSS. The main goal of these experiments was to extract some guidelines for their use in practical applications concerning their efficiency, accuracy, convergence speed, stability, and robustness under the presence of Gaussian noise and in presence of a large number of source signals.
A test bed of experiments with real and artificially generated data has been designed to compare the performance of three well-known algorithms for BSS. The main goal of these experiments was to extract some guidelines for their use in practical applications concerning their efficiency, accuracy, convergence speed, stability, and robustness under the presence of Gaussian noise and in presence of a large number of source signals.
ES2003-71
Neural Net with Two Hidden Layers for Non-Linear Blind Source Separation
R. Martín-Clemente, S. Hornillo-Mellado, J. I. Acha, F. Rojas, C. G. Puntonet
Neural Net with Two Hidden Layers for Non-Linear Blind Source Separation
R. Martín-Clemente, S. Hornillo-Mellado, J. I. Acha, F. Rojas, C. G. Puntonet
Abstract:
In this paper, we present an algorithm that minimizes the mutual information between the outputs of a perceptron with two hidden layers. The neural network is then used as separating system in the NonLinear Blind Source Separation problem.
In this paper, we present an algorithm that minimizes the mutual information between the outputs of a perceptron with two hidden layers. The neural network is then used as separating system in the NonLinear Blind Source Separation problem.
ES2003-111
Acceptability conditions for BSS problems
V. Vigneron, S. Lagrange, C. Jutten
Acceptability conditions for BSS problems
V. Vigneron, S. Lagrange, C. Jutten
ES2003-9
Extraction of fuzzy rules from trained neural network using evolutionary algorithm
U. Markowska-Kaczmar, W. Trelak
Extraction of fuzzy rules from trained neural network using evolutionary algorithm
U. Markowska-Kaczmar, W. Trelak
Abstract:
This paper presents our approach to the rule extraction problem from trained neural network. A method called REX is briefly described. REX acquires a set of fuzzy rules using an evolutionary algorithm. Evolutionary algorithm searches not only fuzzy rules, but also a description of fuzzy sets. The way of coding and evaluation process of an individual is presented. The method was tested using the following benchmark data sets: IRIS, WINE and Wisconsin Breast Cancer Diagnosis. On the basis of the experimental studies shown in this paper, we can conclude that rules obtained by REX can be easily understood by human – they include small number of premises, and their fidelity is very high. Obtained results are compared to other rule extraction methods.
This paper presents our approach to the rule extraction problem from trained neural network. A method called REX is briefly described. REX acquires a set of fuzzy rules using an evolutionary algorithm. Evolutionary algorithm searches not only fuzzy rules, but also a description of fuzzy sets. The way of coding and evaluation process of an individual is presented. The method was tested using the following benchmark data sets: IRIS, WINE and Wisconsin Breast Cancer Diagnosis. On the basis of the experimental studies shown in this paper, we can conclude that rules obtained by REX can be easily understood by human – they include small number of premises, and their fidelity is very high. Obtained results are compared to other rule extraction methods.
ES2003-13
A new rule extraction algorithm based on interval arithmetic
C. Hernandez-Espinosa, M. Fernandez-Redondo, M. Ortiz-Gómez
A new rule extraction algorithm based on interval arithmetic
C. Hernandez-Espinosa, M. Fernandez-Redondo, M. Ortiz-Gómez
Abstract:
In this paper we propose a new algorithm for rule extraction from a trained Multilayer Feedforward network. The algorithm is based on an interval arithmetic network inversion for particular target outputs. The types of rules extracted are N-dimensional intervals in the input space. We have performed experiments with four database and the results are very interesting. One rule extracted by the algorithm can cover 86% of the neural network output and in other cases 64 rules cover 100% of the neural network output.
In this paper we propose a new algorithm for rule extraction from a trained Multilayer Feedforward network. The algorithm is based on an interval arithmetic network inversion for particular target outputs. The types of rules extracted are N-dimensional intervals in the input space. We have performed experiments with four database and the results are very interesting. One rule extracted by the algorithm can cover 86% of the neural network output and in other cases 64 rules cover 100% of the neural network output.
ES2003-10
Searching optimal feature subset using mutual information
D. Huang, T.W.S. Chow
Searching optimal feature subset using mutual information
D. Huang, T.W.S. Chow
Abstract:
A novel feature selection methodology is proposed with the concept of mutual information. The proposed methodology effectively circumvents two major problems in feature selection process: to identify the irrelevancy and redundancy in the feature set, and to estimate the optimal feature subset for classification task.
A novel feature selection methodology is proposed with the concept of mutual information. The proposed methodology effectively circumvents two major problems in feature selection process: to identify the irrelevancy and redundancy in the feature set, and to estimate the optimal feature subset for classification task.
ES2003-19
Statistical downscaling with artificial neural networks
G. C. Cawley, M. Haylock, S. R. Dorling, C. Goodess, P. D. Jones
Statistical downscaling with artificial neural networks
G. C. Cawley, M. Haylock, S. R. Dorling, C. Goodess, P. D. Jones
Abstract:
Statistical downscaling methods seek to model the relationship between large scale atmospheric circulation, on say a European scale, and climatic variables, such as temperature and precipitation, on a regional or sub- regional scale. Downscaling is an important area of research as it bridges the gap between predictions of future circulation generated by General Circulation Models (GCMs) and the effects of climate change on smaller scales, which are often of greater interest to end-users. In this paper we describe a neural network based approach to statistical downscaling, with application to the analysis of events associated with extreme precipitation in the United Kingdom.
Statistical downscaling methods seek to model the relationship between large scale atmospheric circulation, on say a European scale, and climatic variables, such as temperature and precipitation, on a regional or sub- regional scale. Downscaling is an important area of research as it bridges the gap between predictions of future circulation generated by General Circulation Models (GCMs) and the effects of climate change on smaller scales, which are often of greater interest to end-users. In this paper we describe a neural network based approach to statistical downscaling, with application to the analysis of events associated with extreme precipitation in the United Kingdom.
ES2003-31
Online Identification and Control of a PV-Supplied DC Motor Using Universal Learning Networks
A. Hussein, K. Hirasawa, J. Hu
Online Identification and Control of a PV-Supplied DC Motor Using Universal Learning Networks
A. Hussein, K. Hirasawa, J. Hu
Abstract:
This paper describes the use of Universal Learning Networks (ULNs) in the speed control of a separately excited DC motor drives fed from Photovoltaic (PV) generators through intermediate power converters. Two ULNs-based identification and control are used. Their free parameters are updated online concurrently by the forward propagation algorithm. The identifier network is used to capture and emulate the nonlinear mappings between the inputs and outputs of the motor system. The controller network is used to control the converter duty ratio so that the motor speed can follow an arbitrarily reference signal. Moreover the overall system can operate at the Maximum Power Point (MPP) of the PV source. The simulation results showed a good performance for the controller and the identifier during the training mode and the continuous running mode as well.
This paper describes the use of Universal Learning Networks (ULNs) in the speed control of a separately excited DC motor drives fed from Photovoltaic (PV) generators through intermediate power converters. Two ULNs-based identification and control are used. Their free parameters are updated online concurrently by the forward propagation algorithm. The identifier network is used to capture and emulate the nonlinear mappings between the inputs and outputs of the motor system. The controller network is used to control the converter duty ratio so that the motor speed can follow an arbitrarily reference signal. Moreover the overall system can operate at the Maximum Power Point (MPP) of the PV source. The simulation results showed a good performance for the controller and the identifier during the training mode and the continuous running mode as well.
ES2003-34
On radial basis function network equalization in the GSM system
A. Kantsila, M. Lehtokangas, J. Saarinen
On radial basis function network equalization in the GSM system
A. Kantsila, M. Lehtokangas, J. Saarinen
Abstract:
In this paper we have studied adaptive equalization in the GSM (Global System for Mobile communications) environment using radial basis function (RBF) networks. Equalization is here considered as a classification problem, where the idea is to map the received complex-valued signal into desired binary values using RBF network equalizer. Results prove that the RBF network provides very good bit error rates with acceptable computational complexity. Performance comparisons are made to a linear equalizer, a multilayer perceptron (MLP) network equalizer and to a Viterbi equalizer.
In this paper we have studied adaptive equalization in the GSM (Global System for Mobile communications) environment using radial basis function (RBF) networks. Equalization is here considered as a classification problem, where the idea is to map the received complex-valued signal into desired binary values using RBF network equalizer. Results prove that the RBF network provides very good bit error rates with acceptable computational complexity. Performance comparisons are made to a linear equalizer, a multilayer perceptron (MLP) network equalizer and to a Viterbi equalizer.
New directions in support vector machines and kernel based learning
ES2003-602
Reproducing kernels and regularization methods in machine learning
M. Pontil
Reproducing kernels and regularization methods in machine learning
M. Pontil
ES2003-117
On different ensembles of kernel machines
M. Yamana, H. Nakahara, M. Pontil, S. Amari
On different ensembles of kernel machines
M. Yamana, H. Nakahara, M. Pontil, S. Amari
ES2003-98
Kernel PLS variants for regression
L. Hoegaerts, J.A.K. Suykens, J. Vandewalle, B. De Moor
Kernel PLS variants for regression
L. Hoegaerts, J.A.K. Suykens, J. Vandewalle, B. De Moor
Abstract:
We focus on covariance criteria for finding a suitable sub-space for regression in a reproducing kernel Hilbert space: kernel principal component analysis, kernel partial least squares and kernel canonical correlation analysis, and we demonstrate how this fits within a more general context of subspace regression. For the kernel partial least squares case some variants are considered and the methods are illustrated and compared on a number of examples.
We focus on covariance criteria for finding a suitable sub-space for regression in a reproducing kernel Hilbert space: kernel principal component analysis, kernel partial least squares and kernel canonical correlation analysis, and we demonstrate how this fits within a more general context of subspace regression. For the kernel partial least squares case some variants are considered and the methods are illustrated and compared on a number of examples.
ES2003-18
Approximately unbiased estimation of conditional variance in heteroscedastic kernel ridge regression
G.C. Cawley, N.L.C. Talbot, R.J. Foxall, S.R. Dorling, D.P. Mandic
Approximately unbiased estimation of conditional variance in heteroscedastic kernel ridge regression
G.C. Cawley, N.L.C. Talbot, R.J. Foxall, S.R. Dorling, D.P. Mandic
Abstract:
In this paper we extend a form of kernel ridge regression for data characterised by a heteroscedastic noise process (introduced in Foxall et al. [1]) in order to provide approximately unbiased estimates of the conditional variance of the target distribution. This is achieved by the use of the leave-one-out cross-validation estimate of the conditional mean when fitting the model of the conditional variance. The elimination of this bias is demonstrated on synthetic dataset where the true conditional variance is known.
In this paper we extend a form of kernel ridge regression for data characterised by a heteroscedastic noise process (introduced in Foxall et al. [1]) in order to provide approximately unbiased estimates of the conditional variance of the target distribution. This is achieved by the use of the leave-one-out cross-validation estimate of the conditional mean when fitting the model of the conditional variance. The elimination of this bias is demonstrated on synthetic dataset where the true conditional variance is known.
ES2003-33
On the equality of kernel AdaTron and sequential minimal optimization in classification and regression tasks and alike algorithms for kernel machines
V. Kecman, M. Vogt, T.M. Huang
On the equality of kernel AdaTron and sequential minimal optimization in classification and regression tasks and alike algorithms for kernel machines
V. Kecman, M. Vogt, T.M. Huang
Abstract:
The paper presents the equality of a kernel AdaTron (KA) method (originating from a gradient ascent learning approach) and sequential minimal optimization (SMO) learning algorithm (based on an analytic quadratic programming step) in designing the support vector machines (SVMs) having positive definite kernels. The conditions of the equality of two methods are established. The equality is valid for both the nonlinear classification and the nonlinear regression tasks, and it sheds a new light to these seemingly different learning approaches. The paper also introduces other learning techniques related to the two mentioned approaches, such as the nonnegative conjugate gradient, classic Gauss-Seidel (GS) coordinate ascent procedure and its derivative known as the successive over-relaxation (SOR) algorithm as a viable and usually faster training algorithms for performing nonlinear classification and regression tasks. The convergence theorem for these related iterative algorithms is proven.
The paper presents the equality of a kernel AdaTron (KA) method (originating from a gradient ascent learning approach) and sequential minimal optimization (SMO) learning algorithm (based on an analytic quadratic programming step) in designing the support vector machines (SVMs) having positive definite kernels. The conditions of the equality of two methods are established. The equality is valid for both the nonlinear classification and the nonlinear regression tasks, and it sheds a new light to these seemingly different learning approaches. The paper also introduces other learning techniques related to the two mentioned approaches, such as the nonnegative conjugate gradient, classic Gauss-Seidel (GS) coordinate ascent procedure and its derivative known as the successive over-relaxation (SOR) algorithm as a viable and usually faster training algorithms for performing nonlinear classification and regression tasks. The convergence theorem for these related iterative algorithms is proven.
ES2003-46
Finding clusters using support vector classifiers
K. Jong, E. Marchiori, A. van der Vaart
Finding clusters using support vector classifiers
K. Jong, E. Marchiori, A. van der Vaart
Abstract:
This paper shows how clustering can be performed by using support vector classi.ers and model selection. We introduce a heuristic method for non-parametric clustering that uses support vector classifiers for finding support vectors describing portions of clusters and uses a model selection criterion for joining these portions. Clustering is viewed as a two-class classification problem and a soft-margin support vector classifier is used for separating clusters from other points suitably sampled in the data space. The method is tested on five real life data sets, including microarray gene expression data and array-CGH data.
This paper shows how clustering can be performed by using support vector classi.ers and model selection. We introduce a heuristic method for non-parametric clustering that uses support vector classifiers for finding support vectors describing portions of clusters and uses a model selection criterion for joining these portions. Clustering is viewed as a two-class classification problem and a soft-margin support vector classifier is used for separating clusters from other points suitably sampled in the data space. The method is tested on five real life data sets, including microarray gene expression data and array-CGH data.
ES2003-12
Classification of handwritten digits using supervised locally linear embedding algorithm and support vector machine
O. Kouropteva, O. Okun, M. Pietikäinen
Classification of handwritten digits using supervised locally linear embedding algorithm and support vector machine
O. Kouropteva, O. Okun, M. Pietikäinen
Abstract:
The locally linear embedding (LLE) algorithm is an unsupervised technique recently proposed for nonlinear dimensionality reduction. In this paper, we describe its supervised variant (SLLE). This is a conceptually new method, where class membership information is used to map overlapping high dimensional data into disjoint clusters in the embedded space. In experiments, we combined it with support vector machine (SVM) for classifying handwritten digits from the MNIST database.
The locally linear embedding (LLE) algorithm is an unsupervised technique recently proposed for nonlinear dimensionality reduction. In this paper, we describe its supervised variant (SLLE). This is a conceptually new method, where class membership information is used to map overlapping high dimensional data into disjoint clusters in the embedded space. In experiments, we combined it with support vector machine (SVM) for classifying handwritten digits from the MNIST database.
ES2003-51
Improving iterative repair strategies for scheduling with the SVM
K. Gersmann, B. Hammer
Improving iterative repair strategies for scheduling with the SVM
K. Gersmann, B. Hammer
Abstract:
Resource constraint project scheduling (RCPSP) is an NPhard benchmark problem in scheduling which takes into account the limitation of resources’ availabilities in real life production processes. We here present an application of machine learning to adapt simple greedy strategies. The rout-algorithm of reinforcement learning is combined with the support vector machine (SVM) for value function approximation. The specific properties of the SVM allow to reduce the size of the training set and show improved results even after a short period of training.
Resource constraint project scheduling (RCPSP) is an NPhard benchmark problem in scheduling which takes into account the limitation of resources’ availabilities in real life production processes. We here present an application of machine learning to adapt simple greedy strategies. The rout-algorithm of reinforcement learning is combined with the support vector machine (SVM) for value function approximation. The specific properties of the SVM allow to reduce the size of the training set and show improved results even after a short period of training.
ES2003-17
Efficient cross-validation of kernel fisher discriminant classifiers
G. C. Cawley, N. L. C. Talbot
Efficient cross-validation of kernel fisher discriminant classifiers
G. C. Cawley, N. L. C. Talbot
Abstract:
Mika et al. [1] introduce a non-linear formulation of the Fisher discriminant based the well-known "kernel trick", later shown to be equivalent to the Least-Squares Support Vector Machine [2, 3]. In this paper, we show that the cross-validation error can be computed very efficiently for this class of kernel machine, specifically that leave-one-out cross-validation can be performed with a computational complexity of only O(l3) operations (the same as that of the basic training algorithm), rather than the O(l4) of a direct implementation. This makes leave-one-out cross-validation a practical proposition for model selection in much larger scale applications of KFD classifiers.
Mika et al. [1] introduce a non-linear formulation of the Fisher discriminant based the well-known "kernel trick", later shown to be equivalent to the Least-Squares Support Vector Machine [2, 3]. In this paper, we show that the cross-validation error can be computed very efficiently for this class of kernel machine, specifically that leave-one-out cross-validation can be performed with a computational complexity of only O(l3) operations (the same as that of the basic training algorithm), rather than the O(l4) of a direct implementation. This makes leave-one-out cross-validation a practical proposition for model selection in much larger scale applications of KFD classifiers.
ANN models and learning II
ES2003-1
RetinotopicNET: An Efficient Simulator for Retinotopic Visual Architectures
R. C. Muresan
RetinotopicNET: An Efficient Simulator for Retinotopic Visual Architectures
R. C. Muresan
Abstract:
RetinotopicNET is an efficient simulator for neural architectures with retinotopic-like receptive fields. The system has two main characteristics: it is event-driven and it takes advantage of the retinotopic arrangement in the receptive fields. The dynamics of the simulator are driven by the spike events of the simple integrate-and-fire neurons. By using an implicit synaptic rule to represent the synapses, RetinotopicNET achieves a great reduction of memory requirement for simulation. We show that under such conditions the system is fit for the simulation of very large networks of integrate-and-fire neurons. Furthermore we test RetinotopicNET in the simulation of a complex neural architecture for the ventral visual pathway. We prove that the system is linearly scalable with respect to the number of neurons in the simulation.
RetinotopicNET is an efficient simulator for neural architectures with retinotopic-like receptive fields. The system has two main characteristics: it is event-driven and it takes advantage of the retinotopic arrangement in the receptive fields. The dynamics of the simulator are driven by the spike events of the simple integrate-and-fire neurons. By using an implicit synaptic rule to represent the synapses, RetinotopicNET achieves a great reduction of memory requirement for simulation. We show that under such conditions the system is fit for the simulation of very large networks of integrate-and-fire neurons. Furthermore we test RetinotopicNET in the simulation of a complex neural architecture for the ventral visual pathway. We prove that the system is linearly scalable with respect to the number of neurons in the simulation.
ES2003-4
Accelerating the convergence speed of neural networks learning methods using least squares
O. Fontenla-Romero, D. Erdogmus, J. C. Principe, A. Alonso-Betanzos,
Accelerating the convergence speed of neural networks learning methods using least squares
O. Fontenla-Romero, D. Erdogmus, J. C. Principe, A. Alonso-Betanzos,
Abstract:
Abstract In this work a hybrid training scheme for the supervised learning of feedforward neural networks is presented. In the proposed method, the weights of the last layer are obtained employing linear least squares while the weights of the previous layers are updated using a standard learning method. The goal of this hybrid method is to assist the existing learning algorithms in accelerating their convergence. Simulations performed on two data sets show that the proposed method outperforms, in terms of convergence speed, the Levenberg-Marquardt algorithm.
Abstract In this work a hybrid training scheme for the supervised learning of feedforward neural networks is presented. In the proposed method, the weights of the last layer are obtained employing linear least squares while the weights of the previous layers are updated using a standard learning method. The goal of this hybrid method is to assist the existing learning algorithms in accelerating their convergence. Simulations performed on two data sets show that the proposed method outperforms, in terms of convergence speed, the Levenberg-Marquardt algorithm.
ES2003-14
Ensemble Methods for Multilayer Feedforward
C. Hernandez-Espinosa, M. Fernandez-Redondo, M. Ortiz-Gómez
Ensemble Methods for Multilayer Feedforward
C. Hernandez-Espinosa, M. Fernandez-Redondo, M. Ortiz-Gómez
Abstract:
Training an ensemble of networks is an interesting way to improve the performance with respect to a single network. However there are several method to construct the ensemble and there are no results showing which one could be the most appropriate. In this paper we present a comparison of eleven different method. We have trained ensembles of a reduced number of networks (3 and 9) because in this case the computational cost is not high and the method is suitable for applications. The results show that the improvement in performance from three to nine networks is marginal. Also the improvement of performance of the different methods with respect to a simple ensemble is usually less than 1%.
Training an ensemble of networks is an interesting way to improve the performance with respect to a single network. However there are several method to construct the ensemble and there are no results showing which one could be the most appropriate. In this paper we present a comparison of eleven different method. We have trained ensembles of a reduced number of networks (3 and 9) because in this case the computational cost is not high and the method is suitable for applications. The results show that the improvement in performance from three to nine networks is marginal. Also the improvement of performance of the different methods with respect to a simple ensemble is usually less than 1%.
ES2003-23
Robust Vector Quantization for Burst Error Channels Using Genetic Algorithm
W.-J. Hwang, C.-M. Ou, C.-M. Yeh
Robust Vector Quantization for Burst Error Channels Using Genetic Algorithm
W.-J. Hwang, C.-M. Ou, C.-M. Yeh
Abstract:
This paper presents a novel vector quantizer (VQ) design algorithm for a burst error channel (BEC). The algorithm minimizes the average distortion when the BEC is in normal state of operation, while maintaining a minimum fidelity when the BEC is in the undesirable state. An iterative design procedure is first derived in the algorithm for obtaining a local optimal solution. A novel genetic scheme is then proposed for attaining a near global optimal performance. Numerical results show that the algorithm significantly outperforms the VQ techniques optimizing the design only to the simple binary symmetric channels.
This paper presents a novel vector quantizer (VQ) design algorithm for a burst error channel (BEC). The algorithm minimizes the average distortion when the BEC is in normal state of operation, while maintaining a minimum fidelity when the BEC is in the undesirable state. An iterative design procedure is first derived in the algorithm for obtaining a local optimal solution. A novel genetic scheme is then proposed for attaining a near global optimal performance. Numerical results show that the algorithm significantly outperforms the VQ techniques optimizing the design only to the simple binary symmetric channels.
ES2003-24
Neural coding model using the morphoelectrotonic transform theory
N. Watanabe, S. Ishizaki
Neural coding model using the morphoelectrotonic transform theory
N. Watanabe, S. Ishizaki
Abstract:
It is believed that the color and the shape movement etc. of objects are expressed with time series of neuron spikes in the brain. We do not have any answer to realize the process, “What kind of neuron dynamics works and is able to express the processes?” To solve this problem, we need at least to study the neural coding where information is expressed by the spike sequence. In this research, it introduces the neural coding that uses the theory of morphoelectorotonic transform. The morphoelectorotonic transform is a theory based on the experiment on neurophysiology, and the neuron circuit can change dynamically depending on this coding model. Moreover, a simulation model that solved the binding problem to use this coding model is constructed, and significant of coding is verified.
It is believed that the color and the shape movement etc. of objects are expressed with time series of neuron spikes in the brain. We do not have any answer to realize the process, “What kind of neuron dynamics works and is able to express the processes?” To solve this problem, we need at least to study the neural coding where information is expressed by the spike sequence. In this research, it introduces the neural coding that uses the theory of morphoelectorotonic transform. The morphoelectorotonic transform is a theory based on the experiment on neurophysiology, and the neuron circuit can change dynamically depending on this coding model. Moreover, a simulation model that solved the binding problem to use this coding model is constructed, and significant of coding is verified.
ES2003-35
Neural assembly binding in linguistic representation
F. van der Velde, M. de Kamps
Neural assembly binding in linguistic representation
F. van der Velde, M. de Kamps
Abstract:
We present a neural architecture of sentence representation. Words are represented with neural cell assemblies. Relations between words are represented with ‘structure’ assemblies. Word and structure assemblies are bound temporarily to form a sentence representation. We show how multiple sentences can be represented simultaneously, and we simulate how specific information can be retrieved from the architecture. The assemblies are simulated as populations of spiking neurons, in terms of the average firing rate of the neurons in the population.
We present a neural architecture of sentence representation. Words are represented with neural cell assemblies. Relations between words are represented with ‘structure’ assemblies. Word and structure assemblies are bound temporarily to form a sentence representation. We show how multiple sentences can be represented simultaneously, and we simulate how specific information can be retrieved from the architecture. The assemblies are simulated as populations of spiking neurons, in terms of the average firing rate of the neurons in the population.
ES2003-68
Developmental pruning of synapses and category learning
R. Viviani, M. Spitzer
Developmental pruning of synapses and category learning
R. Viviani, M. Spitzer
Abstract:
After an initial peak, the number of synapses in mammalian cerebral cortex decreases in the formative period and throughout adult life. However, if synapses are taken to reflect circuit complexity, the issue arises of how to reconcile pruning with the increasing complexity of the representations acquired in successive stages of development. Taking these two conflicting requirements as an architectural constraint, we show here that a simple topographic self-organization process can learn increasingly complex representations when some of its synapses are progressively pruned. By addressing the learning-theoretic properties of increasing complexity, the model indicates how pruning may be computationally advantageous. This suggests a novel interpretation of the interplay between biological and acquired patterns of neuronal activation determining topographic organization in the cortex.
After an initial peak, the number of synapses in mammalian cerebral cortex decreases in the formative period and throughout adult life. However, if synapses are taken to reflect circuit complexity, the issue arises of how to reconcile pruning with the increasing complexity of the representations acquired in successive stages of development. Taking these two conflicting requirements as an architectural constraint, we show here that a simple topographic self-organization process can learn increasingly complex representations when some of its synapses are progressively pruned. By addressing the learning-theoretic properties of increasing complexity, the model indicates how pruning may be computationally advantageous. This suggests a novel interpretation of the interplay between biological and acquired patterns of neuronal activation determining topographic organization in the cortex.
ES2003-79
An event-driven framework for the simulation of networks of spiking neurons
O. Rochel, D. Martinez
An event-driven framework for the simulation of networks of spiking neurons
O. Rochel, D. Martinez
Abstract:
We propose an event-driven framework dedicated to the design and the simulation of networks of spiking neurons. It consists of an abstract model of spiking neurons and an e.cient event-driven simulation engine so as to achieve good performance in the simulation phase while maintaining a high level of flexibility and programmability in the modelling phase. Our model of neurons encompasses a large class of spiking neurons ranging from usual leaky integrate-and-fire neurons to more abstract neurons, e.g. defined as complex finite state machines. As a result, the proposed framework allows the simulation of large networks that can be composed of unique or different types of neurons.
We propose an event-driven framework dedicated to the design and the simulation of networks of spiking neurons. It consists of an abstract model of spiking neurons and an e.cient event-driven simulation engine so as to achieve good performance in the simulation phase while maintaining a high level of flexibility and programmability in the modelling phase. Our model of neurons encompasses a large class of spiking neurons ranging from usual leaky integrate-and-fire neurons to more abstract neurons, e.g. defined as complex finite state machines. As a result, the proposed framework allows the simulation of large networks that can be composed of unique or different types of neurons.
ES2003-104
Road Singularities Detection and Classification
A.P. Leitão, S. Tilie, M. Mangeas, J.-P. Tarel, V. Vigneron, S. Lelandais
Road Singularities Detection and Classification
A.P. Leitão, S. Tilie, M. Mangeas, J.-P. Tarel, V. Vigneron, S. Lelandais
Abstract:
We propose a detection and classification system for various road situations, which is robust to light changes and di®erent road markings. The road curves in an image are described with a Hough Transform, where histograms accumulate the contrast lines for each pixel. The resulting 2D histograms are used to train a Kohonen Neural Network. The final output classification can be used to improve road security, indicating dangerous situations to the driver or feeding a driving control system.
We propose a detection and classification system for various road situations, which is robust to light changes and di®erent road markings. The road curves in an image are described with a Hough Transform, where histograms accumulate the contrast lines for each pixel. The resulting 2D histograms are used to train a Kohonen Neural Network. The final output classification can be used to improve road security, indicating dangerous situations to the driver or feeding a driving control system.
ES2003-41
Characterization of the absolutely expedient learning algorithms for stochastic automata in a non-discrete space of actions
C. Rivero
Characterization of the absolutely expedient learning algorithms for stochastic automata in a non-discrete space of actions
C. Rivero
Abstract:
This work presents a learning algorithm to reach the optimum action of an arbitrary set of actions contained in Rm. An initial and arbitrary probability measure on IRm allow us to select an action and the probability is sequentially updated by a stochastic automaton using the response of the environment to the selected action. We prove that the corresponding random sequence of probability measures converges in law to a probability measure degenerate on the optimum action, with probability as close to one as we desire.
This work presents a learning algorithm to reach the optimum action of an arbitrary set of actions contained in Rm. An initial and arbitrary probability measure on IRm allow us to select an action and the probability is sequentially updated by a stochastic automaton using the response of the environment to the selected action. We prove that the corresponding random sequence of probability measures converges in law to a probability measure degenerate on the optimum action, with probability as close to one as we desire.
Biologically plausible learning
ES2003-52
A model-based reinforcement learning: a computational model and an fMRI study
W. Yoshida. S. Ishii
A model-based reinforcement learning: a computational model and an fMRI study
W. Yoshida. S. Ishii
ES2003-65
A neural model for heading detection from optic flow
F. Seifart, P. Bayerl, H. Neumann
A neural model for heading detection from optic flow
F. Seifart, P. Bayerl, H. Neumann
Abstract:
This paper describes a neural model developed for computing heading from optic flow caused by 3D translational egomotion. The model uses the distributed representation of optic flow directions in cortical areas MT and MSTd. Model MSTd cells are selective for specific directions of visual motion and have large receptive fields covering approximately a quarter of the visual field at different retinal positions. The estimation of heading is computed in a polar framework by combining the activation of all MSTd cells in a geometrically motivated and biological plausible manner. In our implementation optic flow fields were generated from motion of a simulated camera in a static environment. We analysed the detection error by comparing estimated heading with the ground truth defined by the given camera motion. The results show that the described neural approach provides a robust detection method. We demonstrate that movements inducing radial flow patterns (forward movements) are detected more accurately than motions inducing laminar flow fields (e.g. sideward movements), consistent with psychophysical findings. Most important is that the described properties are a consequence of simple geometrical constraints defined by the spatial arrangement of MSTd cells.
This paper describes a neural model developed for computing heading from optic flow caused by 3D translational egomotion. The model uses the distributed representation of optic flow directions in cortical areas MT and MSTd. Model MSTd cells are selective for specific directions of visual motion and have large receptive fields covering approximately a quarter of the visual field at different retinal positions. The estimation of heading is computed in a polar framework by combining the activation of all MSTd cells in a geometrically motivated and biological plausible manner. In our implementation optic flow fields were generated from motion of a simulated camera in a static environment. We analysed the detection error by comparing estimated heading with the ground truth defined by the given camera motion. The results show that the described neural approach provides a robust detection method. We demonstrate that movements inducing radial flow patterns (forward movements) are detected more accurately than motions inducing laminar flow fields (e.g. sideward movements), consistent with psychophysical findings. Most important is that the described properties are a consequence of simple geometrical constraints defined by the spatial arrangement of MSTd cells.
ES2003-99
Parallel asynchronous distributed computations of optimal control in large state space Markov Decision processes
B. Scherrer
Parallel asynchronous distributed computations of optimal control in large state space Markov Decision processes
B. Scherrer
Abstract:
\N
\N
Mixtures and ensemble learning
ES2003-70
Mixture of Experts and Local-Global Neural Networks
M. S. Fariñas, C. E. Pedreira
Mixture of Experts and Local-Global Neural Networks
M. S. Fariñas, C. E. Pedreira
Abstract:
In this paper we investigate mixture of experts problems in the context of Local-Global Neural Networks. This type of architecture was originaly conceived for functional approximation and interpolation problems. Numerical experiments are presented, showing quite nice solutions. Because of its local characteristics, this type of approach brings the advantage of improving interpretability.
In this paper we investigate mixture of experts problems in the context of Local-Global Neural Networks. This type of architecture was originaly conceived for functional approximation and interpolation problems. Numerical experiments are presented, showing quite nice solutions. Because of its local characteristics, this type of approach brings the advantage of improving interpretability.
ES2003-90
Ensemble of hybrid networks with strong regularization
S. Cohen, N. Intrator
Ensemble of hybrid networks with strong regularization
S. Cohen, N. Intrator
ES2003-92
A new Meta Machine Learning (MML) method based on combining non-significant different neural networks
A. Yanez Escolano, J. Pizarro Junquera, E. Guerrero Vazquez, P.L. Galindo Riano
A new Meta Machine Learning (MML) method based on combining non-significant different neural networks
A. Yanez Escolano, J. Pizarro Junquera, E. Guerrero Vazquez, P.L. Galindo Riano
Abstract:
Model combination provides an alternative to model selection. With a little additional effort we can obtain MML models that improve the generalization capabilities of their individual members. However, it has been recognized that the individual members must be as accurate and diverse as possible. In this paper we present a novel method for building MML models by combining neural networks which are not significantly different from the network selected by some model selection method.
Model combination provides an alternative to model selection. With a little additional effort we can obtain MML models that improve the generalization capabilities of their individual members. However, it has been recognized that the individual members must be as accurate and diverse as possible. In this paper we present a novel method for building MML models by combining neural networks which are not significantly different from the network selected by some model selection method.
Classification
ES2003-5
Detecting Pathologies from Infant Cry Applying Scaled Conjugated Gradient Neural Networks
J. Orozco, C. A. Reyes Garcia
Detecting Pathologies from Infant Cry Applying Scaled Conjugated Gradient Neural Networks
J. Orozco, C. A. Reyes Garcia
Abstract:
This work presents the development of an automatic recognition system of infant cry, with the objective to classify two types of cry: normal and pathological cry from deaf babies. In this study, we used acoustic characteristics obtained by the Linear Prediction technique and as a classifier a neural network that was trained with the scaled conjugate gradient algorithm. Preliminary results are shown, which, up to the moment, are very encouraging.
This work presents the development of an automatic recognition system of infant cry, with the objective to classify two types of cry: normal and pathological cry from deaf babies. In this study, we used acoustic characteristics obtained by the Linear Prediction technique and as a classifier a neural network that was trained with the scaled conjugate gradient algorithm. Preliminary results are shown, which, up to the moment, are very encouraging.
ES2003-85
A Spiking Machine for Human-Computer Interactions (Design methodology)
G. Vaucher
A Spiking Machine for Human-Computer Interactions (Design methodology)
G. Vaucher
Abstract:
The STANNs (Spatio-Temporal Arti.cial Neural Networks) are spiking neural networks. Coming from a bio-inspired data coding, they are adapted to process spatio-temporal patterns. Their capabilities have been studied for several years in the field of the natural HCIs (Human-Computer Interactions): handwritten character recognition, lipreading and speech recognition. Whereas one can observes a renewed interest for this field with the success of mobile telephony and the development of the personal digital assistants, this paper describes how to build a spiking machine with such models with an aim of integrating them in a human-software interface. The approach is illustrated by an industrial application, carried out within the framework of a collaboration between Sup´elec and France Telecom R&D. During this study a detailed attention was given to the evaluation of the ease of integration of this technique in a traditional procedure of software development.
The STANNs (Spatio-Temporal Arti.cial Neural Networks) are spiking neural networks. Coming from a bio-inspired data coding, they are adapted to process spatio-temporal patterns. Their capabilities have been studied for several years in the field of the natural HCIs (Human-Computer Interactions): handwritten character recognition, lipreading and speech recognition. Whereas one can observes a renewed interest for this field with the success of mobile telephony and the development of the personal digital assistants, this paper describes how to build a spiking machine with such models with an aim of integrating them in a human-software interface. The approach is illustrated by an industrial application, carried out within the framework of a collaboration between Sup´elec and France Telecom R&D. During this study a detailed attention was given to the evaluation of the ease of integration of this technique in a traditional procedure of software development.
ES2003-61
A Fuzzy ARTMAP Probability Estimator with Relevance Factor
R. Andonie, L. Sasu
A Fuzzy ARTMAP Probability Estimator with Relevance Factor
R. Andonie, L. Sasu
Abstract:
An incremental, nonparametric probability estimation procedure using a variation of the Fuzzy ARTMAP (FAM) neural network is introduced. The resulted network, called Fuzzy ARTMAP with Relevance factor (FAMR), uses a relevance factor assigned to each sample pair, proportional to the importance of that pair during the learning phase. We prove that our probability estimator is correct. The FAMR can be used both as a classifier and as a probability estimator.
An incremental, nonparametric probability estimation procedure using a variation of the Fuzzy ARTMAP (FAM) neural network is introduced. The resulted network, called Fuzzy ARTMAP with Relevance factor (FAMR), uses a relevance factor assigned to each sample pair, proportional to the importance of that pair during the learning phase. We prove that our probability estimator is correct. The FAMR can be used both as a classifier and as a probability estimator.
Dynamical systems and recurrent networks
ES2003-3
Anticipated synchronization in neuron models
M. Ciszak, O. Calvo, C. Masoller, C. Mirasso, R. Toral
Anticipated synchronization in neuron models
M. Ciszak, O. Calvo, C. Masoller, C. Mirasso, R. Toral
ES2003-57
Autonomous learning algorithm for fully connected recurrent networks
E. Leclercq, F. Druaux, D. Lefebvre
Autonomous learning algorithm for fully connected recurrent networks
E. Leclercq, F. Druaux, D. Lefebvre
Abstract:
In this paper a fully connected RTRL neural network is studied. In order to learn dynamical behaviours of linear-processes or to predict time series, an autonomous learning algorithm has been developed. The originality of this method consists of the gradient based adaptation of the learning rate and time parameter of the neurons using a small perturbations method. Starting from zero initial conditions (neural states, rate of learning, time parameter and matrix of weights) the evolution is completely driven by the dynamic of the learning data. Two examples are proposed, the first one deals with the learning of second order linear process and the second one with the prediction of the chaotic intensity of NH3 laser. This last example illustrates how our network is able to follow high frequencies.
In this paper a fully connected RTRL neural network is studied. In order to learn dynamical behaviours of linear-processes or to predict time series, an autonomous learning algorithm has been developed. The originality of this method consists of the gradient based adaptation of the learning rate and time parameter of the neurons using a small perturbations method. Starting from zero initial conditions (neural states, rate of learning, time parameter and matrix of weights) the evolution is completely driven by the dynamic of the learning data. Two examples are proposed, the first one deals with the learning of second order linear process and the second one with the prediction of the chaotic intensity of NH3 laser. This last example illustrates how our network is able to follow high frequencies.
ES2003-84
Neural Network Algorithms for the p-Median Problem
E. Domínguez Merino, J. Munoz Perez, J. Jerez Aragonés
Neural Network Algorithms for the p-Median Problem
E. Domínguez Merino, J. Munoz Perez, J. Jerez Aragonés
Abstract:
In this paper three recurrent neural network algorithms are proposed for the p-median problem according to different techniques. The competitive recurrent neural network, based on two types of decision variables (location variables and allocation variables), consists of a single layer of 2Np process units (neurons), where N is the number of demand points or customers and p is the number of facilities (medians). The process units form N + p groups, where one neuron per group is active at the same time and neurons in the same group are updated in parallel. Moreover, the energy function (objective function) always decreases as the system evolves according to the dynamical rule proposed. The effectiveness and efficiency of the three algorithms under varying problem sizes are analyzed. The results indicate that the best technique depend on the scale of the problem and the number of medians.
In this paper three recurrent neural network algorithms are proposed for the p-median problem according to different techniques. The competitive recurrent neural network, based on two types of decision variables (location variables and allocation variables), consists of a single layer of 2Np process units (neurons), where N is the number of demand points or customers and p is the number of facilities (medians). The process units form N + p groups, where one neuron per group is active at the same time and neurons in the same group are updated in parallel. Moreover, the energy function (objective function) always decreases as the system evolves according to the dynamical rule proposed. The effectiveness and efficiency of the three algorithms under varying problem sizes are analyzed. The results indicate that the best technique depend on the scale of the problem and the number of medians.
Industrial and agronomical applications of neural networks
ES2003-603
On industrial acceptance of neuro-fuzzy systems
L. M. Reyneri
On industrial acceptance of neuro-fuzzy systems
L. M. Reyneri
ES2003-91
Comparison of traditional and neural systems for train speed estimation
V. Colla, M. Vannucci, B. Allotta, M. Malvezzi
Comparison of traditional and neural systems for train speed estimation
V. Colla, M. Vannucci, B. Allotta, M. Malvezzi
ES2003-115
Neural Networks and M5 model trees in modeling water level-discharge relationship for an Indian river
B. Bhattacharya, D.P. Solomatine
Neural Networks and M5 model trees in modeling water level-discharge relationship for an Indian river
B. Bhattacharya, D.P. Solomatine
Abstract:
In flood management it is important to reliably estimate the discharge in a river. Hydrologists use historic data to establish a rating curve – a relationship between the water level (stage) and discharge. ANN and M5 model trees were used to reconstruct this relationship on an example of an Indian river. The predictive accuracy of these machine learning methods models was found to be superior to a conventional rating curve.
In flood management it is important to reliably estimate the discharge in a river. Hydrologists use historic data to establish a rating curve – a relationship between the water level (stage) and discharge. ANN and M5 model trees were used to reconstruct this relationship on an example of an Indian river. The predictive accuracy of these machine learning methods models was found to be superior to a conventional rating curve.
ES2003-116
Post-failure analysis of an adaptive predictor-corrector neural controller on a flight simulator
M. Battipede, P. Gili, M. Lando, L. Massotti, M.R. Napolitano, G. Campa, M.G. Perhinschi
Post-failure analysis of an adaptive predictor-corrector neural controller on a flight simulator
M. Battipede, P. Gili, M. Lando, L. Massotti, M.R. Napolitano, G. Campa, M.G. Perhinschi
Abstract:
This paper is concerned with the comparison between a classical robust control system and a neural network controller based on the predictorcorrector control scheme featuring different neural network architectures and on-line training algorithms. Both the controllers have been applied to an adaptive flight control system for the F-15 WVU flight simulator and the results are given in terms of performance comparison and control activity evaluation.
This paper is concerned with the comparison between a classical robust control system and a neural network controller based on the predictorcorrector control scheme featuring different neural network architectures and on-line training algorithms. Both the controllers have been applied to an adaptive flight control system for the F-15 WVU flight simulator and the results are given in terms of performance comparison and control activity evaluation.
ANN models and learning III
ES2003-103
An Analysis of Synchrony Conditions for Integrate-and-Fire Neurons
D. Kim
An Analysis of Synchrony Conditions for Integrate-and-Fire Neurons
D. Kim
ES2003-106
Towards the restoration of hand grasp function of quadriplegic patients based on an artificial neural net controller using peripheral nerve stimulation - an approach
M. Bogdan, M. Schröder, W. Rosenstiel
Towards the restoration of hand grasp function of quadriplegic patients based on an artificial neural net controller using peripheral nerve stimulation - an approach
M. Bogdan, M. Schröder, W. Rosenstiel
Abstract:
We propose a closed loop strategy for the control of hand grasp movements for paralyzed patients which is based on an artificial neural network (ANN). For this goal an ANN controller applies functional neuroelectrical stimulation (FNS) to a peripheral nerve with the aim to initiate axonal stimulation patterns similar to those generated by the central nervous system. In this paper we present the results of simulated closed loop position control experiments that were carried out in real time. Training and testing of our control strategy were based on data gained in vivo from a pig’s limb while applying FNS. Despite of muscle fatigue and other nonlinear disturbances our control strategy results in high control quality.
We propose a closed loop strategy for the control of hand grasp movements for paralyzed patients which is based on an artificial neural network (ANN). For this goal an ANN controller applies functional neuroelectrical stimulation (FNS) to a peripheral nerve with the aim to initiate axonal stimulation patterns similar to those generated by the central nervous system. In this paper we present the results of simulated closed loop position control experiments that were carried out in real time. Training and testing of our control strategy were based on data gained in vivo from a pig’s limb while applying FNS. Despite of muscle fatigue and other nonlinear disturbances our control strategy results in high control quality.
ES2003-43
Monitoring technical systems with prototype based clustering
T. Bojer, B. Hammer, C. Koers
Monitoring technical systems with prototype based clustering
T. Bojer, B. Hammer, C. Koers
Abstract:
We present an application of generalized relevance learning vector quantization (GRLVQ) to the supervision of piston compressors in industry. Thereby, GRLVQ constitutes a prototype-based clustering algorithm with adaptive diagonal metric based on LVQ. In the reported application, further adaptation of the distance measure is necessary in order to allow invariance with respect to small time shifts. Depending on the respective sensors, very good classi.cation results are obtained.
We present an application of generalized relevance learning vector quantization (GRLVQ) to the supervision of piston compressors in industry. Thereby, GRLVQ constitutes a prototype-based clustering algorithm with adaptive diagonal metric based on LVQ. In the reported application, further adaptation of the distance measure is necessary in order to allow invariance with respect to small time shifts. Depending on the respective sensors, very good classi.cation results are obtained.
ES2003-53
Evolved Neurodynamics for Robot Control
F. Pasemann, M. Hülse, K. Zahedi
Evolved Neurodynamics for Robot Control
F. Pasemann, M. Hülse, K. Zahedi
Abstract:
Small recurrent neural network with two and three neurons are able to control autonomous robots showing obstacle avoidance and photo-tropic behaviors. They have been generated by evolutionary processes, and they demonstrate, how dynamical properties can be used for an effective behavior control. Presented examples also show how sensor fusion can be obtained by evolution. Additional techniques are used to excavate the relevant neural processing mechanisms underlying specific behavior features.
Small recurrent neural network with two and three neurons are able to control autonomous robots showing obstacle avoidance and photo-tropic behaviors. They have been generated by evolutionary processes, and they demonstrate, how dynamical properties can be used for an effective behavior control. Presented examples also show how sensor fusion can be obtained by evolution. Additional techniques are used to excavate the relevant neural processing mechanisms underlying specific behavior features.
ES2003-54
VLSI Realization of a Two-Dimensional Hamming Distance Comparator ANN for Image Processing Applications
S. Badel, A. Schmid, Y. Leblebici
VLSI Realization of a Two-Dimensional Hamming Distance Comparator ANN for Image Processing Applications
S. Badel, A. Schmid, Y. Leblebici
Abstract:
This paper presents the hardware realization of a Hamming artificial neural network, and demonstrates its use in a high-speed precision alignment system. High degree of parallelism is exploited in the proposed architecture, where the result of NxN array of sum of products is provided simultaneously. The full operation of the artificial neural network requires three clock cycles, which are shown to be completed within a few tens of nanoseconds, depending on the chosen architecture, thus realizing a complex operation using a fast and low-power circuit. Possible applications of the device include industrial image processing such as focus recovery, fast and precise alignment in a noisy environment, and vehicle navigation systems.
This paper presents the hardware realization of a Hamming artificial neural network, and demonstrates its use in a high-speed precision alignment system. High degree of parallelism is exploited in the proposed architecture, where the result of NxN array of sum of products is provided simultaneously. The full operation of the artificial neural network requires three clock cycles, which are shown to be completed within a few tens of nanoseconds, depending on the chosen architecture, thus realizing a complex operation using a fast and low-power circuit. Possible applications of the device include industrial image processing such as focus recovery, fast and precise alignment in a noisy environment, and vehicle navigation systems.
ES2003-55
Recursive Least Squares for an Entropy Regularized MSE Cost Function
D. Erdogmus, Y. N. Rao, J. C. Principe, O. Fontenla-Romero, A. Alonso-Betanzos
Recursive Least Squares for an Entropy Regularized MSE Cost Function
D. Erdogmus, Y. N. Rao, J. C. Principe, O. Fontenla-Romero, A. Alonso-Betanzos
Abstract:
MSE plays an indispensable role in learning and adaptation of neural systems. Nevertheless, the instantaneous value of the modeling error alone does not convey sufficient information about the accuracy of the estimated model in representing the underlying structure of the data. In this paper, we propose an extension to the traditional MSE cost function, a regularization term based on the incremental errors in model output. We demonstrate the stochastic equivalence between the proposed regularization term and the error entropy. Finally, we derive an RLS-type algorithm for the proposed cost function, which we call recursive least squares with entropy regularization (RLSER) algorithm. The performance of RLSER is shown to be better than RLS in supervised training with noisy data.
MSE plays an indispensable role in learning and adaptation of neural systems. Nevertheless, the instantaneous value of the modeling error alone does not convey sufficient information about the accuracy of the estimated model in representing the underlying structure of the data. In this paper, we propose an extension to the traditional MSE cost function, a regularization term based on the incremental errors in model output. We demonstrate the stochastic equivalence between the proposed regularization term and the error entropy. Finally, we derive an RLS-type algorithm for the proposed cost function, which we call recursive least squares with entropy regularization (RLSER) algorithm. The performance of RLSER is shown to be better than RLS in supervised training with noisy data.
ES2003-69
A view-based approach for object recognition from image sequences
A. Zehender, P. Bayerl, H. Neumann
A view-based approach for object recognition from image sequences
A. Zehender, P. Bayerl, H. Neumann
ES2003-72
Extracting Interface Assertions from Neural Networks in Polyhedral Format
S. Breutel, F. Maire, R. Hayward
Extracting Interface Assertions from Neural Networks in Polyhedral Format
S. Breutel, F. Maire, R. Hayward
ES2003-75
The hypersphere neuron
V. Banarer, C. Perwass, G. Sommer
The hypersphere neuron
V. Banarer, C. Perwass, G. Sommer
Abstract:
In this paper a special higher order neuron, the hypersphere neuron, is introduced. By embedding Euclidean space in a conformal space, hyperspheres can be expressed as vectors. The scalar product of points and spheres in conformal space, gives a measure for how far a point lies inside or outside a hypersphere. It will be shown that a hypersphere neuron may be implemented as a perceptron with two bias inputs. By using hyperspheres instead of hyperplanes as decision surfaces, a reduction in computational complexity can be achieved for certain types of problems. Furthermore, in this setup, a reliability measure can be associated with data points in a straight forward way.
In this paper a special higher order neuron, the hypersphere neuron, is introduced. By embedding Euclidean space in a conformal space, hyperspheres can be expressed as vectors. The scalar product of points and spheres in conformal space, gives a measure for how far a point lies inside or outside a hypersphere. It will be shown that a hypersphere neuron may be implemented as a perceptron with two bias inputs. By using hyperspheres instead of hyperplanes as decision surfaces, a reduction in computational complexity can be achieved for certain types of problems. Furthermore, in this setup, a reliability measure can be associated with data points in a straight forward way.
ES2003-113
Fast approximation of the bootstrap for model selection
G. Simon, A. Lendasse, V. Wertz, M. Verleysen
Fast approximation of the bootstrap for model selection
G. Simon, A. Lendasse, V. Wertz, M. Verleysen
Abstract:
The bootstrap resampling method may be efficiently used to estimate the generalization error of a family of nonlinear regression models, as artificial neural networks. The main difficulty associated with the bootstrap in real-world applications is the high computation load. In this paper we propose a simple procedure based on empirical evidence, to considerably reduce the computation time needed to estimate the generalization error of a family of models of increasing number of parameters.
The bootstrap resampling method may be efficiently used to estimate the generalization error of a family of nonlinear regression models, as artificial neural networks. The main difficulty associated with the bootstrap in real-world applications is the high computation load. In this paper we propose a simple procedure based on empirical evidence, to considerably reduce the computation time needed to estimate the generalization error of a family of models of increasing number of parameters.
ES2003-81
Associative morphological memories for spectral unmixing
M. Grana, J. Gallego
Associative morphological memories for spectral unmixing
M. Grana, J. Gallego
Abstract:
Unlimited storage and perfect recall of noiseless real valued patterns has been proved for Autoassociative Morphological Memories (AMM). However AMM's suffer from sensitivity to specific noise models, that can be characterized as erosive and dilative noise. On the other hand, Spectral Unmixing of Hyperespectral Images needs the prior definition of a set of Endmembers, which correspond to material spectra lying on vertices of a convex region covering the image data. These vertices can be characterized as morphologically independent patterns. We present a procedure that takes advantade of the AMM's noise sensitivity to perform Endmember spectra selection based on this characterization.
Unlimited storage and perfect recall of noiseless real valued patterns has been proved for Autoassociative Morphological Memories (AMM). However AMM's suffer from sensitivity to specific noise models, that can be characterized as erosive and dilative noise. On the other hand, Spectral Unmixing of Hyperespectral Images needs the prior definition of a set of Endmembers, which correspond to material spectra lying on vertices of a convex region covering the image data. These vertices can be characterized as morphologically independent patterns. We present a procedure that takes advantade of the AMM's noise sensitivity to perform Endmember spectra selection based on this characterization.
ES2003-93
Adaptive Learning in Changing Environments
M. Rocha, P. Cortez, J. Neves
Adaptive Learning in Changing Environments
M. Rocha, P. Cortez, J. Neves
Abstract:
\N
\N
Digital image processing with neural networks
ES2003-604
Digital Image Processing with Neural Networks
A. Wismüller, U. Seiffert
Digital Image Processing with Neural Networks
A. Wismüller, U. Seiffert
ES2003-94
Semi-automatic acquisition and labelling of image data using SOMs
G. Heidemann, A. Saalbach, H. Ritter
Semi-automatic acquisition and labelling of image data using SOMs
G. Heidemann, A. Saalbach, H. Ritter
Abstract:
Application of neural networks for real world object recognition suffers from the need to acquire large quantities of labelled image data. We propose a solution that acquires images from a domain at random and structures the data in two steps: Data driven mechanisms extract windows of interest, which are clustered by a SOM. Regions of the SOM in which objects form clusters serve as "suggestions" for categories. An interactive visualisation of the SOM combined with distance measures allows the user to determine classes and build training sets. By this means, large labelled data sets for a neural classifier can be easily generated.
Application of neural networks for real world object recognition suffers from the need to acquire large quantities of labelled image data. We propose a solution that acquires images from a domain at random and structures the data in two steps: Data driven mechanisms extract windows of interest, which are clustered by a SOM. Regions of the SOM in which objects form clusters serve as "suggestions" for categories. An interactive visualisation of the SOM combined with distance measures allows the user to determine classes and build training sets. By this means, large labelled data sets for a neural classifier can be easily generated.
ES2003-39
Model-Free Functional MRI Analysis Using Topographic Independent Component Analysis
A. Meyer-Bäse, T. D. Otto, T. Martinetz, D. Auer, A. Wismüller
Model-Free Functional MRI Analysis Using Topographic Independent Component Analysis
A. Meyer-Bäse, T. D. Otto, T. Martinetz, D. Auer, A. Wismüller
ES2003-77
Neural Network Performances in Astronomical Image Processing
R. Cancelliere, M. Gai
Neural Network Performances in Astronomical Image Processing
R. Cancelliere, M. Gai
Abstract:
In this paper we use neural networks to verify the similarity of real astronomical images to predefined reference profiles. We use an innovative technique to encode images that associates each of them with its most convenient moments, evaluated along the {x, y} axes; in this way we obtain a parsimonious but e.ective method with respect to the usual pixel by pixel description.
In this paper we use neural networks to verify the similarity of real astronomical images to predefined reference profiles. We use an innovative technique to encode images that associates each of them with its most convenient moments, evaluated along the {x, y} axes; in this way we obtain a parsimonious but e.ective method with respect to the usual pixel by pixel description.
ES2003-76
A recognition of filaments in solar images with an artificial neural network
V.V. Zharkova, V. Schetinin
A recognition of filaments in solar images with an artificial neural network
V.V. Zharkova, V. Schetinin
Abstract:
A new technique based on the Artificial Neural Network (ANN) was developed for an automated recognition of solar filaments, dark elongated features visible in the hydrogen H-alpha line full disk spectroheliograms. The ANN was trained on a single fragment containing the filament elements depicted on a local background and then tested on the other 54 image fragments depicting filaments on the backgrounds with variations in brightness. Despite the difference in backgrounds, the ANN has properly recognized filaments in all the testing image fragments. This technique can be extended for an automated recognition of solar filaments in the existing solar catalogues.
A new technique based on the Artificial Neural Network (ANN) was developed for an automated recognition of solar filaments, dark elongated features visible in the hydrogen H-alpha line full disk spectroheliograms. The ANN was trained on a single fragment containing the filament elements depicted on a local background and then tested on the other 54 image fragments depicting filaments on the backgrounds with variations in brightness. Despite the difference in backgrounds, the ANN has properly recognized filaments in all the testing image fragments. This technique can be extended for an automated recognition of solar filaments in the existing solar catalogues.
ES2003-108
Locally Linear Embedding versus Isotop
J.A. Lee, C. Archambeau, M. Verleysen
Locally Linear Embedding versus Isotop
J.A. Lee, C. Archambeau, M. Verleysen
Abstract:
Recently, a new method intended to realize conformal mappings has been published. Called Locally Linear Embedding (LLE), this method can map high-dimensional data lying on a manifold to a representation of lower dimensionality that preserves the angles. Although LLE is claimed to solve problems that are usually managed by neural networks like Kohonen’s Self-Organizing Maps (SOMs), the method reduces to an elegant eigenproblem with desirable properties (no parameter tuning, no local minima, etc.). The purpose of this paper consists in comparing the capabilities of LLE with a newly developed neural method called Isotop and based on ideas like neighborhood preservation, which has been the key of the SOMs’ success. To illustrate the di erences between the algebraic and the neural approach, LLE and Isotop are first briefly described and then compared with well known dimensionality reduction problems.
Recently, a new method intended to realize conformal mappings has been published. Called Locally Linear Embedding (LLE), this method can map high-dimensional data lying on a manifold to a representation of lower dimensionality that preserves the angles. Although LLE is claimed to solve problems that are usually managed by neural networks like Kohonen’s Self-Organizing Maps (SOMs), the method reduces to an elegant eigenproblem with desirable properties (no parameter tuning, no local minima, etc.). The purpose of this paper consists in comparing the capabilities of LLE with a newly developed neural method called Isotop and based on ideas like neighborhood preservation, which has been the key of the SOMs’ success. To illustrate the di erences between the algebraic and the neural approach, LLE and Isotop are first briefly described and then compared with well known dimensionality reduction problems.
ES2003-114
Meter value recognition using locally connected hierarchical networks
S. Behnke
Meter value recognition using locally connected hierarchical networks
S. Behnke
Abstract:
This paper describes a two-stage system for the recognition of postage meter values. A feed-forward Neural Abstraction Pyramid is initialized in an unsupervised manner and trained in a supervised fashion to classify an entire digit block. It does not need prior digit segmentation. If the block recognition is not confident enough, a second stage tries to recognize single digits, taking into account the block classifier output for a neighboring digit as context. The system is evaluated on a large database. It can recognize meter values that are hard to read for humans.
This paper describes a two-stage system for the recognition of postage meter values. A feed-forward Neural Abstraction Pyramid is initialized in an unsupervised manner and trained in a supervised fashion to classify an entire digit block. It does not need prior digit segmentation. If the block recognition is not confident enough, a second stage tries to recognize single digits, taking into account the block classifier output for a neighboring digit as context. The system is evaluated on a large database. It can recognize meter values that are hard to read for humans.