Bruges, Belgium, April 28-29-30
Content of the proceedings
-
Learning I
Theory and applications of neural maps
Bayesian learning and Markov processes
Sof-computing techniques for time series forecasting
Learning II
Indepedent compent analysis and non-linear projection
Industrial applications of neural networks
Neural methods for non-standard data
Learning III
Hardware systems for neural devices
Support vector machines
Neural networks for data mining
Learning IV
Learning I
ES2004-7
A New Learning Rates Adaptation Strategy for the Resilient Propagation Algorithm
Aristoklis Anastasiadis, George Magoulas, Michael Vrahatis
A New Learning Rates Adaptation Strategy for the Resilient Propagation Algorithm
Aristoklis Anastasiadis, George Magoulas, Michael Vrahatis
Abstract:
In this paper we propose an Rprop modification that builds on a mathematical framework for the convergence analysis to equip Rprop with a learning rates adaptation strategy that ensures the search direction is a descent one. Our analysis is supported by experiments illustrating how the new learning rates adaptation strategy works in the test cases to ameliorate the convergence behaviour of the Rprop. Empirical results indicate that the new modification provides benefits when compared against the Rprop and a modification proposed recently, the Improved Rprop.
In this paper we propose an Rprop modification that builds on a mathematical framework for the convergence analysis to equip Rprop with a learning rates adaptation strategy that ensures the search direction is a descent one. Our analysis is supported by experiments illustrating how the new learning rates adaptation strategy works in the test cases to ameliorate the convergence behaviour of the Rprop. Empirical results indicate that the new modification provides benefits when compared against the Rprop and a modification proposed recently, the Improved Rprop.
ES2004-63
high-accuracy value-function approximation with neural networks applied to the acrobot
Rémi Coulom
high-accuracy value-function approximation with neural networks applied to the acrobot
Rémi Coulom
Abstract:
Several reinforcement-learning techniques have already been applied to the Acrobot control problem, using linear function approximators to estimate the value function. In this paper, we present experimental results obtained by using a feedforward neural network instead. The learning algorithm used was model-based continuous TD(lambda). It generated an efficient controller, producing a high-accuracy state-value function. A striking feature of this value function is a very sharp 4-dimensional ridge that is extremely hard to evaluate with linear parametric approximators. From a broader point of view, this experimental success demonstrates some of the qualities of feedforward neural networks in comparison with linear approximators in reinforcement learning.
Several reinforcement-learning techniques have already been applied to the Acrobot control problem, using linear function approximators to estimate the value function. In this paper, we present experimental results obtained by using a feedforward neural network instead. The learning algorithm used was model-based continuous TD(lambda). It generated an efficient controller, producing a high-accuracy state-value function. A striking feature of this value function is a very sharp 4-dimensional ridge that is extremely hard to evaluate with linear parametric approximators. From a broader point of view, this experimental success demonstrates some of the qualities of feedforward neural networks in comparison with linear approximators in reinforcement learning.
ES2004-84
Input Space Bifurcation Manifolds of RNNs
Robert Haschke, Jochen J. Steil
Input Space Bifurcation Manifolds of RNNs
Robert Haschke, Jochen J. Steil
Abstract:
We derive analytical expressions of local codim-1-bifurcations for a fully connected, additive, discrete-time RNN, where we regard the external inputs as bifurcation parameters. The complexity of the bifurcation diagrams obtained increases exponentially with the number of neurons. We show that a three-neuron cascaded network can serve as a universal oscillator, whose amplitude and frequency can be completely controlled by input parameters.
We derive analytical expressions of local codim-1-bifurcations for a fully connected, additive, discrete-time RNN, where we regard the external inputs as bifurcation parameters. The complexity of the bifurcation diagrams obtained increases exponentially with the number of neurons. We show that a three-neuron cascaded network can serve as a universal oscillator, whose amplitude and frequency can be completely controlled by input parameters.
ES2004-90
online policy adaptation for ensemble classifiers
Dimitrakakis Christos, Bengio Samy
online policy adaptation for ensemble classifiers
Dimitrakakis Christos, Bengio Samy
Abstract:
Ensemble algorithms can improve the performance of a given learning algorithm through the combination of multiple base classifiers into an ensemble.In this paper, the idea of using an adaptive policy for training and combining the base classifiers is put forward. The effectiveness of this approach for online learning is demonstrated by experimental results on several UCI benchmark databases.
Ensemble algorithms can improve the performance of a given learning algorithm through the combination of multiple base classifiers into an ensemble.In this paper, the idea of using an adaptive policy for training and combining the base classifiers is put forward. The effectiveness of this approach for online learning is demonstrated by experimental results on several UCI benchmark databases.
Theory and applications of neural maps
ES2004-182
Theory and applications of neural maps
Thomas Villmann, Udo Seiffert, Axel Wismüller
Theory and applications of neural maps
Thomas Villmann, Udo Seiffert, Axel Wismüller
Abstract:
In this tutorial paper about neural maps we review the current state in theoretical aspects like mathematical treatment of convergence, ordering and topography, magnification and others. Therby we concentrate on two well-known examples: Self-Organizing Maps and Neural Gas. Moreover we briefly reflect outstanding applications showing the power of neural maps.
In this tutorial paper about neural maps we review the current state in theoretical aspects like mathematical treatment of convergence, ordering and topography, magnification and others. Therby we concentrate on two well-known examples: Self-Organizing Maps and Neural Gas. Moreover we briefly reflect outstanding applications showing the power of neural maps.
ES2004-73
self-organizing context learning
Marc Strickert, Barbara Hammer
self-organizing context learning
Marc Strickert, Barbara Hammer
Abstract:
This work is designed to contribute to a deeper understanding of the recently proposed Merging SOM (MSOM). Its context model aims at the representation of sequences, an important subclass of structured data. In this work, we revisit the model with a focus on its fractal context encoding and the convergence of its recursive dynamic. Experiments with artificial and real world data support our findings and demonstrate the power of the MSOM model.
This work is designed to contribute to a deeper understanding of the recently proposed Merging SOM (MSOM). Its context model aims at the representation of sequences, an important subclass of structured data. In this work, we revisit the model with a focus on its fractal context encoding and the convergence of its recursive dynamic. Experiments with artificial and real world data support our findings and demonstrate the power of the MSOM model.
ES2004-45
Visual person tracking with a Supervised Conditioning-SOM
David Buldain, Elias Herrero
Visual person tracking with a Supervised Conditioning-SOM
David Buldain, Elias Herrero
Abstract:
The classification problem of determining if a surveillance camera sees persons is tackled with two neural models: the Self-Organizing Map (SOM) with supervision as in a classical conditioning analogy and Multi Layer Perceptrons (MLP). The first model, that we call Conditioning-SOM (C-SOM) allowed a quick selection of input features with a good tradeoff between computational cost and classification performance. Finally, MLP classifiers were trained with the selected features. The classification performance of both neural models was very good with very simple features.
The classification problem of determining if a surveillance camera sees persons is tackled with two neural models: the Self-Organizing Map (SOM) with supervision as in a classical conditioning analogy and Multi Layer Perceptrons (MLP). The first model, that we call Conditioning-SOM (C-SOM) allowed a quick selection of input features with a good tradeoff between computational cost and classification performance. Finally, MLP classifiers were trained with the selected features. The classification performance of both neural models was very good with very simple features.
ES2004-22
Forbidden Magnification? I.
Abha Jain, Erzsebet Merenyi
Forbidden Magnification? I.
Abha Jain, Erzsebet Merenyi
Abstract:
This paper presents some interesting results obtained by the algorithm by Bauer, Der and Hermann (BDH) [1] for magnification control in Self-Organizing Maps. "Magnification control" in SOMs refers to the modification of the relationship between the probability density functions of the input samples and their prototypes (SOM weights). The above mentioned algorithm enables explicit control of the magnification properties of a SOM, however, the available theory restricts its validity to 1-D data or 2-D data when the stimulus density separates. This discourages the use of the BDH algorithm for practical applications. In this paper we present results of careful simulations that show the scope of this algorithm when applied to more general, "forbidden" data. We also demonstrate the application of negative magnification to magnify rare classes in the data to enhance their detectability.
This paper presents some interesting results obtained by the algorithm by Bauer, Der and Hermann (BDH) [1] for magnification control in Self-Organizing Maps. "Magnification control" in SOMs refers to the modification of the relationship between the probability density functions of the input samples and their prototypes (SOM weights). The above mentioned algorithm enables explicit control of the magnification properties of a SOM, however, the available theory restricts its validity to 1-D data or 2-D data when the stimulus density separates. This discourages the use of the BDH algorithm for practical applications. In this paper we present results of careful simulations that show the scope of this algorithm when applied to more general, "forbidden" data. We also demonstrate the application of negative magnification to magnify rare classes in the data to enhance their detectability.
ES2004-28
Forbidden magnification? II.
Erzsebet Merenyi, Abha Jain
Forbidden magnification? II.
Erzsebet Merenyi, Abha Jain
Abstract:
The twin of this paper, ``Forbidden Magnification? I." cite{MagnI04}, presents systematic SOM simulations with the explicit magnification control scheme of Bauer, Der, and Herrmann cite{Bauer96a} on data for which the theory does not guarantee success, namely data that are $n$-D, $n > 2$ and/or data whose components in the different dimensions are not statistically independent. For the unsupported n = 2 cases that we investigated the simulations show that even though the magnification exponent $alpha_{achieved}$ achieved by magnification control is not the same as the intended $alpha_{intended}$, the direction and sign of $alpha_{achieved}$ systematically follows $alpha_{intended}$ with a more or less constant offset. We experimentally showed that for simple synthetic higher dimensional data negative magnification has the desired effect of improving the detectability of rare classes. In this paper we study further theoretically unsupported cases, including experiments with real data.
The twin of this paper, ``Forbidden Magnification? I." cite{MagnI04}, presents systematic SOM simulations with the explicit magnification control scheme of Bauer, Der, and Herrmann cite{Bauer96a} on data for which the theory does not guarantee success, namely data that are $n$-D, $n > 2$ and/or data whose components in the different dimensions are not statistically independent. For the unsupported n = 2 cases that we investigated the simulations show that even though the magnification exponent $alpha_{achieved}$ achieved by magnification control is not the same as the intended $alpha_{intended}$, the direction and sign of $alpha_{achieved}$ systematically follows $alpha_{intended}$ with a more or less constant offset. We experimentally showed that for simple synthetic higher dimensional data negative magnification has the desired effect of improving the detectability of rare classes. In this paper we study further theoretically unsupported cases, including experiments with real data.
ES2004-150
Description of the Group Dynamic of Funds’ Managers using Kohonen’s Map
catherine Aaron, Yamina Tadjeddine
Description of the Group Dynamic of Funds’ Managers using Kohonen’s Map
catherine Aaron, Yamina Tadjeddine
Abstract:
Abstract : the aim of this paper is to observe the group dynamic of funds’ managers during an interesting period : January 99- July 01 that include many financial events as financial cracks, high speculation on new technologies… We defined a strategy as the sensitivity on French stock market indexes. We projected strategies on a Kohonen Map. We propose a new approach to analyse the group dynamic by studying moving on this map.
Abstract : the aim of this paper is to observe the group dynamic of funds’ managers during an interesting period : January 99- July 01 that include many financial events as financial cracks, high speculation on new technologies… We defined a strategy as the sensitivity on French stock market indexes. We projected strategies on a Kohonen Map. We propose a new approach to analyse the group dynamic by studying moving on this map.
Bayesian learning and Markov processes
ES2004-13
Robust Bayesian Mixture Modelling
Christopher Bishop, Markus Svensén
Robust Bayesian Mixture Modelling
Christopher Bishop, Markus Svensén
Abstract:
Bayesian approaches to density estimation and clustering using mixture distributions allow the automatic determination of the number of components in the mixture. Previous treatments have focussed on mixtures having Gaussian components, but these are well known to be sensitive to outliers. This can lead to excessive sensitivity to small numbers of data points and consequent overestimates of the number of components. In this paper we develop a Bayesian approach to mixture modelling based on Student-t distributions, which are heavier tailed than Gaussians and hence more robust. By expressing the Student-t distribution as a marginalisation over additional latent variables we are able to derive a tractable variational inference algorithm for this model, which includes Gaussian mixtures as a special case. Results on a variety of real data sets demonstrate the improved robustness of our approach.
Bayesian approaches to density estimation and clustering using mixture distributions allow the automatic determination of the number of components in the mixture. Previous treatments have focussed on mixtures having Gaussian components, but these are well known to be sensitive to outliers. This can lead to excessive sensitivity to small numbers of data points and consequent overestimates of the number of components. In this paper we develop a Bayesian approach to mixture modelling based on Student-t distributions, which are heavier tailed than Gaussians and hence more robust. By expressing the Student-t distribution as a marginalisation over additional latent variables we are able to derive a tractable variational inference algorithm for this model, which includes Gaussian mixtures as a special case. Results on a variety of real data sets demonstrate the improved robustness of our approach.
ES2004-163
Flexible and Robust Bayesian Classification by Finite Mixture Models
Cédric Archambeau, Frédéric Vrins, Michel Verleysen
Flexible and Robust Bayesian Classification by Finite Mixture Models
Cédric Archambeau, Frédéric Vrins, Michel Verleysen
Abstract:
The regularized Mahalanobis distance is proposed in the framework of finite mixture models to avoid commonly faced numerical difficulties when estimating the model parameters by EM. Its principle is applied to Gaussian mixtures and t-Student mixtures, resulting in reliable density estimates, the model complexity being kept low. Besides, the regularized models are robust to various noise types. Finally, the quality of the associated Bayesian classification is near optimal.
The regularized Mahalanobis distance is proposed in the framework of finite mixture models to avoid commonly faced numerical difficulties when estimating the model parameters by EM. Its principle is applied to Gaussian mixtures and t-Student mixtures, resulting in reliable density estimates, the model complexity being kept low. Besides, the regularized models are robust to various noise types. Finally, the quality of the associated Bayesian classification is near optimal.
ES2004-105
protein secondary structure prediction using sigmoid belief networks to parameterize segmental semi-Markov models
Wei Chu, Zoubin Ghahramani, David Wild
protein secondary structure prediction using sigmoid belief networks to parameterize segmental semi-Markov models
Wei Chu, Zoubin Ghahramani, David Wild
Abstract:
In this paper, we merge the parametric structure of neural networks into a segmental semi-Markov model to set up a Bayesian framework for protein structure prediction. The parametric model, which can also be regarded as an extension of a sigmoid belief network, captures the underlying dependency in residue sequences. The results of numerical experiments indicate the usefulness of this approach.
In this paper, we merge the parametric structure of neural networks into a segmental semi-Markov model to set up a Bayesian framework for protein structure prediction. The parametric model, which can also be regarded as an extension of a sigmoid belief network, captures the underlying dependency in residue sequences. The results of numerical experiments indicate the usefulness of this approach.
ES2004-149
Non-linear Analysis of Shocks when Financial Markets are Subject to Changes in Regime
Maillet Bertrand, Olteanu Madalina, Rynkiewicz Joseph
Non-linear Analysis of Shocks when Financial Markets are Subject to Changes in Regime
Maillet Bertrand, Olteanu Madalina, Rynkiewicz Joseph
Abstract:
Violent turbulences are often striking the financial markets and an Index of Market Shocks (IMS) was recently introduced in the attempt of quantifying these turbulences. Regime switching linear models have already been used in modelling the conditional volatility of returns. In this paper we propose a description of the IMS with hybrid models integrating multi-layer perceptrons and hidden Markov chains. After sudying the prediction performance of these models, we focus on the series separation and the index behaviour subject to the hidden states.
Violent turbulences are often striking the financial markets and an Index of Market Shocks (IMS) was recently introduced in the attempt of quantifying these turbulences. Regime switching linear models have already been used in modelling the conditional volatility of returns. In this paper we propose a description of the IMS with hybrid models integrating multi-layer perceptrons and hidden Markov chains. After sudying the prediction performance of these models, we focus on the series separation and the index behaviour subject to the hidden states.
Sof-computing techniques for time series forecasting
ES2004-186
Soft-computing techniques for time series forecasting
Ignacio Rojas-Ruiz, Héctor Pomares
Soft-computing techniques for time series forecasting
Ignacio Rojas-Ruiz, Héctor Pomares
Abstract:
One way to contrast the behaviour of different algorithms in the field of time-series forecasting is to compare the prediction error using a benchmark problem. Another interesting way is to perform a competition. In this paper we shortly discuss the competition organized by EUNITE for an electricity load forecasting. Given the temperature and the electricity load from 1997 to 1998, the competitors are asked to supply the prediction of maximum daily values of electrical loads for January 1999 (31 data values altogether, including some holidays). In total, 56 registered competitors from 21 countries were submitted. A summary of the contribution of the best papers along with some remarks about the use of soft-computing in time-series forecasting and future trends are presented in this paper.
One way to contrast the behaviour of different algorithms in the field of time-series forecasting is to compare the prediction error using a benchmark problem. Another interesting way is to perform a competition. In this paper we shortly discuss the competition organized by EUNITE for an electricity load forecasting. Given the temperature and the electricity load from 1997 to 1998, the competitors are asked to supply the prediction of maximum daily values of electrical loads for January 1999 (31 data values altogether, including some holidays). In total, 56 registered competitors from 21 countries were submitted. A summary of the contribution of the best papers along with some remarks about the use of soft-computing in time-series forecasting and future trends are presented in this paper.
ES2004-151
Disruption Anticipation in Tokamak Reactors: A Two-Factors Fuzzy Time Series Approach
Francesco Carlo Morabito, Mario Versaci
Disruption Anticipation in Tokamak Reactors: A Two-Factors Fuzzy Time Series Approach
Francesco Carlo Morabito, Mario Versaci
Abstract:
Disruption in a Tokamak reactor is a sudden loss of confinement that can cause a damage of the machine walls and support structures. In this paper, we propose the use of the Fuzzy Time Series (FTS) approach for anticipating the onset of disruption in Tokamaks. Two-Factors Fuzzy Time Series models will be shown to be advantageously used for making prediction of the disruption’s onset in Joint European Torus (JET) machine. The use of soft computing technique is suggested by the very nature of the variables involved and by the consideration that a single time series of a physical variable is hardly representative of the whole kind of disruptions experimentally observed.
Disruption in a Tokamak reactor is a sudden loss of confinement that can cause a damage of the machine walls and support structures. In this paper, we propose the use of the Fuzzy Time Series (FTS) approach for anticipating the onset of disruption in Tokamaks. Two-Factors Fuzzy Time Series models will be shown to be advantageously used for making prediction of the disruption’s onset in Joint European Torus (JET) machine. The use of soft computing technique is suggested by the very nature of the variables involved and by the consideration that a single time series of a physical variable is hardly representative of the whole kind of disruptions experimentally observed.
ES2004-8
Time Series Analysis for Quality Improvement: a Soft Computing Approach
Kai Xu, S.H. Ng, S.L. Ho
Time Series Analysis for Quality Improvement: a Soft Computing Approach
Kai Xu, S.H. Ng, S.L. Ho
Abstract:
Quality improvement provides organizations with significant opportunities to reduce costs, increase sales, provide on time deliveries and foster better customer relationships. The design and manufacturing as well as services are among the critical processes for continuous quality improvement. Time series data collected from these processes will be the useful source. While there are various techniques to explore these processes, Neural networks (NN) approach is deemed as a promising alternative. However, as NN is a relatively new approach in quality engineering which is traditionally dominated by statistical analysis, there is still much doubt in its effectiveness compared with statistical modeling. The main focus here then is to construct a statistically reliable neural network model with an appropriate architecture to conduct the time series analysis. The purpose of this paper is thus two-fold. Firstly we develop the statistical interval analysis for various types of neural network models which provide a statistical guide towards a reliable modeling architecture. Secondly, we apply the developed approach for quality improvement in various industries.
Quality improvement provides organizations with significant opportunities to reduce costs, increase sales, provide on time deliveries and foster better customer relationships. The design and manufacturing as well as services are among the critical processes for continuous quality improvement. Time series data collected from these processes will be the useful source. While there are various techniques to explore these processes, Neural networks (NN) approach is deemed as a promising alternative. However, as NN is a relatively new approach in quality engineering which is traditionally dominated by statistical analysis, there is still much doubt in its effectiveness compared with statistical modeling. The main focus here then is to construct a statistically reliable neural network model with an appropriate architecture to conduct the time series analysis. The purpose of this paper is thus two-fold. Firstly we develop the statistical interval analysis for various types of neural network models which provide a statistical guide towards a reliable modeling architecture. Secondly, we apply the developed approach for quality improvement in various industries.
ES2004-47
Dynamic functional-link neural networks genetically evolved applied to system identification
Teodor Marcu, Birgit Koeppen-Seliger
Dynamic functional-link neural networks genetically evolved applied to system identification
Teodor Marcu, Birgit Koeppen-Seliger
Abstract:
The contribution concerns the design of a generalised functional-link neural network with internal dynamics and its applicability to system identification by means of multi-input single output non-linear models of auto-regressive with exogenous inputs’ type. An evolutionary search of genetic type and multi-objective optimisation in the Pareto-sense is used to determine the optimal architecture of that dynamic network. The minimised objectives characterise the accuracy of the network and its complexity. Two case studies are included, referring to the identification of an evaporator from a sugar factory, and of a hydraulic looper from a hot rolling mill plant.
The contribution concerns the design of a generalised functional-link neural network with internal dynamics and its applicability to system identification by means of multi-input single output non-linear models of auto-regressive with exogenous inputs’ type. An evolutionary search of genetic type and multi-objective optimisation in the Pareto-sense is used to determine the optimal architecture of that dynamic network. The minimised objectives characterise the accuracy of the network and its complexity. Two case studies are included, referring to the identification of an evaporator from a sugar factory, and of a hydraulic looper from a hot rolling mill plant.
ES2004-86
ON-LINE SUPPORT VECTOR MACHINES AND OPTIMIZATION STRATEGIES
Juan Manuel Gorriz, Carlos G. Puntonet, MOISES SALMERON, JULIO ORTEGA
ON-LINE SUPPORT VECTOR MACHINES AND OPTIMIZATION STRATEGIES
Juan Manuel Gorriz, Carlos G. Puntonet, MOISES SALMERON, JULIO ORTEGA
Abstract:
In this paper we show a new on-line parametric model for time series forecasting based on Vapnik-Chervonenkis (VC) theory. Using the strong connection between support vector machines (SVM) and Regularization Theory (RT), we propose a regularization operator in order to obtain a suitable expression of Radial Basis Functions (RBFs) with the corresponding expressions for updating neural parameters. This operator seeks for the "flattest" function in a feature space, minimizing the risk functional. Finally we mention some modifications and exgtensions that can be applied to control neural resources and select relevant input space variables in order to avoid high computational effort (batch learning).
In this paper we show a new on-line parametric model for time series forecasting based on Vapnik-Chervonenkis (VC) theory. Using the strong connection between support vector machines (SVM) and Regularization Theory (RT), we propose a regularization operator in order to obtain a suitable expression of Radial Basis Functions (RBFs) with the corresponding expressions for updating neural parameters. This operator seeks for the "flattest" function in a feature space, minimizing the risk functional. Finally we mention some modifications and exgtensions that can be applied to control neural resources and select relevant input space variables in order to avoid high computational effort (batch learning).
ES2004-134
MultiGrid-Based Fuzzy Systems for Time Series: Forecasting: Overcoming the curse of dimensionality
Luis Javier Herrera, Ignacio Rojas-Ruiz, Héctor Pomares, Olga Valenzuela, Jesús González, Mohammed Awad
MultiGrid-Based Fuzzy Systems for Time Series: Forecasting: Overcoming the curse of dimensionality
Luis Javier Herrera, Ignacio Rojas-Ruiz, Héctor Pomares, Olga Valenzuela, Jesús González, Mohammed Awad
Abstract:
This work introduces a modified Grid Based Fuzzy System architecture, which is especially suited for the problem of time series prediction. This new architecture overcomes the problem inherent to all grid-based fuzzy systems when dealing with high dimensional input data. This new architecture together with the proposed algorithm allows the possibility of incorporating a higher number of input variables, keeping low both the computational complexity of the algorithm and the complexity of the architecture.
This work introduces a modified Grid Based Fuzzy System architecture, which is especially suited for the problem of time series prediction. This new architecture overcomes the problem inherent to all grid-based fuzzy systems when dealing with high dimensional input data. This new architecture together with the proposed algorithm allows the possibility of incorporating a higher number of input variables, keeping low both the computational complexity of the algorithm and the complexity of the architecture.
Learning II
ES2004-17
Sparse Bayesian kernel logistic regression
Gavin Cawley, Nicola Talbot
Sparse Bayesian kernel logistic regression
Gavin Cawley, Nicola Talbot
Abstract:
In this paper we present a simple hierarchical Bayesian treatment of the sparse kernel logistic regression (KLR) model based MacKay’s evidence approximation. The model is re-parameterised such that an isotropic Gaussian prior over parameters in the kernel induced feature space is replaced by an isotropic Gaussian prior over the transformed parameters, facilitating a Bayesian analysis using standard methods. The Bayesian approach allows the selection of “good” values for the usual regularisation and kernel parameters through maximisation of the marginal likelihood. Results obtained on a variety of benchmark datasets are provided indicating that the Bayesian kernel logistic regression model is competitive, whilst having one less parameter to determine during model selection.
In this paper we present a simple hierarchical Bayesian treatment of the sparse kernel logistic regression (KLR) model based MacKay’s evidence approximation. The model is re-parameterised such that an isotropic Gaussian prior over parameters in the kernel induced feature space is replaced by an isotropic Gaussian prior over the transformed parameters, facilitating a Bayesian analysis using standard methods. The Bayesian approach allows the selection of “good” values for the usual regularisation and kernel parameters through maximisation of the marginal likelihood. Results obtained on a variety of benchmark datasets are provided indicating that the Bayesian kernel logistic regression model is competitive, whilst having one less parameter to determine during model selection.
ES2004-25
Evolutionary Optimization of Neural Networks for Face Detection
Stefan Wiegand, Christian Igel, Uwe Handmann
Evolutionary Optimization of Neural Networks for Face Detection
Stefan Wiegand, Christian Igel, Uwe Handmann
Abstract:
For face recognition from video streams speed and accuracy are vital aspects. The first decision whether a preprocessed image region represents a human face or not is often made by a neural network, e.g., in the Viisage-FaceFINDER(R) video surveillance system. We describe the optimization of such a network by a hybrid algorithm combining evolutionary computation and gradient-based learning. The evolved solutions perform considerably faster than an expert-designed architecture without loss of accuracy.
For face recognition from video streams speed and accuracy are vital aspects. The first decision whether a preprocessed image region represents a human face or not is often made by a neural network, e.g., in the Viisage-FaceFINDER(R) video surveillance system. We describe the optimization of such a network by a hybrid algorithm combining evolutionary computation and gradient-based learning. The evolved solutions perform considerably faster than an expert-designed architecture without loss of accuracy.
ES2004-27
Speaker verification by means of ANNs
Urs Niesen, Beat Pfister
Speaker verification by means of ANNs
Urs Niesen, Beat Pfister
Abstract:
In text-dependent speaker verification the speech signals have to be time-aligned. For that purpose dynamic time warping (DTW) can be used which performs the alignment by minimizing the Euclidean cepstral distance between the test and the reference utterance. While the cumulative Euclidean cepstral distance, which can be gathered from the DTW algorithm, could be used directly to discriminate between a pair of signals spoken by the same and by two different speakers, we show that a distance measure learned by an artificial neural network performs significantly better for the same task.
In text-dependent speaker verification the speech signals have to be time-aligned. For that purpose dynamic time warping (DTW) can be used which performs the alignment by minimizing the Euclidean cepstral distance between the test and the reference utterance. While the cumulative Euclidean cepstral distance, which can be gathered from the DTW algorithm, could be used directly to discriminate between a pair of signals spoken by the same and by two different speakers, we show that a distance measure learned by an artificial neural network performs significantly better for the same task.
ES2004-41
a chaotic basis for neural coding
Nigel Crook
a chaotic basis for neural coding
Nigel Crook
Abstract:
Recent neurobiological data has demonstrated that some neurons communicate with each other via the timing of individual spikes. The possibility of a neural code based on time-structured spike trains is a departure from established theories based on rate coding. Precisely how these time-structured spike trains communicate information is still open for debate. In this paper we consider the possibility that these spike trains communicate discrete internal neuronal states that are generated from the stabilised orbits of a chaotic attractor.
Recent neurobiological data has demonstrated that some neurons communicate with each other via the timing of individual spikes. The possibility of a neural code based on time-structured spike trains is a departure from established theories based on rate coding. Precisely how these time-structured spike trains communicate information is still open for debate. In this paper we consider the possibility that these spike trains communicate discrete internal neuronal states that are generated from the stabilised orbits of a chaotic attractor.
ES2004-48
a biologically plausible neuromorphic system for object recognition and depth analysis
Zhijun Yang, Alan Murray
a biologically plausible neuromorphic system for object recognition and depth analysis
Zhijun Yang, Alan Murray
Abstract:
We present a large-scale neuromorphic model based on integrate-and-fire (IF) neurons that analyses objects and their depth within a moving visual scene. A feature-based algorithm builds a luminosity receptor field as an artificial retina, in which the IF neurons act both as photoreceptors and processing units. We show that the IF neurons can trace an object's path and depth using an adaptive time-window and Temporally Asymmetric Hebbian (TAH) training.
We present a large-scale neuromorphic model based on integrate-and-fire (IF) neurons that analyses objects and their depth within a moving visual scene. A feature-based algorithm builds a luminosity receptor field as an artificial retina, in which the IF neurons act both as photoreceptors and processing units. We show that the IF neurons can trace an object's path and depth using an adaptive time-window and Temporally Asymmetric Hebbian (TAH) training.
ES2004-50
Regularizing generalization error estimators: a novel approach to robust model selection
Masashi Sugiyama, Motoaki Kawanabe, Klaus-Robert Mueller
Regularizing generalization error estimators: a novel approach to robust model selection
Masashi Sugiyama, Motoaki Kawanabe, Klaus-Robert Mueller
Abstract:
A well-known result by Stein shows that regularized estimators with small bias often yield better estimates than unbiased estimators. In this paper, we adapt this spirit to model selection, and propose regularizing unbiased generalization error estimators for stabilization. We trade a small bias in a model selection criterion against a larger variance reduction which has the beneficial effect of being more precise on a single training set.
A well-known result by Stein shows that regularized estimators with small bias often yield better estimates than unbiased estimators. In this paper, we adapt this spirit to model selection, and propose regularizing unbiased generalization error estimators for stabilization. We trade a small bias in a model selection criterion against a larger variance reduction which has the beneficial effect of being more precise on a single training set.
ES2004-60
Dimensionality reduction and classification using the distribution mapping exponent
Marcel Jirina
Dimensionality reduction and classification using the distribution mapping exponent
Marcel Jirina
Abstract:
Probability distribution mapping function which maps multivariate data distribution to the function of one variable is introduced. Distribution-mapping exponent (DME) is something like effective dimensionality of multidimensional space. The method for classification of multivariate data is based on the local estimate of distribution mapping exponent for each point. Distances of all points of a given class of the training set from a given (unknown) point are searched and it is shown that the sum of reciprocals of the DME-th power of these distances can be used as a probability density estimate. The classification quality was tested and compared with other methods using multivariate data from UCI Machine Learning Repository. The method has no tuning parameters.
Probability distribution mapping function which maps multivariate data distribution to the function of one variable is introduced. Distribution-mapping exponent (DME) is something like effective dimensionality of multidimensional space. The method for classification of multivariate data is based on the local estimate of distribution mapping exponent for each point. Distances of all points of a given class of the training set from a given (unknown) point are searched and it is shown that the sum of reciprocals of the DME-th power of these distances can be used as a probability density estimate. The classification quality was tested and compared with other methods using multivariate data from UCI Machine Learning Repository. The method has no tuning parameters.
ES2004-62
A Modular Framework for Multi category feature selection in Digital mammography
Ranadhir Ghosh, Moumita Ghosh, John Yearwood
A Modular Framework for Multi category feature selection in Digital mammography
Ranadhir Ghosh, Moumita Ghosh, John Yearwood
Abstract:
Many existing researches utilized many different approaches for recognition in digital mammography using various ANN classifier-modeling techniques. Different types of feature extraction techniques are also used. It has been observed that, beyond a certain point, the inclusion of additional features leads to a worse rather than better performance. Moreover, the choice of features to represent the patterns affects several aspects of pattern recognition problem such as accuracy, required learning time and necessary number of samples. A common problem with the multi category feature classification is the conflict between the categories. None of the feasible solutions allow simultaneous optimal solution for all categories. In order to find an optimal solutions the searching space can be divided based on individual category in each sub region and finally merging them through decision spport system. In this paper we propose a canonical GA based modular feature selection approach combined with standard MLP.
Many existing researches utilized many different approaches for recognition in digital mammography using various ANN classifier-modeling techniques. Different types of feature extraction techniques are also used. It has been observed that, beyond a certain point, the inclusion of additional features leads to a worse rather than better performance. Moreover, the choice of features to represent the patterns affects several aspects of pattern recognition problem such as accuracy, required learning time and necessary number of samples. A common problem with the multi category feature classification is the conflict between the categories. None of the feasible solutions allow simultaneous optimal solution for all categories. In order to find an optimal solutions the searching space can be divided based on individual category in each sub region and finally merging them through decision spport system. In this paper we propose a canonical GA based modular feature selection approach combined with standard MLP.
ES2004-65
Non-Euclidean norms and data normalisation
Kevin Doherty, Rod Adams, Neil Davey, Neil Davey
Non-Euclidean norms and data normalisation
Kevin Doherty, Rod Adams, Neil Davey, Neil Davey
Abstract:
In this paper, we empirically examine the use of a range of Minkowski norms for the clustering of real world data. We also investigate whether normalisation of the data prior to clustering affects the quality of the result. In a nearest neighbour search on raw real world data sets, fractional norms outperform the Euclidean and higher-order norms. However, when the data are normalised, the results of the nearest neighbour search with the fractional norms are very similar to the results obtained with the Euclidean norm. We show with the classic statistical technique, K-means clustering, and with the Neural Gas artificial neural network that on raw real world data the use of a fractional norm does not improve the recovery of cluster structure. However, the normalisation of the data results in improved recovery accuracy and minimises the effect of the differing norms.
In this paper, we empirically examine the use of a range of Minkowski norms for the clustering of real world data. We also investigate whether normalisation of the data prior to clustering affects the quality of the result. In a nearest neighbour search on raw real world data sets, fractional norms outperform the Euclidean and higher-order norms. However, when the data are normalised, the results of the nearest neighbour search with the fractional norms are very similar to the results obtained with the Euclidean norm. We show with the classic statistical technique, K-means clustering, and with the Neural Gas artificial neural network that on raw real world data the use of a fractional norm does not improve the recovery of cluster structure. However, the normalisation of the data results in improved recovery accuracy and minimises the effect of the differing norms.
ES2004-68
On fields of nonlinear regression models
Bruno Pelletier, Robert Frouin
On fields of nonlinear regression models
Bruno Pelletier, Robert Frouin
Abstract:
In the context of nonlinear regression, we consider the problem of explaining a variable $y$ from a vector $mathbf{x}$ of explanatory variables and from a vector $mathbf{t}$ of conditionning variables, that influences the link function between $y$ and $mathbf{x}$. A neural based solution is proposed in the form of a field of nonlinear regression models, by which it is meant that the relation between those variables is modeled by a map from some space to a function space. This approach results in a broader class of neural models than that of perceptrons, which therefore inherits the interesting approximation theoretical properties of the latter. The interest of such a modeling is illustrated by a real-world geophysical application, namely ocean color remote sensing.
In the context of nonlinear regression, we consider the problem of explaining a variable $y$ from a vector $mathbf{x}$ of explanatory variables and from a vector $mathbf{t}$ of conditionning variables, that influences the link function between $y$ and $mathbf{x}$. A neural based solution is proposed in the form of a field of nonlinear regression models, by which it is meant that the relation between those variables is modeled by a map from some space to a function space. This approach results in a broader class of neural models than that of perceptrons, which therefore inherits the interesting approximation theoretical properties of the latter. The interest of such a modeling is illustrated by a real-world geophysical application, namely ocean color remote sensing.
ES2004-79
a sliding mode controller using neural networks for robot manipulator
Hajoon Lee, Dongkyung Nam, Cheol Hoon Park
a sliding mode controller using neural networks for robot manipulator
Hajoon Lee, Dongkyung Nam, Cheol Hoon Park
Abstract:
This paper proposes a new sliding mode controller using neural networks. Multilayer neural networks with the error back-propagation learning algorithm are used to compensate for the system uncertainty in order to reduce tracking errors and control torques. The stability of the proposed control scheme is proved with the Lyapunov function method. Computer simulation shows that the proposed neuro-controller yields better control performance than the conventional sliding mode controller in the view of tracking errors and overall control torque.
This paper proposes a new sliding mode controller using neural networks. Multilayer neural networks with the error back-propagation learning algorithm are used to compensate for the system uncertainty in order to reduce tracking errors and control torques. The stability of the proposed control scheme is proved with the Lyapunov function method. Computer simulation shows that the proposed neuro-controller yields better control performance than the conventional sliding mode controller in the view of tracking errors and overall control torque.
ES2004-87
HMM and IOHMM modeling of EEG rhythms for asynchronous BCI systems
Silvia Chiappa, Nicolas Donckers, Bengio Samy, Frédéric Vrins
HMM and IOHMM modeling of EEG rhythms for asynchronous BCI systems
Silvia Chiappa, Nicolas Donckers, Bengio Samy, Frédéric Vrins
Abstract:
We compare the use of two Markovian models, HMMs and IOHMMs, to discriminate between three mental tasks for brain computer interface systems using an asynchronous protocol. We show that the discriminant properties of IOHMMs give superior classification performance but that, probably due to the lack of prior knowledge in the design of an appropriate topology, none of these models are able to use temporal information adequately.
We compare the use of two Markovian models, HMMs and IOHMMs, to discriminate between three mental tasks for brain computer interface systems using an asynchronous protocol. We show that the discriminant properties of IOHMMs give superior classification performance but that, probably due to the lack of prior knowledge in the design of an appropriate topology, none of these models are able to use temporal information adequately.
Indepedent compent analysis and non-linear projection
ES2004-108
Linearization identification and an application to BSS using a SOM
Fabian J. Theis, Elmar Wolfgang Lang
Linearization identification and an application to BSS using a SOM
Fabian J. Theis, Elmar Wolfgang Lang
Abstract:
The one-dimensional functional equation g(y(t))=cg(z(t)) with known functions y and z and constant c is considered. The indeterminacies are calculated, and an algorithm for approximating g given y and z at finitely many time instants is proposed. This linearization identification algorithm is applied to the postnonlinear blind source separation (BSS) problem in the case of independent sources with bounded densities. A self-organizing map (SOM) is used to approximate the boundary, and the postnonlinearity estimation in this multivariate case is reduced to the one-dimensional functional equation from above.
The one-dimensional functional equation g(y(t))=cg(z(t)) with known functions y and z and constant c is considered. The indeterminacies are calculated, and an algorithm for approximating g given y and z at finitely many time instants is proposed. This linearization identification algorithm is applied to the postnonlinear blind source separation (BSS) problem in the case of independent sources with bounded densities. A self-organizing map (SOM) is used to approximate the boundary, and the postnonlinearity estimation in this multivariate case is reduced to the one-dimensional functional equation from above.
ES2004-129
Towards a Local Separation Performances Estimator Using Common ICA Contrast Functions ?
Frédéric Vrins, Cédric Archambeau, Michel Verleysen
Towards a Local Separation Performances Estimator Using Common ICA Contrast Functions ?
Frédéric Vrins, Cédric Archambeau, Michel Verleysen
Abstract:
In most ICA algorithms, the separation performances are estimated through the evaluation of a <i>contrast function</i> Phi, used in the update rule of elements of the unmixing matrix. In particular situations, optimizing Phi does not lead to optimize the extraction of each source, one by one. However, in some applications, one can be interested in quantifying the extraction performances of a specific signal. In this paper, we emphasize that none of the usual Phi's could be directly applied, without further precautions, to evaluate the performances of the local separation.
In most ICA algorithms, the separation performances are estimated through the evaluation of a <i>contrast function</i> Phi, used in the update rule of elements of the unmixing matrix. In particular situations, optimizing Phi does not lead to optimize the extraction of each source, one by one. However, in some applications, one can be interested in quantifying the extraction performances of a specific signal. In this paper, we emphasize that none of the usual Phi's could be directly applied, without further precautions, to evaluate the performances of the local separation.
ES2004-118
Separability of analytic postnonlinear blind source separation with bounded sources
Fabian J. Theis, Peter Gruber
Separability of analytic postnonlinear blind source separation with bounded sources
Fabian J. Theis, Peter Gruber
Abstract:
The aim of blind source separation (BSS) is to transform a mixed random vector such that the original sources are recovered. If the sources are assumed to be statistically independent, independent component analysis (ICA) can be applied to perform BSS. An important aspect of successfully analysing data with BSS is to know the indeterminacies of the problem, that is how the separating model is related to the original mixing model. In the case of linear ICA-based BSS it is well known that the mixing matrix can be found except for permutation and scaling, but for more general settings not many results exist. In this work we only consider random variables with bounded densities. We will shortly describe the bounded BSS problem for linear mixtures. Then, based on, we generalize these ideas to the postnonlinear mixing model with analytic nonlinearities and calculate its indeterminacies.
The aim of blind source separation (BSS) is to transform a mixed random vector such that the original sources are recovered. If the sources are assumed to be statistically independent, independent component analysis (ICA) can be applied to perform BSS. An important aspect of successfully analysing data with BSS is to know the indeterminacies of the problem, that is how the separating model is related to the original mixing model. In the case of linear ICA-based BSS it is well known that the mixing matrix can be found except for permutation and scaling, but for more general settings not many results exist. In this work we only consider random variables with bounded densities. We will shortly describe the bounded BSS problem for linear mixtures. Then, based on, we generalize these ideas to the postnonlinear mixing model with analytic nonlinearities and calculate its indeterminacies.
ES2004-161
How to project `circular' manifolds using geodesic distances?
John A. Lee, Michel Verleysen
How to project `circular' manifolds using geodesic distances?
John A. Lee, Michel Verleysen
Abstract:
Recent papers have clearly shown the advantage of using the geodesic distance instead of the Euclidean one in methods performing non-linear dimensionality reduction by means of distance preservation. This new metric greatly improves the performances of existing algorithms, especially when strongly crumpled manifolds have to be unfolded. Nevertheless, neither the Euclidean nor the geodesic distance address the issue of `circular' manifolds like a cylinder or a torus. Such manifolds should ideally be torn before to be unfolded. This paper describes how this can be done in practice when using the geodesic distance.
Recent papers have clearly shown the advantage of using the geodesic distance instead of the Euclidean one in methods performing non-linear dimensionality reduction by means of distance preservation. This new metric greatly improves the performances of existing algorithms, especially when strongly crumpled manifolds have to be unfolded. Nevertheless, neither the Euclidean nor the geodesic distance address the issue of `circular' manifolds like a cylinder or a torus. Such manifolds should ideally be torn before to be unfolded. This paper describes how this can be done in practice when using the geodesic distance.
ES2004-91
Representing hierarchical relationships using a modified Asymmetric SOM Algorithm
Manuel Martin-Merino, Alberto Muñoz
Representing hierarchical relationships using a modified Asymmetric SOM Algorithm
Manuel Martin-Merino, Alberto Muñoz
Abstract:
Self organizing maps (SOM) are useful visualization techniques that have been successfully applied to the analysis of multivariate data. However most of the algorithms proposed in the literature are not able to handle asymmetric similarities. Therefore they are not able to deal with hierarchical relationships in an appropriate manner although this feature is crucial for many practical applications. In this paper we propose a new asymmetric SOM batch algorithm (ASOM) suitable to derive hierarchical relationships from an asymmetric similarity measure. Object relations are shown by a trapezoidal grid of neurons. The first coordinate represents the object proximities while the second one visualizes the object's degree of generality. A kernel version is also proposed that improves the network organization by transforming nonlinearly the original dissimilarity. The new models have been applied to the challenging problem of thesaurus generation with remarkable results.
Self organizing maps (SOM) are useful visualization techniques that have been successfully applied to the analysis of multivariate data. However most of the algorithms proposed in the literature are not able to handle asymmetric similarities. Therefore they are not able to deal with hierarchical relationships in an appropriate manner although this feature is crucial for many practical applications. In this paper we propose a new asymmetric SOM batch algorithm (ASOM) suitable to derive hierarchical relationships from an asymmetric similarity measure. Object relations are shown by a trapezoidal grid of neurons. The first coordinate represents the object proximities while the second one visualizes the object's degree of generality. A kernel version is also proposed that improves the network organization by transforming nonlinearly the original dissimilarity. The new models have been applied to the challenging problem of thesaurus generation with remarkable results.
Industrial applications of neural networks
ES2004-185
Classification and Prediction in Highly Dimensional Spaces - An Application to Tribology
Leonardo Reyneri
Classification and Prediction in Highly Dimensional Spaces - An Application to Tribology
Leonardo Reyneri
Abstract:
This paper presents a few applications of neuro-fuzzy systems to tribology. Classification and prediction act on a highly dimensional input space, posing severe problems of generalization capability and reliability of results. The paper shows how the major problems have been solved for the specific application domain.
This paper presents a few applications of neuro-fuzzy systems to tribology. Classification and prediction act on a highly dimensional input space, posing severe problems of generalization capability and reliability of results. The paper shows how the major problems have been solved for the specific application domain.
ES2004-100
Shear strength prediction using dimensional analysis and functional networks
Amparo Alonso-Betanzos, Enrique Castillo, Oscar Fontenla-Romero, Noelia Sánchez-Maroño
Shear strength prediction using dimensional analysis and functional networks
Amparo Alonso-Betanzos, Enrique Castillo, Oscar Fontenla-Romero, Noelia Sánchez-Maroño
Abstract:
This paper presents a three steps methodology for predicting the failure shear effort in concrete beams. In the first step, dimensional analysis is applied to obtain several sets of dimensionless variables; in the second step, functional and neural networks are used to estimate a relation between those variables and, in the last step, the failure shear effort is recovered from the relations learnt. Finally, the performance of the methodology was validated using data from shear strength experiments.
This paper presents a three steps methodology for predicting the failure shear effort in concrete beams. In the first step, dimensional analysis is applied to obtain several sets of dimensionless variables; in the second step, functional and neural networks are used to estimate a relation between those variables and, in the last step, the failure shear effort is recovered from the relations learnt. Finally, the performance of the methodology was validated using data from shear strength experiments.
ES2004-106
Enhanced unsupervised segmentation of multispectral Magnetic Resonance images
Lia Morra, Silvia Delsanto, Leonardo Reyneri
Enhanced unsupervised segmentation of multispectral Magnetic Resonance images
Lia Morra, Silvia Delsanto, Leonardo Reyneri
Abstract:
Image segmentation is an established necessity for an improved analysis of Magnetic Resonance images. Neural network-based clustering has been shown in literature to yield good results, yet the possibility of transforming the input feature space in order to enhance the clustering process has gone largely unexplored. In this paper we focus on brain imaging and present a new algorithm for unsupervised segmentation of multi-spectral images, based on the research, through neuro-fuzzy techniques, of an optimized space in which to perform clustering. Tests performed on both real and simulated MR images show promising results, encouraging the application to different medical targets and further investigation.
Image segmentation is an established necessity for an improved analysis of Magnetic Resonance images. Neural network-based clustering has been shown in literature to yield good results, yet the possibility of transforming the input feature space in order to enhance the clustering process has gone largely unexplored. In this paper we focus on brain imaging and present a new algorithm for unsupervised segmentation of multi-spectral images, based on the research, through neuro-fuzzy techniques, of an optimized space in which to perform clustering. Tests performed on both real and simulated MR images show promising results, encouraging the application to different medical targets and further investigation.
ES2004-153
comparison of different classification methods on castabilty data coming from steelmaking practice
Marco Vannucci, Valentina Colla
comparison of different classification methods on castabilty data coming from steelmaking practice
Marco Vannucci, Valentina Colla
Abstract:
The problem of the prediction of a critical situation during continuous casting in common steelmaking practice is faced through different traditional soft--computing techniques: the task is to divide the data in two classes corresponding to good and bad casting behaviour respectively. Moreover, a novel algorithm, the Model--based method, is presented. The performance obtained by the different techniques on the real data coming from an important stellmaking industry are compared.
The problem of the prediction of a critical situation during continuous casting in common steelmaking practice is faced through different traditional soft--computing techniques: the task is to divide the data in two classes corresponding to good and bad casting behaviour respectively. Moreover, a novel algorithm, the Model--based method, is presented. The performance obtained by the different techniques on the real data coming from an important stellmaking industry are compared.
ES2004-159
Neural network-based calibration of positron emission tomograph detector modules
Beatrice Lazzerini, Francesco Marcelloni, Giovanni Marola, Simone Galigani
Neural network-based calibration of positron emission tomograph detector modules
Beatrice Lazzerini, Francesco Marcelloni, Giovanni Marola, Simone Galigani
Abstract:
In this paper we describe a neural network-based method aimed at automatically calibrating the detector module contained in a scanner for a high-resolution positron emission tomography (PET) system for small animals. The detector module is composed of crystal elements, arranged in a regular matrix and sensitive to gamma rays emitted by a radioactive source. The crystal matrix is optically coupled to a position-sensitive photo-multiplier tube, which reconstructs the original image. Calibration, required to cope with spatial distortions introduced by the optical system, consists of a segmentation process of the image produced after the photo-multiplier tube into a fixed number of areas. The purpose of this segmentation is to map each pixel of the perceived image onto the pertinent crystal, which was actually struck by the gamma ray emitted by the radioactive source.
In this paper we describe a neural network-based method aimed at automatically calibrating the detector module contained in a scanner for a high-resolution positron emission tomography (PET) system for small animals. The detector module is composed of crystal elements, arranged in a regular matrix and sensitive to gamma rays emitted by a radioactive source. The crystal matrix is optically coupled to a position-sensitive photo-multiplier tube, which reconstructs the original image. Calibration, required to cope with spatial distortions introduced by the optical system, consists of a segmentation process of the image produced after the photo-multiplier tube into a fixed number of areas. The purpose of this segmentation is to map each pixel of the perceived image onto the pertinent crystal, which was actually struck by the gamma ray emitted by the radioactive source.
ES2004-96
reduced dimensionality space for post placement quality inspection of components based on neural networks
Stefanos Goumas, Michalis Zervakis, George Rovithakis
reduced dimensionality space for post placement quality inspection of components based on neural networks
Stefanos Goumas, Michalis Zervakis, George Rovithakis
Abstract:
The emergence of surface mount technology devices has resulted in several important advantages including increased component density and size reduction on the printed circuit board, on the expense of quality inspection. Classical visual inspection techniques require time-consuming image processing to improve the accuracy of the inspected results. In this paper we reduce the computational complexity of classical machine vision approaches by proposing two neural network based techniques. In the first we maintain image information only in the form of edges, whereas the second we preserve the entire content of info but compressed in a single dimension through image projections. Both algorithms are tested on real industrial data. The quality of inspection is preserved while reducing the computational time.
The emergence of surface mount technology devices has resulted in several important advantages including increased component density and size reduction on the printed circuit board, on the expense of quality inspection. Classical visual inspection techniques require time-consuming image processing to improve the accuracy of the inspected results. In this paper we reduce the computational complexity of classical machine vision approaches by proposing two neural network based techniques. In the first we maintain image information only in the form of edges, whereas the second we preserve the entire content of info but compressed in a single dimension through image projections. Both algorithms are tested on real industrial data. The quality of inspection is preserved while reducing the computational time.
Neural methods for non-standard data
ES2004-181
Neural methods for non-standard data
Barbara Hammer, Brijnesh J. Jain
Neural methods for non-standard data
Barbara Hammer, Brijnesh J. Jain
Abstract:
Standard pattern recognition provides effective and noise-tolerant tools for machine learning tasks; however, most approaches only deal with real vectors of a finite and fixed dimensionality. In this tutorial paper, we give an overview about extensions of pattern recognition towards non-standard data which are not contained in a finite dimensional space, such as strings, sequences, trees, graphs, or functions. Two major directions can be distinguished in the neural networks literature: models can be based on a similarity measure adapted to non-standard data, including kernel methods for structures as a very prominent approach, but also alternative metric based algorithms and functional networks; alternatively, non-standard data can be processed recursively within supervised and unsupervised recurrent and recursive networks and fully recurrent systems.
Standard pattern recognition provides effective and noise-tolerant tools for machine learning tasks; however, most approaches only deal with real vectors of a finite and fixed dimensionality. In this tutorial paper, we give an overview about extensions of pattern recognition towards non-standard data which are not contained in a finite dimensional space, such as strings, sequences, trees, graphs, or functions. Two major directions can be distinguished in the neural networks literature: models can be based on a similarity measure adapted to non-standard data, including kernel methods for structures as a very prominent approach, but also alternative metric based algorithms and functional networks; alternatively, non-standard data can be processed recursively within supervised and unsupervised recurrent and recursive networks and fully recurrent systems.
ES2004-136
a preliminary experimental comparison of recursive neural networks and a tree kernel method for QSAR/QSPR regression tasks
Alessio Micheli, Filippo Portera, Alessandro Sperduti
a preliminary experimental comparison of recursive neural networks and a tree kernel method for QSAR/QSPR regression tasks
Alessio Micheli, Filippo Portera, Alessandro Sperduti
Abstract:
We consider two different methods for QSAR/QSPR regression tasks: Recursive Neural Networks (RecNN) and a Support Vector Regression (SVR) machine using a Tree kernel. Experimental results on two specific regression tasks involving alkanes and benzodiazepines are obtained for the two approaches.
We consider two different methods for QSAR/QSPR regression tasks: Recursive Neural Networks (RecNN) and a Support Vector Regression (SVR) machine using a Tree kernel. Experimental results on two specific regression tasks involving alkanes and benzodiazepines are obtained for the two approaches.
ES2004-93
SVM learning with the SH inner product
Peter Geibel, Brijnesh J. Jain, Fritz Wysotzki
SVM learning with the SH inner product
Peter Geibel, Brijnesh J. Jain, Fritz Wysotzki
Abstract:
We apply support vector learning to attributed graphs where the kernel matrices are based on approximations of the Schur-Hadamard (SH) inner product. We present and discuss experimental results of different classifiers constructed by a SVM operating on positive semi-definite (psd) and non-psd kernel matrices.
We apply support vector learning to attributed graphs where the kernel matrices are based on approximations of the Schur-Hadamard (SH) inner product. We present and discuss experimental results of different classifiers constructed by a SVM operating on positive semi-definite (psd) and non-psd kernel matrices.
ES2004-34
Clustering functional data with the SOM algorithm
Fabrice Rossi, Brieuc Conan-Guez, Aïcha El Golli
Clustering functional data with the SOM algorithm
Fabrice Rossi, Brieuc Conan-Guez, Aïcha El Golli
Abstract:
In many situations, high dimensional data can be considered as sampled functions. We show in this paper how to implement a Self-Organizing Map (SOM) on such data by approximating a theoretical SOM on functions thanks to basis expansion. We illustrate the proposed method on real world spectrometric data for which functional preprocessing is very successful.
In many situations, high dimensional data can be considered as sampled functions. We show in this paper how to implement a Self-Organizing Map (SOM) on such data by approximating a theoretical SOM on functions thanks to basis expansion. We illustrate the proposed method on real world spectrometric data for which functional preprocessing is very successful.
ES2004-155
functional radial basis function networks
Nicolas Delannay, Fabrice Rossi, Brieuc Conan-Guez, Michel Verleysen
functional radial basis function networks
Nicolas Delannay, Fabrice Rossi, Brieuc Conan-Guez, Michel Verleysen
Abstract:
There has been recently a lot of interest for functional data analysis and extensions of well-known methods to functional inputs (clustering algorithm, non-parametric models, MLP). The main motivation of these methods is to benefit from the enforced inner structure of the data. This paper presents how functional data can be used with RBFN, and how the inner structure of the former can help design the network.
There has been recently a lot of interest for functional data analysis and extensions of well-known methods to functional inputs (clustering algorithm, non-parametric models, MLP). The main motivation of these methods is to benefit from the enforced inner structure of the data. This paper presents how functional data can be used with RBFN, and how the inner structure of the former can help design the network.
ES2004-156
Functional preprocessing for multilayer perceptrons
Fabrice Rossi, Brieuc Conan-Guez
Functional preprocessing for multilayer perceptrons
Fabrice Rossi, Brieuc Conan-Guez
Abstract:
In many applications, high dimensional input data can be considered as sampled functions. We show in this paper how to use this prior knowledge to implement functional preprocessings that allow to consistantly reduce the dimension of the data even when they have missing values. Preprocessed functions are then handled by a numerical MLP which approximate the theoretical functional MLP. A successful application to spectrometric data is proposed to illustrate the method.
In many applications, high dimensional input data can be considered as sampled functions. We show in this paper how to use this prior knowledge to implement functional preprocessings that allow to consistantly reduce the dimension of the data even when they have missing values. Preprocessed functions are then handled by a numerical MLP which approximate the theoretical functional MLP. A successful application to spectrometric data is proposed to illustrate the method.
ES2004-40
Recursive networks for processing graphs with labelled edges
Monica Bianchini, Marco Maggini, Lorenzo Sarti, Franco Scarselli
Recursive networks for processing graphs with labelled edges
Monica Bianchini, Marco Maggini, Lorenzo Sarti, Franco Scarselli
Abstract:
In this paper, we propose a new recursive neural network model, able to process directed acyclic graphs with labelled edges. The model is based on a different definition of the state transition function, which is independent both from the number and the order of the children of each node. In fact, the particular contribution of each child is encoded in the label attached to the corresponding edge. The computational capabilities of the new recursive architecture are also assessed.
In this paper, we propose a new recursive neural network model, able to process directed acyclic graphs with labelled edges. The model is based on a different definition of the state transition function, which is independent both from the number and the order of the children of each node. In fact, the particular contribution of each child is encoded in the label attached to the corresponding edge. The computational capabilities of the new recursive architecture are also assessed.
ES2004-92
The maximum weighted clique problem and Hopfield networks
Brijnesh J. Jain, Fritz Wysotzki
The maximum weighted clique problem and Hopfield networks
Brijnesh J. Jain, Fritz Wysotzki
Abstract:
We propose a neural network solution of the maximum weighted clique problem (MWCP). The MWCP problem comprises the well-known maximum clique and maximum vertex-weighted clique problem as special cases. We present bounds for the parameter settings of a special Hopfield network to ensure energy descent to feasible solutions of the MWCP. To verify the theoretical results we show the effectiveness of the proposed approach in an experimental study.
We propose a neural network solution of the maximum weighted clique problem (MWCP). The MWCP problem comprises the well-known maximum clique and maximum vertex-weighted clique problem as special cases. We present bounds for the parameter settings of a special Hopfield network to ensure energy descent to feasible solutions of the MWCP. To verify the theoretical results we show the effectiveness of the proposed approach in an experimental study.
Learning III
ES2004-2
Lattice ICA for the separation of speech signals
Manuel Rodríguez-Alvarez, Fernando Rojas-Ruiz, Elmar Wolfgang Lang, Ignacio Rojas-Ruiz, Carlos García-Puntonet, Moisés Salmerón-Campos
Lattice ICA for the separation of speech signals
Manuel Rodríguez-Alvarez, Fernando Rojas-Ruiz, Elmar Wolfgang Lang, Ignacio Rojas-Ruiz, Carlos García-Puntonet, Moisés Salmerón-Campos
Abstract:
This work explains a method for blind separation of a linear mixture of sources, through geometrical considerations concerning the scatter plot. This method is applied to a mixture of several sources and it obtains the estimated coefficients of the unknown mixture matrix A and separates the unknown sources.
This work explains a method for blind separation of a linear mixture of sources, through geometrical considerations concerning the scatter plot. This method is applied to a mixture of several sources and it obtains the estimated coefficients of the unknown mixture matrix A and separates the unknown sources.
ES2004-125
Robust overcomplete matrix recovery for sparse sources using a generalized Hough transform
Fabian J. Theis, Pando Georgiev, Andrzej Cichocki
Robust overcomplete matrix recovery for sparse sources using a generalized Hough transform
Fabian J. Theis, Pando Georgiev, Andrzej Cichocki
Abstract:
We propose an algorithm for recovering the matrix A in X=AS where X is a random vector of lower dimension than S. S is assumed to be sparse in the sense that S has less nonzero elements than the dimension of X at any given time instant. In contrast to previous approaches, the computational time of the presented algorithm is linear in the sample number and independent of source dimension, and the algorithm is robust against noise. Experiments confirm these theoretical results.
We propose an algorithm for recovering the matrix A in X=AS where X is a random vector of lower dimension than S. S is assumed to be sparse in the sense that S has less nonzero elements than the dimension of X at any given time instant. In contrast to previous approaches, the computational time of the presented algorithm is linear in the sample number and independent of source dimension, and the algorithm is robust against noise. Experiments confirm these theoretical results.
ES2004-21
SOM algorithms and their stability consideration
Youichi Kobuchi, Masataka Tanoue
SOM algorithms and their stability consideration
Youichi Kobuchi, Masataka Tanoue
Abstract:
A two layered neural network is considered as Kohonen’s dot-product type SOM model which defines a winner function f from an input pattern set into an output unit set. It defines pattern classifiers through step by step self-organization. What kinds of classifier it can ultimately be and how it is attained asymptotically are the basic problems to be answered. Our main result is that property of being topographic can be preserved by appropriately choosing learning rates and hence the winner function becomes stable in this case. To get the result, we first found conditions that f becomes stable under an ordinary one-step learning dynamics ("alpha"-conservativeness). If the learning rate is smaller, the more stable f can be. We obtained the condition from the requirement that W and W’ defines the same winner function where W’ is an updated weight matrix after any one-step learning. Then we deduce conservativeness concept as the one where "alpha" -conservativeness holds for arbitrary "alpha" in (0, 1). Since the conservative condition is not necessarily mutually exclusive, we consider more strict case as topographic. So what we call the topographic property is deduced as a special case of conservativeness.
A two layered neural network is considered as Kohonen’s dot-product type SOM model which defines a winner function f from an input pattern set into an output unit set. It defines pattern classifiers through step by step self-organization. What kinds of classifier it can ultimately be and how it is attained asymptotically are the basic problems to be answered. Our main result is that property of being topographic can be preserved by appropriately choosing learning rates and hence the winner function becomes stable in this case. To get the result, we first found conditions that f becomes stable under an ordinary one-step learning dynamics ("alpha"-conservativeness). If the learning rate is smaller, the more stable f can be. We obtained the condition from the requirement that W and W’ defines the same winner function where W’ is an updated weight matrix after any one-step learning. Then we deduce conservativeness concept as the one where "alpha" -conservativeness holds for arbitrary "alpha" in (0, 1). Since the conservative condition is not necessarily mutually exclusive, we consider more strict case as topographic. So what we call the topographic property is deduced as a special case of conservativeness.
ES2004-24
input arrival-time-dependent decoding scheme for a spiking neural network
Hesham Amin, Robert Fujii
input arrival-time-dependent decoding scheme for a spiking neural network
Hesham Amin, Robert Fujii
Abstract:
Spiking neurons model a type of biological neural system where information is encoded with spike times. In this paper, a new method for decoding input spikes according to their absolute arrival times is proposed. The output times, which are responses to different input patterns, can differentiate these input patterns uniquely. Features of Spiking Neural Networks (SNN) such as actual spike input time and synaptic weights are utilized. Only a limited number of neurons are needed to implement the decoding scheme.
Spiking neurons model a type of biological neural system where information is encoded with spike times. In this paper, a new method for decoding input spikes according to their absolute arrival times is proposed. The output times, which are responses to different input patterns, can differentiate these input patterns uniquely. Features of Spiking Neural Networks (SNN) such as actual spike input time and synaptic weights are utilized. Only a limited number of neurons are needed to implement the decoding scheme.
ES2004-94
Novel approximations for inference and learning in nonlinear dynamical systems
Alexander Ypma, Tom Heskes
Novel approximations for inference and learning in nonlinear dynamical systems
Alexander Ypma, Tom Heskes
Abstract:
We formulate the problem of inference in nonlinear dynamical systems in the EP-framework, and propose two novel inference algorithms based on Laplace approximation and the Unscented transform. The algorithms are compared empirically and then employed as E-step in a conjugate gradient learning algorithm. We illustrate its use for data mining with a high-dimensional time series from marketing research.
We formulate the problem of inference in nonlinear dynamical systems in the EP-framework, and propose two novel inference algorithms based on Laplace approximation and the Unscented transform. The algorithms are compared empirically and then employed as E-step in a conjugate gradient learning algorithm. We illustrate its use for data mining with a high-dimensional time series from marketing research.
ES2004-97
Computational model of amygdala network supported by neurobiological data
Mélanie Falgairolle, Agnès Gorge, Jean-Marc Salotti, Marc-Michel Corsini
Computational model of amygdala network supported by neurobiological data
Mélanie Falgairolle, Agnès Gorge, Jean-Marc Salotti, Marc-Michel Corsini
Abstract:
The amygdala has repeatedly been involved in the processing of emotional reactions and conditioning. This paper presents a neurobiologically inspired computational model of the emotional memory in aversive behaviors. This artificial neural network aims at partially reproduce the same characteristics as the amygdala when it induces the aversive state experienced by individuals with a withdrawal effect.
The amygdala has repeatedly been involved in the processing of emotional reactions and conditioning. This paper presents a neurobiologically inspired computational model of the emotional memory in aversive behaviors. This artificial neural network aims at partially reproduce the same characteristics as the amygdala when it induces the aversive state experienced by individuals with a withdrawal effect.
ES2004-109
Reducing connectivity by using cortical modular bands
Julien Vitay, Nicolas Rougier, Frédéric Alexandre
Reducing connectivity by using cortical modular bands
Julien Vitay, Nicolas Rougier, Frédéric Alexandre
Abstract:
The way information is represented and processed in a neural network may have important consequences on its computational power and complexity. Basically, information representation refers to distributed or localist encoding and information processing refers to schemes of connectivity that can be complete or minimal. In the past, theoretical and biologically inspired approaches of neural computation have insisted on complementary views (respectively distributed and complete versus localist and minimal) with complementary arguments (complexity versus expressiveness). In this paper, we report experiments on biologically inspired neural networks performing sensorimotor coordination that indicate that a localist and minimal view may have good performances if some connectivity constraints (also coming from biological inspiration) are respected.
The way information is represented and processed in a neural network may have important consequences on its computational power and complexity. Basically, information representation refers to distributed or localist encoding and information processing refers to schemes of connectivity that can be complete or minimal. In the past, theoretical and biologically inspired approaches of neural computation have insisted on complementary views (respectively distributed and complete versus localist and minimal) with complementary arguments (complexity versus expressiveness). In this paper, we report experiments on biologically inspired neural networks performing sensorimotor coordination that indicate that a localist and minimal view may have good performances if some connectivity constraints (also coming from biological inspiration) are respected.
ES2004-110
Modelling of biologically plausible excitatory networks: emergence and modulation of neural synchrony
Karsten Kube, Andreas Herzog, Vadym Spravedlyvyy, Bernd Michaelis, Thoralf Opitz, Ana de Lima, Thomas Voigt
Modelling of biologically plausible excitatory networks: emergence and modulation of neural synchrony
Karsten Kube, Andreas Herzog, Vadym Spravedlyvyy, Bernd Michaelis, Thoralf Opitz, Ana de Lima, Thomas Voigt
Abstract:
To emphasize the electrical nature of information processing in the brain we use a compartmental model of single neurons. The realistic simulation of wave-like activity in the recurrent excitatory network is similar to the intracellular activation in rat embryonal cerebral cortex cultures [1]. The natural structure of the network is reproduced by including interactions between different functional neurons. We start by reproducing spontaneous electrical activity of single neurons. After massive simulations selective influences are comparable to in vitro measured activity. We show adaptation of the network behavior by introducing external stimulation.
To emphasize the electrical nature of information processing in the brain we use a compartmental model of single neurons. The realistic simulation of wave-like activity in the recurrent excitatory network is similar to the intracellular activation in rat embryonal cerebral cortex cultures [1]. The natural structure of the network is reproduced by including interactions between different functional neurons. We start by reproducing spontaneous electrical activity of single neurons. After massive simulations selective influences are comparable to in vitro measured activity. We show adaptation of the network behavior by introducing external stimulation.
ES2004-111
Learning by geometrical shape changes of dendritic spines
Andreas Herzog, Vadym Spravedlyvyy, Karsten Kube, Reinhild Schnabel, Eduard Korkotian, Katharina Braun, Bernd Michaelis
Learning by geometrical shape changes of dendritic spines
Andreas Herzog, Vadym Spravedlyvyy, Karsten Kube, Reinhild Schnabel, Eduard Korkotian, Katharina Braun, Bernd Michaelis
Abstract:
Abstract. The role of dendritic spines in neuronal information processing is still not completely clear. However, it is known that spines can change shape rapidly during development and during learning and these morphological changes might be relevant for information storage (memory formation). We demonstrate the impact of shape variations on electrical signal propagation via dendritic spines using a biologically realistic electrical simulation procedure. Basic properties of electrical signal transduction of single spines are estimated and approximated in relation to their individual shape features. Learning processes to adjust specific electrical properties are discussed and a possible mechanism is introduced.
Abstract. The role of dendritic spines in neuronal information processing is still not completely clear. However, it is known that spines can change shape rapidly during development and during learning and these morphological changes might be relevant for information storage (memory formation). We demonstrate the impact of shape variations on electrical signal propagation via dendritic spines using a biologically realistic electrical simulation procedure. Basic properties of electrical signal transduction of single spines are estimated and approximated in relation to their individual shape features. Learning processes to adjust specific electrical properties are discussed and a possible mechanism is introduced.
ES2004-112
Neuro-predictive control based self-tuning of PID controllers
Corneliu Lazar, Sorin Carari, Draguna Vrabie, Marius Kloetzer
Neuro-predictive control based self-tuning of PID controllers
Corneliu Lazar, Sorin Carari, Draguna Vrabie, Marius Kloetzer
Abstract:
In this paper we present a new self-tuning procedure for PID controllers based on neuro-predictive control. A finite horizon optimal control problem is solved on-line, permitting to calculate the tuning parameters of the PID controller. The proposed method is implemented on a level-flow pilot plant and a comparison with conventional auto-tuning methods is also given.
In this paper we present a new self-tuning procedure for PID controllers based on neuro-predictive control. A finite horizon optimal control problem is solved on-line, permitting to calculate the tuning parameters of the PID controller. The proposed method is implemented on a level-flow pilot plant and a comparison with conventional auto-tuning methods is also given.
Hardware systems for neural devices
ES2004-187
Neural Hardware: beyond ones and zeros
Fleury Patrice, Bofill-i-Petit Adria, Alan Murray
Neural Hardware: beyond ones and zeros
Fleury Patrice, Bofill-i-Petit Adria, Alan Murray
Abstract:
An overview of research on the implementation of neural systems is presented in this paper. We focus on implementations where the algorithms and their physical support are tightly coupled. First, we concentrate on the potential of probabilistic algorithms to compensate for hardware non-idealities. Then, electronic circuits which aim to reproduce the structure of neurobiological systems in hardware are introduced. Finally, we extend to neuroengineering whose focus is placed on interfacing artificial devices with biological systems
An overview of research on the implementation of neural systems is presented in this paper. We focus on implementations where the algorithms and their physical support are tightly coupled. First, we concentrate on the potential of probabilistic algorithms to compensate for hardware non-idealities. Then, electronic circuits which aim to reproduce the structure of neurobiological systems in hardware are introduced. Finally, we extend to neuroengineering whose focus is placed on interfacing artificial devices with biological systems
ES2004-55
A VLSI reconfigurable network of integrate-and-fire neurons with spike-based learning synapses
Giacomo Indiveri, Elisabetta Chicca, Rodney Douglas
A VLSI reconfigurable network of integrate-and-fire neurons with spike-based learning synapses
Giacomo Indiveri, Elisabetta Chicca, Rodney Douglas
Abstract:
We present a VLSI device comprising an array of leaky integrate-and-fire (I&F) neurons and adaptive synapses with spike-timing dependent plasticity (STDP) learning. The neurons can transmit spikes off chip and the synapses can receive spikes from external devices using an "Address-Event Representation" (AER). We described the network architecture, presented its response properties to uniform input currents, measuring its AER outputs, and demonstrated the properties of the STDP synapses using AER input spike trains. Our results indicate that these circuits can be reliably used in massively parallel VLSI networks of I&F neurons to simulate in real-time complex spike-based learning algorithms.
We present a VLSI device comprising an array of leaky integrate-and-fire (I&F) neurons and adaptive synapses with spike-timing dependent plasticity (STDP) learning. The neurons can transmit spikes off chip and the synapses can receive spikes from external devices using an "Address-Event Representation" (AER). We described the network architecture, presented its response properties to uniform input currents, measuring its AER outputs, and demonstrated the properties of the STDP synapses using AER input spike trains. Our results indicate that these circuits can be reliably used in massively parallel VLSI networks of I&F neurons to simulate in real-time complex spike-based learning algorithms.
ES2004-164
BIOSEG: a bioinspired vlsi analog system for image segmentation
Jordi Madrenas, Jordi Cosp, Lucas Oscar, Eduard Alarcón, Eva Vidal, Gerard Villar
BIOSEG: a bioinspired vlsi analog system for image segmentation
Jordi Madrenas, Jordi Cosp, Lucas Oscar, Eduard Alarcón, Eva Vidal, Gerard Villar
Abstract:
The architecture of a complete image segmentation system and the development of an embedded VLSI low-power integrated circuit are reported. A neuromorphic engineering approach is adopted, with the purpose of reproducing behaviour of biological neural networks by taking advantage of the microelectronic implementation properties, especially low power consumption and reduced volume and weight. The system is divided in parallel-processing stages. After phototransduction, nonlinear filtering is applied to the image. Then, segmentation is performed and an output stage delivers segmentation and object properties information. Each stage is briefly described and simulations and experimental results are shown. The final goal is to develop a single-chip integrated system that performs all the described operations in focal-plane.
The architecture of a complete image segmentation system and the development of an embedded VLSI low-power integrated circuit are reported. A neuromorphic engineering approach is adopted, with the purpose of reproducing behaviour of biological neural networks by taking advantage of the microelectronic implementation properties, especially low power consumption and reduced volume and weight. The system is divided in parallel-processing stages. After phototransduction, nonlinear filtering is applied to the image. Then, segmentation is performed and an output stage delivers segmentation and object properties information. Each stage is briefly described and simulations and experimental results are shown. The final goal is to develop a single-chip integrated system that performs all the described operations in focal-plane.
ES2004-77
Implementation and coupling of dynamic neurons through optoelectronics
Alexandre Romariz, Kelvin Wagner
Implementation and coupling of dynamic neurons through optoelectronics
Alexandre Romariz, Kelvin Wagner
Abstract:
In this work we describe experimental results regarding an optoelectronic implementation of a dynamic neuron model. The model is a variation of the FitzHugh-Nagumo equations, and it is implemented with linear optics and simple linear electronic feedback. The demonstration of dynamic features of the isolated neuron and of optical coupling between neurons is discussed, as well as the computational perspective of large arrays of such neurons.
In this work we describe experimental results regarding an optoelectronic implementation of a dynamic neuron model. The model is a variation of the FitzHugh-Nagumo equations, and it is implemented with linear optics and simple linear electronic feedback. The demonstration of dynamic features of the isolated neuron and of optical coupling between neurons is discussed, as well as the computational perspective of large arrays of such neurons.
ES2004-141
Architectures for Nanoelectronic Neural Networks: New Results
Ozgur Turel, Jung Hoon Lee, Xiaolong Ma, Konstantin K. Likharev
Architectures for Nanoelectronic Neural Networks: New Results
Ozgur Turel, Jung Hoon Lee, Xiaolong Ma, Konstantin K. Likharev
Abstract:
Our group is developing artificial neural networks that may be implemented using hybrid semiconductor/molecular (“CMOL”) circuits. Estimates show that such networks (“CrossNets”) may eventually exceed the mammal brain in areal density, at much higher speed and acceptable power consumption. In this report, we demonstrate that CrossNets based on simple (two-terminal) molecular devices can work well in at least two modes: as Hopfield networks with high defect tolerance, as well as simple and multilayer perceptrons.
Our group is developing artificial neural networks that may be implemented using hybrid semiconductor/molecular (“CMOL”) circuits. Estimates show that such networks (“CrossNets”) may eventually exceed the mammal brain in areal density, at much higher speed and acceptable power consumption. In this report, we demonstrate that CrossNets based on simple (two-terminal) molecular devices can work well in at least two modes: as Hopfield networks with high defect tolerance, as well as simple and multilayer perceptrons.
Support vector machines
ES2004-29
Fuzzy LP-SVMs for Multiclass Problems
Shigeo Abe
Fuzzy LP-SVMs for Multiclass Problems
Shigeo Abe
Abstract:
In this paper, we propose fuzzy linear programming support vector machines (LP-SVMs) that resolve unclassifiable regions for multiclass problems. Namely, in the directions orthogonal to the decision functions obtained by training the LP-SVM, we define membership functions. Then by the minimum or average operation for these membership functions we define a membership function for each class. We evaluate one-against-all and pairwise fuzzy LP-SVMs for some benchmark data sets and demonstrate the superiority of our fuzzy LP-SVMs over conventional LP-SVMs.
In this paper, we propose fuzzy linear programming support vector machines (LP-SVMs) that resolve unclassifiable regions for multiclass problems. Namely, in the directions orthogonal to the decision functions obtained by training the LP-SVM, we define membership functions. Then by the minimum or average operation for these membership functions we define a membership function for each class. We evaluate one-against-all and pairwise fuzzy LP-SVMs for some benchmark data sets and demonstrate the superiority of our fuzzy LP-SVMs over conventional LP-SVMs.
ES2004-116
sparse LS-SVMs using additive regularization with a penalized validation criterion
Kristiaan Pelckmans, Johan A.K. Suykens, Bart De Moor
sparse LS-SVMs using additive regularization with a penalized validation criterion
Kristiaan Pelckmans, Johan A.K. Suykens, Bart De Moor
Abstract:
This paper is based on a new way for determining the regularization trade-off in least squares support vector machines (LS-SVMs) via a mechanism of additive regularization which has been recently introduced in cite{areg}. This framework enables computational fusion of training and validation levels and allows to train the model together with finding the regularization constants by solving a single linear system at once. In this paper we show that this framework allows to consider a penalized validation criterion that leads to sparse LS-SVMs. The model, regularization constants and sparseness follow from a convex quadratic program in this case.
This paper is based on a new way for determining the regularization trade-off in least squares support vector machines (LS-SVMs) via a mechanism of additive regularization which has been recently introduced in cite{areg}. This framework enables computational fusion of training and validation levels and allows to train the model together with finding the regularization constants by solving a single linear system at once. In this paper we show that this framework allows to consider a penalized validation criterion that leads to sparse LS-SVMs. The model, regularization constants and sparseness follow from a convex quadratic program in this case.
ES2004-11
Bias Term b in SVMs Again
TE-MING HUANG, Vojislav Kecman
Bias Term b in SVMs Again
TE-MING HUANG, Vojislav Kecman
Abstract:
The paper discusses and presents the use and calculation of the explicit bias term b in the support vector machines (SVMs) within the Iterative Single training Data learning Algorithm (ISDA). The approach proposed can be used for both nonlinear classification and nonlinear regression tasks. Unlike the other iterative methods in solving the SVMs learning problems containing the huge data sets, such as sequential minimal optimization (SMO) and its variants that must use at least two training data pairs, the algorithms shown here use the single training data based iteration routine for solving QP learning problem. In this way the various 2nd order heuristics in choosing the data for an updating is avoided. This makes the proposed ISD learning method remarkably quick. The algorithm can also be thought off as an application of a classic Gauss-Seidel (GS) coordinate ascent procedure and its derivative known as the successive over-relaxation (SOR) algorithm in SVMs learning from huge data sets subject to both the box constraints and the equality ones. (The later coming from minimizing the primal objective function in respect to the bias term b). The final solution in a dual domain is not an approximate one, but it is the optimal set of dual variables which would have been obtained by using any of existing and proven QP problem solvers if they only could deal with huge data sets.
The paper discusses and presents the use and calculation of the explicit bias term b in the support vector machines (SVMs) within the Iterative Single training Data learning Algorithm (ISDA). The approach proposed can be used for both nonlinear classification and nonlinear regression tasks. Unlike the other iterative methods in solving the SVMs learning problems containing the huge data sets, such as sequential minimal optimization (SMO) and its variants that must use at least two training data pairs, the algorithms shown here use the single training data based iteration routine for solving QP learning problem. In this way the various 2nd order heuristics in choosing the data for an updating is avoided. This makes the proposed ISD learning method remarkably quick. The algorithm can also be thought off as an application of a classic Gauss-Seidel (GS) coordinate ascent procedure and its derivative known as the successive over-relaxation (SOR) algorithm in SVMs learning from huge data sets subject to both the box constraints and the equality ones. (The later coming from minimizing the primal objective function in respect to the bias term b). The final solution in a dual domain is not an approximate one, but it is the optimal set of dual variables which would have been obtained by using any of existing and proven QP problem solvers if they only could deal with huge data sets.
Neural networks for data mining
ES2004-180
Neural networks for data mining: constrains and open problems
Razvan Andonie, Boris Kovalerchuk
Neural networks for data mining: constrains and open problems
Razvan Andonie, Boris Kovalerchuk
Abstract:
When we talk about using neural networks for data mining we have in mind the original data mining scope and challenge. How did neural networks meet this challenge? Can we run neural networks on a dataset with gigabytes of data and millions of records? Can we provide explanations of discovered patterns? How useful that patterns are? How to distinguish useful, interesting patterns automatically? We aim to summarize here the state-of-the-art of the principles beyond using neural models in data mining.
When we talk about using neural networks for data mining we have in mind the original data mining scope and challenge. How did neural networks meet this challenge? Can we run neural networks on a dataset with gigabytes of data and millions of records? Can we provide explanations of discovered patterns? How useful that patterns are? How to distinguish useful, interesting patterns automatically? We aim to summarize here the state-of-the-art of the principles beyond using neural models in data mining.
ES2004-10
Visualization and classification with categorical topological map
mustapha lebbah, fouad Badran, Sylvie Thiria
Visualization and classification with categorical topological map
mustapha lebbah, fouad Badran, Sylvie Thiria
Abstract:
This paper introduces a topological map dedicated to cluster analysis and visualization of categorical data. Usually, when dealing with symbolic data, topological maps use an encoding stage: symbolic data are changed into numerical vectors and traditional numerical algorithms are run. In the present paper, we propose a probabilistic formalism where neurons are now represented by probability tables. Two examples using actual and synthetic data allow to validate the approach. The results show the good quality of the topological order obtained as well as its performances in classification.
This paper introduces a topological map dedicated to cluster analysis and visualization of categorical data. Usually, when dealing with symbolic data, topological maps use an encoding stage: symbolic data are changed into numerical vectors and traditional numerical algorithms are run. In the present paper, we propose a probabilistic formalism where neurons are now represented by probability tables. Two examples using actual and synthetic data allow to validate the approach. The results show the good quality of the topological order obtained as well as its performances in classification.
ES2004-61
Visualizing distortions in continuous projection techniques
Michaël Aupetit
Visualizing distortions in continuous projection techniques
Michaël Aupetit
Abstract:
Visualization of multi-dimensional data has been studied for a long time. Here we propose to visualize any distortion measure associated to a projected datum in continuous projection techniques, by coloring its corresponding Voronoï cell in the projection space. We apply this approach to detect where the high-dimensional manifold has been torn or glued during the projection. We experiment this technique with Principal Component Analysis and Curvilinear Component Analysis for different databases.
Visualization of multi-dimensional data has been studied for a long time. Here we propose to visualize any distortion measure associated to a projected datum in continuous projection techniques, by coloring its corresponding Voronoï cell in the projection space. We apply this approach to detect where the high-dimensional manifold has been torn or glued during the projection. We experiment this technique with Principal Component Analysis and Curvilinear Component Analysis for different databases.
ES2004-38
An informational energy LVQ approach for feature ranking
Razvan Andonie, Angel Cataron
An informational energy LVQ approach for feature ranking
Razvan Andonie, Angel Cataron
Abstract:
Input feature ranking and selection is a necessary preprocessing stage in classification, especially when one is required to manage large quantities of data. We introduce a weighted LVQ algorithm, called Energy Relevance LVQ (ERLVQ), based on informational energy. ERLVQ is an incremental learning algorithm for supervised classification and feature ranking.
Input feature ranking and selection is a necessary preprocessing stage in classification, especially when one is required to manage large quantities of data. We introduce a weighted LVQ algorithm, called Energy Relevance LVQ (ERLVQ), based on informational energy. ERLVQ is an incremental learning algorithm for supervised classification and feature ranking.
ES2004-124
Using Andrews Curves for Clustering and Sub-clustering Self-Organizing Maps
Cesar Garcia-Osorio, Jesus Maudes, Colin Fyfe
Using Andrews Curves for Clustering and Sub-clustering Self-Organizing Maps
Cesar Garcia-Osorio, Jesus Maudes, Colin Fyfe
Abstract:
The use of self-organizing maps to analyze data often depends on finding effective methods to visualize the SOM's structure. In this paper we propose a new way to perform that visualization using a variant of Andrews' Curves. Also we show that the interaction between these two methods allows us to find sub-clusters within identified clusters.
The use of self-organizing maps to analyze data often depends on finding effective methods to visualize the SOM's structure. In this paper we propose a new way to perform that visualization using a variant of Andrews' Curves. Also we show that the interaction between these two methods allows us to find sub-clusters within identified clusters.
ES2004-23
Data Mining Techniques on the Evaluation of Wireless Churn
Jorge Ferreira, Marley Vellasco, Marco Aurélio Pacheco, Carlos Barbosa
Data Mining Techniques on the Evaluation of Wireless Churn
Jorge Ferreira, Marley Vellasco, Marco Aurélio Pacheco, Carlos Barbosa
Abstract:
This work focuses on one of the most critical issues to plague the wireless telecommunications industry today: the loss of a valuable subscriber to a competitor, also defined as churn. Analytical methods and models intrinsic to decision technology and machine learning are here evaluated, in an effort to provide the necessary intelligence to identify and understand troublesome customers in order to act upon them before they churn. Making use of a large real-world database, a thorough analysis is performed. First, due attention is given to data representation, with input selection methods being employed in the search of the most relevant attributes. Then, the predictive and explanatory power of four families of models is compared: neural networks, decision trees, genetic algorithms and neuro-fuzzy systems. To conclude, light is shed upon the possible savings and profits resulting from the application of the developed methodology in the retention strategies of wireless carriers.
This work focuses on one of the most critical issues to plague the wireless telecommunications industry today: the loss of a valuable subscriber to a competitor, also defined as churn. Analytical methods and models intrinsic to decision technology and machine learning are here evaluated, in an effort to provide the necessary intelligence to identify and understand troublesome customers in order to act upon them before they churn. Making use of a large real-world database, a thorough analysis is performed. First, due attention is given to data representation, with input selection methods being employed in the search of the most relevant attributes. Then, the predictive and explanatory power of four families of models is compared: neural networks, decision trees, genetic algorithms and neuro-fuzzy systems. To conclude, light is shed upon the possible savings and profits resulting from the application of the developed methodology in the retention strategies of wireless carriers.
ES2004-35
meaningful discretization of continuous features for association rules mining by means of a SOM
Marco Vannucci, Valentina Colla
meaningful discretization of continuous features for association rules mining by means of a SOM
Marco Vannucci, Valentina Colla
Abstract:
Abstract. The paper presents the problem of the unsupervised discretization of continuous attributes for association rules mining. It shows commonly used techniques for this aim and highlights their principal limitations. To overcome such limitations a method based on the use of a SOM is presented and tested over various real world datasets.
Abstract. The paper presents the problem of the unsupervised discretization of continuous attributes for association rules mining. It shows commonly used techniques for this aim and highlights their principal limitations. To overcome such limitations a method based on the use of a SOM is presented and tested over various real world datasets.
ES2004-37
Convergence properties of a fuzzy ARTMAP network
Razvan Andonie, Lucian Sasu
Convergence properties of a fuzzy ARTMAP network
Razvan Andonie, Lucian Sasu
Abstract:
FAMR (Fuzzy ARTMAP with Relevance factor) is a FAM (Fuzzy ARTMAP) neural network used for classification, probability estimation, and function approximation. FAMR uses a relevance factor assigned to each sample pair, proportional to the importance of that pair during the learning phase. Due to its incremental learning capability, FAMR can efficiently process large data sets and is an appropriate tool for data mining applications. We present new theoretical results characterizing the stochastic convergence of FAMR.
FAMR (Fuzzy ARTMAP with Relevance factor) is a FAM (Fuzzy ARTMAP) neural network used for classification, probability estimation, and function approximation. FAMR uses a relevance factor assigned to each sample pair, proportional to the importance of that pair during the learning phase. Due to its incremental learning capability, FAMR can efficiently process large data sets and is an appropriate tool for data mining applications. We present new theoretical results characterizing the stochastic convergence of FAMR.
ES2004-57
knowledge discovery in DNA microarray data of cancer patients with emergent self organizing maps
Alfred Ultsch, David Kämpf
knowledge discovery in DNA microarray data of cancer patients with emergent self organizing maps
Alfred Ultsch, David Kämpf
Abstract:
DNA microarrays provide a powerful means of monitoring thousands of gene expression levels at the same time. They consist of high dimensional data sets which challenge conventional clustering methods. The data's high dimensionality calls for Self Organizing Maps (SOMs) to cluster DNA microarray data. This paper shows that a precise estimation of the variables' variances is, however, the key to successful clustering of such data with SOMs. We propose PDEplots to verify the estimation of variances. PDEplots are probability density estimations based on information optimal sets. This paper demonstrates the application of PDEplots for clustering DNA microarray data of leukemia with the U-Matrix. Our approach reveals new insights into the structure of the leukemia dataset: PDEplots show two different distributions in the raw data. Three new subclasses are found with the U-Matrix.
DNA microarrays provide a powerful means of monitoring thousands of gene expression levels at the same time. They consist of high dimensional data sets which challenge conventional clustering methods. The data's high dimensionality calls for Self Organizing Maps (SOMs) to cluster DNA microarray data. This paper shows that a precise estimation of the variables' variances is, however, the key to successful clustering of such data with SOMs. We propose PDEplots to verify the estimation of variances. PDEplots are probability density estimations based on information optimal sets. This paper demonstrates the application of PDEplots for clustering DNA microarray data of leukemia with the U-Matrix. Our approach reveals new insights into the structure of the leukemia dataset: PDEplots show two different distributions in the raw data. Three new subclasses are found with the U-Matrix.
ES2004-102
Fast semi-automatic segmentation algorithm for Self-Organizing Maps
David Opolon, Fabien Moutarde
Fast semi-automatic segmentation algorithm for Self-Organizing Maps
David Opolon, Fabien Moutarde
Abstract:
Self-Organizing Maps (SOM) are very powerful tools for data mining, in particular for visualizing the distribution of the data in very high-dimensional data sets. Moreover, the 2D map produced by SOM can be used for unsupervised partitioning of the original data set into categories, provided that this map is somehow adequately segmented in clusters. This is usually done either manually by visual inspection, or by applying a classical clustering technique (such as agglomerative clustering) to the set of prototypes corresponding to the map. In this paper, we present a new approach for the segmentation of Self-Organizing Maps after training, which is both very simple and efficient. Our algorithm is based on a post-processing of the U-matrix (the matrix of distances between adjacent map units), which is directly derived from an elementary image-processing technique. It is shown on some simulated data sets that our partitioning algorithm appears to give very good results in terms of segmentation quality. Preliminary results on a real data set also seem to indicate that our algorithm can produce meaningful clusters on real data.
Self-Organizing Maps (SOM) are very powerful tools for data mining, in particular for visualizing the distribution of the data in very high-dimensional data sets. Moreover, the 2D map produced by SOM can be used for unsupervised partitioning of the original data set into categories, provided that this map is somehow adequately segmented in clusters. This is usually done either manually by visual inspection, or by applying a classical clustering technique (such as agglomerative clustering) to the set of prototypes corresponding to the map. In this paper, we present a new approach for the segmentation of Self-Organizing Maps after training, which is both very simple and efficient. Our algorithm is based on a post-processing of the U-matrix (the matrix of distances between adjacent map units), which is directly derived from an elementary image-processing technique. It is shown on some simulated data sets that our partitioning algorithm appears to give very good results in terms of segmentation quality. Preliminary results on a real data set also seem to indicate that our algorithm can produce meaningful clusters on real data.
ES2004-70
Integrated low noise signal conditioning interface for neuroengineering applications
Emanuele Bottino, Sergio Martinoia, Maurizio Valle
Integrated low noise signal conditioning interface for neuroengineering applications
Emanuele Bottino, Sergio Martinoia, Maurizio Valle
Abstract:
The birth of Neuroengineering, a new research field recently introduced by the synergic overlap between neuroscience and electronic engineering disciplines, injected a great enthusiasm in researchers. In fact, it opened a new perspective for addressing complex problems such as the understanding of the brain functions and the development of novel and advanced brain-computers. In this article, we offer a brief overview on implementations of integrated interface systems for neurobiological and electrophysiological in-vitro applications. After, we propose a system, still under development, aimed to achieving some hundreds of input channels. The architecture comprises a low-noise preamplifier stage and exhibits - for each channel - a power consumption of 90.05 µW and a silicon area of about 0.17 mm^2. Work is currently in progress to implement a fully integrated recording circuitry.
The birth of Neuroengineering, a new research field recently introduced by the synergic overlap between neuroscience and electronic engineering disciplines, injected a great enthusiasm in researchers. In fact, it opened a new perspective for addressing complex problems such as the understanding of the brain functions and the development of novel and advanced brain-computers. In this article, we offer a brief overview on implementations of integrated interface systems for neurobiological and electrophysiological in-vitro applications. After, we propose a system, still under development, aimed to achieving some hundreds of input channels. The architecture comprises a low-noise preamplifier stage and exhibits - for each channel - a power consumption of 90.05 µW and a silicon area of about 0.17 mm^2. Work is currently in progress to implement a fully integrated recording circuitry.
Learning IV
ES2004-15
Evolutionary tuning of multiple SVM parameters
Frauke Friedrichs, Christian Igel
Evolutionary tuning of multiple SVM parameters
Frauke Friedrichs, Christian Igel
Abstract:
We consider the problem of choosing multiple hyperparameters for support vector machines. We present a novel, general approach using an evolution strategy (ES) to determine the kernel from a parameterized kernel space and to control the regularization. We demonstrate on benchmark datasets that the ES improves the results achieved by grid search and can handle much more kernel parameters. In particular, we optimize generalized Gaussian kernels with arbitrary scaling and rotation.
We consider the problem of choosing multiple hyperparameters for support vector machines. We present a novel, general approach using an evolution strategy (ES) to determine the kernel from a parameterized kernel space and to control the regularization. We demonstrate on benchmark datasets that the ES improves the results achieved by grid search and can handle much more kernel parameters. In particular, we optimize generalized Gaussian kernels with arbitrary scaling and rotation.
ES2004-162
fast bootstrap for least-square support vector machines
Amaury Lendasse, Geoffroy Simon, Vincent Wertz, Michel Verleysen
fast bootstrap for least-square support vector machines
Amaury Lendasse, Geoffroy Simon, Vincent Wertz, Michel Verleysen
Abstract:
The Bootstrap resampling method may be efficiently used to estimate the generalization error of nonlinear regression models, as artificial neural networks and especially Least-square Support Vector Machines. Nevertheless, the use of the Bootstrap implies a high computational load. In this paper we present a simple procedure to obtain a fast approximation of this generalization error with a reduced computation time. This proposal is based on empirical evidence and included in a simulation procedure.
The Bootstrap resampling method may be efficiently used to estimate the generalization error of nonlinear regression models, as artificial neural networks and especially Least-square Support Vector Machines. Nevertheless, the use of the Bootstrap implies a high computational load. In this paper we present a simple procedure to obtain a fast approximation of this generalization error with a reduced computation time. This proposal is based on empirical evidence and included in a simulation procedure.
ES2004-115
Neural dynamics for task-oriented grouping of communicating agents
Jochen J. Steil
Neural dynamics for task-oriented grouping of communicating agents
Jochen J. Steil
Abstract:
Many real world problems are given in the form of multiple measurements comprising local descriptions or tasks. We propose that a dynamical organization of a population of communicating agents into groups oriented towards locally similar clusters of subtasks can identify higher level structure and solve such tasks. We assume that an agent may compute the compatibility of its resources with the input descriptions and that it can compare this compatibility with that of other agents. Based on dynamically updated soft assignment variables each agent computes its action preference distribution and communicates it to other agents. Applying theory developed for the competitive-layer model (CLM, Wersing, Steil, Ritter, Neural Computation 13, 357-387, 2001), a recurrent linear threshold network for feature binding and sensory segmentation, we give constructive conditions on the choice of the agents' compatibility functions and dynamical parameters to assure convergence. They guarantee that each agent unambiguously either decides for an action or not to be active at all. We give an approximative stochastic algorithm to sample the decision dynamics and discuss two examples.
Many real world problems are given in the form of multiple measurements comprising local descriptions or tasks. We propose that a dynamical organization of a population of communicating agents into groups oriented towards locally similar clusters of subtasks can identify higher level structure and solve such tasks. We assume that an agent may compute the compatibility of its resources with the input descriptions and that it can compare this compatibility with that of other agents. Based on dynamically updated soft assignment variables each agent computes its action preference distribution and communicates it to other agents. Applying theory developed for the competitive-layer model (CLM, Wersing, Steil, Ritter, Neural Computation 13, 357-387, 2001), a recurrent linear threshold network for feature binding and sensory segmentation, we give constructive conditions on the choice of the agents' compatibility functions and dynamical parameters to assure convergence. They guarantee that each agent unambiguously either decides for an action or not to be active at all. We give an approximative stochastic algorithm to sample the decision dynamics and discuss two examples.
ES2004-122
Learning from Reward as an emergent property of Physics-like interactions between neurons in an artificial neural network
Frédéric Davesne
Learning from Reward as an emergent property of Physics-like interactions between neurons in an artificial neural network
Frédéric Davesne
Abstract:
We study a class of artificial neural networks in which a physics-like conservation law upon the activity of connected neurons is imposed at each time. We postulate that the modification of the network activities may be interpreted as a learning capability if a judicious conservation law is chosen. We illustrate our claim by modeling a rat behavior in a labyrinth: the exploration of the labyrinth permits to create connections between neurons (latent learning), whereas the discovery of food induces a one step backpropagation process over the activities (reinforcement learning). We give theoretical results about our learning algorithm CbL and show it is intrinsically faster than Q-Learning.
We study a class of artificial neural networks in which a physics-like conservation law upon the activity of connected neurons is imposed at each time. We postulate that the modification of the network activities may be interpreted as a learning capability if a judicious conservation law is chosen. We illustrate our claim by modeling a rat behavior in a labyrinth: the exploration of the labyrinth permits to create connections between neurons (latent learning), whereas the discovery of food induces a one step backpropagation process over the activities (reinforcement learning). We give theoretical results about our learning algorithm CbL and show it is intrinsically faster than Q-Learning.
ES2004-9
Three dimensional frames of reference transformations using gain modulated populations of neurons
Eric Sauser, Aude Billard
Three dimensional frames of reference transformations using gain modulated populations of neurons
Eric Sauser, Aude Billard
Abstract:
This work investigates whether population vector coding could be a principle mechanism for sensorimotor transformations. This paper presents a formal demonstration of how population vector coding can proceed arbitrary 3-dimensional rotations and translations. The model suggests that population coding could be a possible mechanism for frames of reference transformations across multiple sensori-motor systems.
This work investigates whether population vector coding could be a principle mechanism for sensorimotor transformations. This paper presents a formal demonstration of how population vector coding can proceed arbitrary 3-dimensional rotations and translations. The model suggests that population coding could be a possible mechanism for frames of reference transformations across multiple sensori-motor systems.
ES2004-127
Using classification to determine the number of finger strokes on a multi-touch tactile device
Caspar von Wrede, Pavel Laskov
Using classification to determine the number of finger strokes on a multi-touch tactile device
Caspar von Wrede, Pavel Laskov
Abstract:
On certain types of multi-touch touchpads, determining the number of finger stroke is a non-trivial problem. We investigate the application of several classification algorithms to this problem. Our experiments are based on a flat prototype of the spherical Touchglobe touchpad. We demonstrate that with a very short delay after the stroke, the number of touches can determined by a Support Vector Machine with an RBF kernel with an accuracy of about 90% (on a 5-class problem).
On certain types of multi-touch touchpads, determining the number of finger stroke is a non-trivial problem. We investigate the application of several classification algorithms to this problem. Our experiments are based on a flat prototype of the spherical Touchglobe touchpad. We demonstrate that with a very short delay after the stroke, the number of touches can determined by a Support Vector Machine with an RBF kernel with an accuracy of about 90% (on a 5-class problem).
ES2004-133
Classification of Bioacoustic Time Series by Training a Decision Fusion mapping
Friedhelm Schwenker
Classification of Bioacoustic Time Series by Training a Decision Fusion mapping
Friedhelm Schwenker
Abstract:
The classification of time series is the topic of this paper. In particular we discuss the combination of local classifier decisions from several feature spaces with decision templates. averaging and decision templates. Decision templates are adaptive classifier fusion schemes calculated over a set of feature vectors extracted from through local time windows. The decison template approach is discussed in the context of neural network learning, and its behaviour is demonstrated to an application from the field of bioacoustics.
The classification of time series is the topic of this paper. In particular we discuss the combination of local classifier decisions from several feature spaces with decision templates. averaging and decision templates. Decision templates are adaptive classifier fusion schemes calculated over a set of feature vectors extracted from through local time windows. The decison template approach is discussed in the context of neural network learning, and its behaviour is demonstrated to an application from the field of bioacoustics.
ES2004-146
Spatial-Temporal artificial neurons applied to online cursive handwritten recognition
Rauf Baig
Spatial-Temporal artificial neurons applied to online cursive handwritten recognition
Rauf Baig
Abstract:
In this paper we present our latest experiments on the utilization of the recently developed Spatio-Temporal Artificial Neuron (STAN). This neuron has the capability to process asynchronous (continuous) spatio-temporal data sequences and compare them with the help of Hermitian distance. The problem addressed is that of online cursive (non-isolated) handwritten character recognition. We develop a system based on three modules: pre-processing, feature detection and character classification. The second and third modules are based on neural architectures, which have STANs as their neurons. The architecture and training of weights of the second module is based on a spatio-temporal adaptation of Kmeans algorithm and the third module is based on an adaptation of the RCE algorithm. The results obtained are encouraging and we also suggest further avenues of improvement in the system.
In this paper we present our latest experiments on the utilization of the recently developed Spatio-Temporal Artificial Neuron (STAN). This neuron has the capability to process asynchronous (continuous) spatio-temporal data sequences and compare them with the help of Hermitian distance. The problem addressed is that of online cursive (non-isolated) handwritten character recognition. We develop a system based on three modules: pre-processing, feature detection and character classification. The second and third modules are based on neural architectures, which have STANs as their neurons. The architecture and training of weights of the second module is based on a spatio-temporal adaptation of Kmeans algorithm and the third module is based on an adaptation of the RCE algorithm. The results obtained are encouraging and we also suggest further avenues of improvement in the system.
ES2004-42
Face Recognition Using Recurrent High-Order Associative Memories
Iulian Ciocoiu
Face Recognition Using Recurrent High-Order Associative Memories
Iulian Ciocoiu
Abstract:
A novel face recognition approach is proposed, based on the use of compressed discriminative features and recurrent neural classifiers. Low-dimensional feature vectors are extracted through a combined effect of wavelet decomposition and subspace projections. The classifier is implemented as a special gradient-type recurrent analog neural network acting as an associative memory. The system exhibits stable equilibrium points in predefined positions given by the feature vectors of the training set. Experimental results for the Olivetti database are reported, indicating improved performances over standard PCA and LDA-based face recognition approaches.
A novel face recognition approach is proposed, based on the use of compressed discriminative features and recurrent neural classifiers. Low-dimensional feature vectors are extracted through a combined effect of wavelet decomposition and subspace projections. The classifier is implemented as a special gradient-type recurrent analog neural network acting as an associative memory. The system exhibits stable equilibrium points in predefined positions given by the feature vectors of the training set. Experimental results for the Olivetti database are reported, indicating improved performances over standard PCA and LDA-based face recognition approaches.