Bruges, Belgium, April 26-27-28
Content of the proceedings
-
Data and signal analysis
Support Vector Machines
Model selection and evaluation
Artificial neural networks and robotics
ANN models and learning I
Non-linear dynamics and control
Neural networks in medicine
ANN models and learning II
Self-organizing maps for data analysis
Recurrent networks
Time series prediction
ANN models and learning III
Artificial neural networks for energy management systems
Learning in biological and artificial systems
Data and signal analysis
ES2000-16
A generative model for sparse discrete binary data with non-uniform categorical priors
M. Girolami
A generative model for sparse discrete binary data with non-uniform categorical priors
M. Girolami
Abstract:
The Generative Topographic Mapping (GTM) was developed and introduced as a principled alternative to the Self-Organising Map for, principally, visualising high dimensional continuous data. There are many cases where the observation data is ordinal and discrete and the application of methods developed specifically for continuous data is inappropriate. Based on the continuous GTM data model a non-linear latent variable model for modeling sparse high dimensional binary data is presented. The primary motivation for this work is the requirement for a dense and low dimensional representation of sparse binary vector space models of text documents based on the multivariate Bernoulli event model. The method is however applicable to binary data in general.
The Generative Topographic Mapping (GTM) was developed and introduced as a principled alternative to the Self-Organising Map for, principally, visualising high dimensional continuous data. There are many cases where the observation data is ordinal and discrete and the application of methods developed specifically for continuous data is inappropriate. Based on the continuous GTM data model a non-linear latent variable model for modeling sparse high dimensional binary data is presented. The primary motivation for this work is the requirement for a dense and low dimensional representation of sparse binary vector space models of text documents based on the multivariate Bernoulli event model. The method is however applicable to binary data in general.
ES2000-39
Using Growing hierarchical self-organizing maps for document classification
M. Dittenbach, D. Merkl, A. Rauber
Using Growing hierarchical self-organizing maps for document classification
M. Dittenbach, D. Merkl, A. Rauber
Abstract:
The self-organizing map has shown to be a stable neural network model for high-dimensional data analysis. However, its applicability is limited by the fact that some knowledge about the data is required to define the size of the network. In this paper we present the Growing Hierarchical SOM. This dynamically growing architecture evolves into a hierarchical structure of; self-organizing maps according to the characteristics of the input data. Furthermore, each map is expanded until it represents the correspondig subset of the data at a specific level of granularity. We demonstrate the benefits of this novel model using a real world example from the text classification domain.
The self-organizing map has shown to be a stable neural network model for high-dimensional data analysis. However, its applicability is limited by the fact that some knowledge about the data is required to define the size of the network. In this paper we present the Growing Hierarchical SOM. This dynamically growing architecture evolves into a hierarchical structure of; self-organizing maps according to the characteristics of the input data. Furthermore, each map is expanded until it represents the correspondig subset of the data at a specific level of granularity. We demonstrate the benefits of this novel model using a real world example from the text classification domain.
ES2000-35
A robust non-linear projection method
J. Lee, A. Lendasse, N. Donckers, M. Verleysen
A robust non-linear projection method
J. Lee, A. Lendasse, N. Donckers, M. Verleysen
Abstract:
This paper describes a new nonlinear projection method. The aim is to design a user-friendly method, tentatively as easy to use as the linear PCA (Principal Component Analysis). The method is based on CCA (Curvilinear Component Analysis). This paper presents two improvements with respect to the original CCA: a better behavior in the projection of highly nonlinear databases (like spirals) and a complete automation in the choice of the parameters value.
This paper describes a new nonlinear projection method. The aim is to design a user-friendly method, tentatively as easy to use as the linear PCA (Principal Component Analysis). The method is based on CCA (Curvilinear Component Analysis). This paper presents two improvements with respect to the original CCA: a better behavior in the projection of highly nonlinear databases (like spirals) and a complete automation in the choice of the parameters value.
ES2000-54
Parametric approach to blind deconvolution of nonlinear channels
J. Sole i Casals, A. Taleb, C. Jutten
Parametric approach to blind deconvolution of nonlinear channels
J. Sole i Casals, A. Taleb, C. Jutten
Abstract:
A parametric procedure for the blind inversion of nonlinear channels is proposed, based on a recent method of blind source separation in nonlinear mixtures. Experiments show that the proposed algorithms perform efficiently, even in the presence of hard distortion. The method, based on the minimisation of the output mutual information, needs the knowledge of log-derivative of input distribution (the so-called score function). Each algorithm consists of three adaptive blocks: one devoted to adaptive estimation of the score function, and two other blocks estimating the inverses of the linear and nonlinear parts of the channel, (quasi-)optimally adapted using the estimated score functions. This paper is mainly concerned by the nonlinear part, for which we propose two parametric models, the first based on a polynomial model and the second on a neural network, while [12, 13] proposed nonparametric approaches.
A parametric procedure for the blind inversion of nonlinear channels is proposed, based on a recent method of blind source separation in nonlinear mixtures. Experiments show that the proposed algorithms perform efficiently, even in the presence of hard distortion. The method, based on the minimisation of the output mutual information, needs the knowledge of log-derivative of input distribution (the so-called score function). Each algorithm consists of three adaptive blocks: one devoted to adaptive estimation of the score function, and two other blocks estimating the inverses of the linear and nonlinear parts of the channel, (quasi-)optimally adapted using the estimated score functions. This paper is mainly concerned by the nonlinear part, for which we propose two parametric models, the first based on a polynomial model and the second on a neural network, while [12, 13] proposed nonparametric approaches.
Support Vector Machines
ES2000-355
Algorithmic approaches to training Support Vector Machines: a survey
C. Campbell
Algorithmic approaches to training Support Vector Machines: a survey
C. Campbell
Abstract:
Support Vector Machines (SVMs) have become an increasingly popular tool for machine learning tasks involving classification, regression or novelty detection. They exhibit good generalisation performance on many real life datasets and the approach is wellmotivated theoretically. Training involves optimisation of a convex cost function, there are relatively few free parameters to adjust and the architecture does not have to be found by experimentation. In this tutorial we survey methods for training SVMs including model selection strategies for determining the free parameters and new techniques for active selection of training examples.
Support Vector Machines (SVMs) have become an increasingly popular tool for machine learning tasks involving classification, regression or novelty detection. They exhibit good generalisation performance on many real life datasets and the approach is wellmotivated theoretically. Training involves optimisation of a convex cost function, there are relatively few free parameters to adjust and the architecture does not have to be found by experimentation. In this tutorial we survey methods for training SVMs including model selection strategies for determining the free parameters and new techniques for active selection of training examples.
ES2000-352
Sparse least squares Support Vector Machine classifiers
J.A.K. Suykens, L. Lukas, J. Vandewalle
Sparse least squares Support Vector Machine classifiers
J.A.K. Suykens, L. Lukas, J. Vandewalle
Abstract:
In least squares support vector machine (LS-SVM) classifiers the original SVM formulation of Vapnik is modified by considering equality constraints within a form of ridge regression instead of inequality constraints. As a result the solution follows from solving a set of linear equations instead of a quadratic programming problem. However, a drawback is that sparseness is lost in the LS-SVM case due to the choice of 2-norms. In this paper we propose a method for imposing sparseness to the LS-SVM solution. This is done by pruning the support value spectrum which is revealing the relative importance of the training data points and is immediately available as solution to the linear systems.
In least squares support vector machine (LS-SVM) classifiers the original SVM formulation of Vapnik is modified by considering equality constraints within a form of ridge regression instead of inequality constraints. As a result the solution follows from solving a set of linear equations instead of a quadratic programming problem. However, a drawback is that sparseness is lost in the LS-SVM case due to the choice of 2-norms. In this paper we propose a method for imposing sparseness to the LS-SVM solution. This is done by pruning the support value spectrum which is revealing the relative importance of the training data points and is immediately available as solution to the linear systems.
ES2000-351
Support Vector Committee Machines
D. Martinez, G. Millerioux
Support Vector Committee Machines
D. Martinez, G. Millerioux
Abstract:
This paper proposes a mathematical programming framework for combining SVMs with possibly different kernels. Compared to single SVMs, the advantage of this approach is twofold: it creates SVMs with local domains of expertise leading to local enlargements of the margin, and it allows the use of simple linear kernels combined with a fixed boolean operation that is particularly well suited for building dedicated hardware.
This paper proposes a mathematical programming framework for combining SVMs with possibly different kernels. Compared to single SVMs, the advantage of this approach is twofold: it creates SVMs with local domains of expertise leading to local enlargements of the margin, and it allows the use of simple linear kernels combined with a fixed boolean operation that is particularly well suited for building dedicated hardware.
ES2000-354
Robust Bayes Point Machines
R. Herbich, T. Graepel, C. Campbell
Robust Bayes Point Machines
R. Herbich, T. Graepel, C. Campbell
Abstract:
Support Vector Machines choose the hypothesis corre- sponding to the centre of the largest hypersphere that can be inscribed in version space. If version space is elongated or irregularly shaped a potentially superior approach is take into account the whole of version space. We propose to construct the Bayes point which is approximated by the centre of mass. Our implementation of a Bayes Point Machine (BPM) uses an ergodic billiard to estimate this point in the kernel space. We show that BPMs outperform hard margin Support Vector Machines (SVMs) on real world datasets. We introduce a technique that allows the BPM to construct hypotheses with non{zero training error similar to soft margin SVMs with quadratic penelisation of the margin slacks. An experimental study reveals that with decreasing penalisation of train- ing error the improvement of BPMs over SVMs decays, a finding that is explained by geometrical considerations.
Support Vector Machines choose the hypothesis corre- sponding to the centre of the largest hypersphere that can be inscribed in version space. If version space is elongated or irregularly shaped a potentially superior approach is take into account the whole of version space. We propose to construct the Bayes point which is approximated by the centre of mass. Our implementation of a Bayes Point Machine (BPM) uses an ergodic billiard to estimate this point in the kernel space. We show that BPMs outperform hard margin Support Vector Machines (SVMs) on real world datasets. We introduce a technique that allows the BPM to construct hypotheses with non{zero training error similar to soft margin SVMs with quadratic penelisation of the margin slacks. An experimental study reveals that with decreasing penalisation of train- ing error the improvement of BPMs over SVMs decays, a finding that is explained by geometrical considerations.
Model selection and evaluation
ES2000-46
A statistical model selection strategy applied to neural networks
J. Pizarro, E. Guerrero, P. L. Galindo
A statistical model selection strategy applied to neural networks
J. Pizarro, E. Guerrero, P. L. Galindo
Abstract:
In statistical modelling, an investigator must often choose a suitable model among a collection of viable candidates. There is no consensus in the research community on how such a comparative study is performed in a methodologically sound way. The ranking of several methods is usually performed by the use of a selection criterion, which assigns a score to every model based on some underlying statistical principles. The fitted model that is favoured is the one corresponding to the minimum (or the maximum) score. Statistical significance testing can extend this method. However, when enough pairwise tests are performed the multiplicity effect appears which can be taken into account by considering multiple comparison procedures. The existing comparison procedures can roughly be categorized as analytical or resampling based. This paper describes a resampling based multiple comparison technique. This method is illustrated on the estimate of the number of hidden units for feed-forward neural networks
In statistical modelling, an investigator must often choose a suitable model among a collection of viable candidates. There is no consensus in the research community on how such a comparative study is performed in a methodologically sound way. The ranking of several methods is usually performed by the use of a selection criterion, which assigns a score to every model based on some underlying statistical principles. The fitted model that is favoured is the one corresponding to the minimum (or the maximum) score. Statistical significance testing can extend this method. However, when enough pairwise tests are performed the multiplicity effect appears which can be taken into account by considering multiple comparison procedures. The existing comparison procedures can roughly be categorized as analytical or resampling based. This paper describes a resampling based multiple comparison technique. This method is illustrated on the estimate of the number of hidden units for feed-forward neural networks
ES2000-56
Bootstrap for neural model selection
R. Kallel, M. Cottrell, V. Vigneron
Bootstrap for neural model selection
R. Kallel, M. Cottrell, V. Vigneron
Abstract:
Bootstrap techniques (also called resampling computations techniques) have introduced new advances in modeling and model evaluation. Using resampling methods, the information contained in one observed data set is extended to many typical generated data sets. These procedures based on computer simulation and cross validation are the last resort when no classical inference is possible due to the intrinsic complexity of the problem: they can avoid to estimate the noise distribution from the residuals, like in Monte-Carlo approach which is based on a hypothesized noise distribution. Resampling allows the modeler to construct a series of new samples which are based on the original data set, and then to estimate the stability of the parameters. Properties such as convergence and asymptotic normality can be checked for any particular observed data set. In most cases, the statistics computed on the generated data sets give a good idea of the confidence regions of the estimates. In this paper, we debate on the contribution of such methods for model selection, in the case of feedforward neural networks. The method is described and its effectiveness is checked through a number of examples.
Bootstrap techniques (also called resampling computations techniques) have introduced new advances in modeling and model evaluation. Using resampling methods, the information contained in one observed data set is extended to many typical generated data sets. These procedures based on computer simulation and cross validation are the last resort when no classical inference is possible due to the intrinsic complexity of the problem: they can avoid to estimate the noise distribution from the residuals, like in Monte-Carlo approach which is based on a hypothesized noise distribution. Resampling allows the modeler to construct a series of new samples which are based on the original data set, and then to estimate the stability of the parameters. Properties such as convergence and asymptotic normality can be checked for any particular observed data set. In most cases, the statistics computed on the generated data sets give a good idea of the confidence regions of the estimates. In this paper, we debate on the contribution of such methods for model selection, in the case of feedforward neural networks. The method is described and its effectiveness is checked through a number of examples.
ES2000-401
A new information criterion for the selection of subspace models
M. Sugiyama, H. Ogawa
A new information criterion for the selection of subspace models
M. Sugiyama, H. Ogawa
Abstract:
The problem of model selection is considerably important for acquiring higher levels of generalization capability in supervised learning. In this paper, we propose a new criterion for model selection named the subspace information criterion (SIC). Computer simulations show that SIC works well even when the number of training examples is small.
The problem of model selection is considerably important for acquiring higher levels of generalization capability in supervised learning. In this paper, we propose a new criterion for model selection named the subspace information criterion (SIC). Computer simulations show that SIC works well even when the number of training examples is small.
ES2000-27
Confidence estimation methods for neural networks : a practical comparison
G. Papadopoulos, P.J. Edwards, A.F. Murray
Confidence estimation methods for neural networks : a practical comparison
G. Papadopoulos, P.J. Edwards, A.F. Murray
Abstract:
Feed-forward neural networks (Multi-Layered Perceptrons) are used widely in real-world regression or classification tasks. A reliable and practical measure of prediction ``confidence'' is essential in real-world tasks. This paper compares three approaches to prediction confidence estimation, using both artificial and real data. The three methods are maximum likelihood, approximate Bayesian and bootstrap. Both noise inherent to the data and model uncertainty are considered.
Feed-forward neural networks (Multi-Layered Perceptrons) are used widely in real-world regression or classification tasks. A reliable and practical measure of prediction ``confidence'' is essential in real-world tasks. This paper compares three approaches to prediction confidence estimation, using both artificial and real data. The three methods are maximum likelihood, approximate Bayesian and bootstrap. Both noise inherent to the data and model uncertainty are considered.
Artificial neural networks and robotics
ES2000-303
Using higher order synapses and nodes to improve sensing capabilities of mobile robots
R.J. Duro, J. Santos, J.A. Becerra, F. Bellas, J.L. Crespo
Using higher order synapses and nodes to improve sensing capabilities of mobile robots
R.J. Duro, J. Santos, J.A. Becerra, F. Bellas, J.L. Crespo
Abstract:
In this paper we present three types of higher order artificial neural networks that may be included in heterogeneous ANN architectures to improve the perceptual performance of mobile robots. Two of the networks are based on synaptic processing, with the advantage that this type of processing works with the raw data and not an average, as is the case of nodes. The first one of the structures is designed for handling temporal relations using synaptic delays. The second one, through gaussian functions in the synapses, endows the networks with the capacity of recognizing particular objects in images independently of the background. By integrating these gaussian synapse networks in a global visual architecture, this detection becomes independent of position, orientation and scale. Finally, the third network presented is based on the use of persistence by means of the implementation of habituation neurons as input nodes of networks.
In this paper we present three types of higher order artificial neural networks that may be included in heterogeneous ANN architectures to improve the perceptual performance of mobile robots. Two of the networks are based on synaptic processing, with the advantage that this type of processing works with the raw data and not an average, as is the case of nodes. The first one of the structures is designed for handling temporal relations using synaptic delays. The second one, through gaussian functions in the synapses, endows the networks with the capacity of recognizing particular objects in images independently of the background. By integrating these gaussian synapse networks in a global visual architecture, this detection becomes independent of position, orientation and scale. Finally, the third network presented is based on the use of persistence by means of the implementation of habituation neurons as input nodes of networks.
ES2000-301
Competitive neural networks for robust computation of the optical flow
E. Fernandez, I. Echave, M. Grana
Competitive neural networks for robust computation of the optical flow
E. Fernandez, I. Echave, M. Grana
Abstract:
The Self Organizing Map and the Simple Competitive Learning are used to compute adaptively the vector quantizers of color image sequences. The codebook computed for each image in the sequence is then used as a smoothing filter, the VQ Bayesian Filter (VQ-BF), for the preprocessing of the images in the sequence. The optical flow is then robustly and efficiently computed over the filtered images applying a correlation method on the isolated pixels.
The Self Organizing Map and the Simple Competitive Learning are used to compute adaptively the vector quantizers of color image sequences. The codebook computed for each image in the sequence is then used as a smoothing filter, the VQ Bayesian Filter (VQ-BF), for the preprocessing of the images in the sequence. The optical flow is then robustly and efficiently computed over the filtered images applying a correlation method on the isolated pixels.
ES2000-302
Learning VOR-like stabilization reflexes in robots
F. Panerai, G. Metta, G. Sandini
Learning VOR-like stabilization reflexes in robots
F. Panerai, G. Metta, G. Sandini
Abstract:
We present a binocular robot that learns compensatory camera movements for image stabilization purposes. Most essential in achieving satisfactory image stabilization performance is the exploitation/integration of different self-motion information. In our robot, self-motion is measured inertially through an artificial vestibular apparatus and visually using basic motion detection algorithms. The first sensory system codes rotations and translations of the robot’s head, the second, the shift of the visual world across the image plane. An adaptive neural network learns to map these sensory signals to motor commands, transforming non homogeneous self-motion information into compensatory camera movements. We describe the network architecture, the convergence of the learning scheme and the performance of the stabilization reflex evaluated quantitatively by means of direct measurements on the image plane.
We present a binocular robot that learns compensatory camera movements for image stabilization purposes. Most essential in achieving satisfactory image stabilization performance is the exploitation/integration of different self-motion information. In our robot, self-motion is measured inertially through an artificial vestibular apparatus and visually using basic motion detection algorithms. The first sensory system codes rotations and translations of the robot’s head, the second, the shift of the visual world across the image plane. An adaptive neural network learns to map these sensory signals to motor commands, transforming non homogeneous self-motion information into compensatory camera movements. We describe the network architecture, the convergence of the learning scheme and the performance of the stabilization reflex evaluated quantitatively by means of direct measurements on the image plane.
ES2000-304
Learning of perceptual states in the design of an adaptive wall-following behavior
R. Iglesias, M. Fernandez-Delgado, S. Barro
Learning of perceptual states in the design of an adaptive wall-following behavior
R. Iglesias, M. Fernandez-Delgado, S. Barro
Abstract:
In this work we propose a new model that aims to overcome some of the limitations that are associated with reinforcement learning. In order to do so, we include not only prior knowledge of the task to be undertaken, by means of the so-called Supervised Reinforcement Learning (SRL), but also the creation and adaptation of the perceptual states of the environment during the learning process itself, by means of the MART neural network.We have tested the application of this model on a wall following behaviour. The results obtained confirm the great usefulness and advantages that are derived from its employment.
In this work we propose a new model that aims to overcome some of the limitations that are associated with reinforcement learning. In order to do so, we include not only prior knowledge of the task to be undertaken, by means of the so-called Supervised Reinforcement Learning (SRL), but also the creation and adaptation of the perceptual states of the environment during the learning process itself, by means of the MART neural network.We have tested the application of this model on a wall following behaviour. The results obtained confirm the great usefulness and advantages that are derived from its employment.
ANN models and learning I
ES2000-3
Fuzzy entropy-constrained competitive learning algorithm
W.-J. Hwang, C. Ou, S.-C. Liao, C.-F. Chine
Fuzzy entropy-constrained competitive learning algorithm
W.-J. Hwang, C. Ou, S.-C. Liao, C.-F. Chine
Abstract:
A novel variable-rate vector quantizer (VQ) design algorithm using both fuzzy and competitive learning technique is presented. The algorithm enjoys better rate-distortion performance than that of other existing fuzzy clustering and competitive learning algorithms. In addition, the learning algorithm is less senstive to the selection of initial reproduction vectors. Therefore,the algorithm can be an effective alternative to the existing variable-rate VQ algorithm for signal compression.
A novel variable-rate vector quantizer (VQ) design algorithm using both fuzzy and competitive learning technique is presented. The algorithm enjoys better rate-distortion performance than that of other existing fuzzy clustering and competitive learning algorithms. In addition, the learning algorithm is less senstive to the selection of initial reproduction vectors. Therefore,the algorithm can be an effective alternative to the existing variable-rate VQ algorithm for signal compression.
ES2000-11
Specification, estimation and evaluation of single hidden-layer feedforward autoregressive artificial neural network models
G. Rech
Specification, estimation and evaluation of single hidden-layer feedforward autoregressive artificial neural network models
G. Rech
Abstract:
This paper considers artificial neural network modelling from a statistical point of view. Specification, estimation and evaluation are carried out using Lagrange multiplier testing. Simulations in samples of moderate size demonstrate the performances of the overall procedure.
This paper considers artificial neural network modelling from a statistical point of view. Specification, estimation and evaluation are carried out using Lagrange multiplier testing. Simulations in samples of moderate size demonstrate the performances of the overall procedure.
ES2000-18
A neural network for undercomplete independent component analysis
L. Wei, J. C. Rajapakse
A neural network for undercomplete independent component analysis
L. Wei, J. C. Rajapakse
Abstract:
The existing independent component neural networks (ICNNs) in the literature need same number of output neurons as the input nodes to achieve independence among output activations. We present a technique to learn the undercomplete ICNNs to produce an output with an lower dimension than the input by using joint entropy of a multidimensional Gaussian to approximate the mutual entropy of the output. Our approach is not restricted by the squared Jacobian matrix of outputs with respect to the inputs, and gives a general rule and some criteria to extract both super- and sub-Gaussianly distributed signals and remove the Gaussian distributed noise. Simulation results with simulated signals and audio signals are provided.
The existing independent component neural networks (ICNNs) in the literature need same number of output neurons as the input nodes to achieve independence among output activations. We present a technique to learn the undercomplete ICNNs to produce an output with an lower dimension than the input by using joint entropy of a multidimensional Gaussian to approximate the mutual entropy of the output. Our approach is not restricted by the squared Jacobian matrix of outputs with respect to the inputs, and gives a general rule and some criteria to extract both super- and sub-Gaussianly distributed signals and remove the Gaussian distributed noise. Simulation results with simulated signals and audio signals are provided.
ES2000-20
Distributed clustering and local regression for knowledge discovery in multiple spatial databases
A. Lazarevic, D. Pokrajac, Z. Obradovic
Distributed clustering and local regression for knowledge discovery in multiple spatial databases
A. Lazarevic, D. Pokrajac, Z. Obradovic
Abstract:
Many large-scale spatial data analysis problems involve an investigation of relationships in heterogeneous databases. In such situations, instead of making predictions uniformly across entire spatial data sets, in a previous study we used clustering for identifying similar spatial regions and then constructed local regression models describing the relationship between data characteristics and the target value inside each cluster. This approach requires all the data to be resident on a central machine, and it is not applicable when a large volume of spatial data is distributed at multiple sites. Here, a novel distributed method for learning from heterogeneous spatial databases is proposed. Similar regions in multiple databases are identified by independently applying a spatial clustering algorithm on all sites, followed by transferring convex hulls corresponding to identified clusters and their integration. For each discovered region, the local regression models are built and transferred among data sites. The proposed method is shown to be computationally efficient and fairly accurate when compared to an approach where all the data are available at a central location.
Many large-scale spatial data analysis problems involve an investigation of relationships in heterogeneous databases. In such situations, instead of making predictions uniformly across entire spatial data sets, in a previous study we used clustering for identifying similar spatial regions and then constructed local regression models describing the relationship between data characteristics and the target value inside each cluster. This approach requires all the data to be resident on a central machine, and it is not applicable when a large volume of spatial data is distributed at multiple sites. Here, a novel distributed method for learning from heterogeneous spatial databases is proposed. Similar regions in multiple databases are identified by independently applying a spatial clustering algorithm on all sites, followed by transferring convex hulls corresponding to identified clusters and their integration. For each discovered region, the local regression models are built and transferred among data sites. The proposed method is shown to be computationally efficient and fairly accurate when compared to an approach where all the data are available at a central location.
ES2000-23
Nonlinear, statistical data-analysis for the optimal construction of neural-network inputs with the concept of a mutual information
F. Heister, G. Schock
Nonlinear, statistical data-analysis for the optimal construction of neural-network inputs with the concept of a mutual information
F. Heister, G. Schock
Abstract:
In this article we focus on a statistical method for nonlinear time series analysis of data-sets used in supervised neural network training. A new method for identifying a minimal neural input-vector with maximum information content is proposed. Further, we demonstrate the capability of the mutual information for nonlinear time series analysis of real measurement data. From the viewpoint of information theory this approach provides optimal solutions for a large variety of problems.
In this article we focus on a statistical method for nonlinear time series analysis of data-sets used in supervised neural network training. A new method for identifying a minimal neural input-vector with maximum information content is proposed. Further, we demonstrate the capability of the mutual information for nonlinear time series analysis of real measurement data. From the viewpoint of information theory this approach provides optimal solutions for a large variety of problems.
ES2000-48
Influence of weight-decay training in input selection methods
M. Fernandez-Redondo, C. Hernandez-Espinosa
Influence of weight-decay training in input selection methods
M. Fernandez-Redondo, C. Hernandez-Espinosa
Abstract:
We describe the results of a research on the effect of weight-decay (WD) in input selection methods based on the analysis of a trained multilayer feedforward network. It was proposed by some authors to train the network with WD before applying this type of methods. The influence of WD in sixteen different input selection methods is empirically analyzed with a total of seven classification problems. We show that the performance variation of the input selection methods by introducing WD depends on the particular method. But for some of them, the use of WD can deteriorate their efficiency. Furthermore, it seems that WD improves the efficiency of the worst methods and deteriorates the performance of the best ones. In that sense, it diminishes the differences among different methods. We think that the use of weight-decay with this type of input selection methods should be avoided because the results are not good and also the use of weight-decay supposes a complication of the procedure.
We describe the results of a research on the effect of weight-decay (WD) in input selection methods based on the analysis of a trained multilayer feedforward network. It was proposed by some authors to train the network with WD before applying this type of methods. The influence of WD in sixteen different input selection methods is empirically analyzed with a total of seven classification problems. We show that the performance variation of the input selection methods by introducing WD depends on the particular method. But for some of them, the use of WD can deteriorate their efficiency. Furthermore, it seems that WD improves the efficiency of the worst methods and deteriorates the performance of the best ones. In that sense, it diminishes the differences among different methods. We think that the use of weight-decay with this type of input selection methods should be avoided because the results are not good and also the use of weight-decay supposes a complication of the procedure.
ES2000-8
Committee formation for reliable and accurate neural prediction in industry
P.J. Edwards, A.F. Murray
Committee formation for reliable and accurate neural prediction in industry
P.J. Edwards, A.F. Murray
Abstract:
This paper describes "cranking", a new committee formation algorithm. Cranking results in accurate and reliable committee predictions, even when applied to complex industrial tasks. Prediction error estimates are used to rank a pool of models trained on bootstrap data samples. The best are then used to form a committee. This paper presents a comparison of prediction error estimates that may be used for the ranking process. In addition, it shows how the influence of poor models, due to training being unreliable,; may be minimised. Experiments are carried out on an artificial task, and a real-world, decision-support task taken from the papermaking industry. In summary, this paper studies committee formation for accurate and reliable neural prediction in industrial tasks.
This paper describes "cranking", a new committee formation algorithm. Cranking results in accurate and reliable committee predictions, even when applied to complex industrial tasks. Prediction error estimates are used to rank a pool of models trained on bootstrap data samples. The best are then used to form a committee. This paper presents a comparison of prediction error estimates that may be used for the ranking process. In addition, it shows how the influence of poor models, due to training being unreliable,; may be minimised. Experiments are carried out on an artificial task, and a real-world, decision-support task taken from the papermaking industry. In summary, this paper studies committee formation for accurate and reliable neural prediction in industrial tasks.
Non-linear dynamics and control
ES2000-6
Toward encryption with neural network analogy
T. Ohira
Toward encryption with neural network analogy
T. Ohira
Abstract:
We propose here a new model of encryption of binary data taking advantage of the complexity of dynamics of a model motivated by a neural network with delays. In this scheme, the encryption process is a coupling dynamics with various time delays between different bits (or neurons) in the original data. It is observed that decoding of the encrypted data may be extremely difficult without a complete knowledge of the coupling manner with associated delays for all bits of the data.
We propose here a new model of encryption of binary data taking advantage of the complexity of dynamics of a model motivated by a neural network with delays. In this scheme, the encryption process is a coupling dynamics with various time delays between different bits (or neurons) in the original data. It is observed that decoding of the encrypted data may be extremely difficult without a complete knowledge of the coupling manner with associated delays for all bits of the data.
ES2000-2
Iterative learning neural network control for nonlinear system trajectory tracking
P. Jiang, R. Unbehauen
Iterative learning neural network control for nonlinear system trajectory tracking
P. Jiang, R. Unbehauen
Abstract:
This paper presents a neural network controller for nonlinear system trajectory tracking, which works in an iterative learning manner. The controller is composed of many local neural networks and every point along the desired trajectory has its own one for approximating nonlinearity only nearby. This makes that every local neural network can be possessed of a simple structure and less neurons. Because the neural networks are independent from each other, the whole trajectory training can be divided into several segments training, where we train a segment repetitively and extend the trained segment step by step. Stability of the controller is ensured.
This paper presents a neural network controller for nonlinear system trajectory tracking, which works in an iterative learning manner. The controller is composed of many local neural networks and every point along the desired trajectory has its own one for approximating nonlinearity only nearby. This makes that every local neural network can be possessed of a simple structure and less neurons. Because the neural networks are independent from each other, the whole trajectory training can be divided into several segments training, where we train a segment repetitively and extend the trained segment step by step. Stability of the controller is ensured.
ES2000-45
A comparative design of a MIMO neural adaptive rate damping for a nonlinear helicopter model
P. A. Gili, M. Battipede
A comparative design of a MIMO neural adaptive rate damping for a nonlinear helicopter model
P. A. Gili, M. Battipede
Abstract:
Using a nonlinear 15-state helicopter model in 6 DOF, two different neural control systems, both acting as rate damping, have been designed and compared. They are both based on the reference model direct inverse scheme, but they differ each from the other for the identification of the inverse model: the first one is a MIMO feedforward two-layered neural network, while the; second one is a combination of three MISO feedforward two-layered neural networks connected in parallel. The strong dynamic cross-coupling, that characterizes the model, has enabled us to verify the actual MIMO capability of both the neural rate damping configurations. However the multi-MISO version has demonstrated to have a more robust adaptive ability.
Using a nonlinear 15-state helicopter model in 6 DOF, two different neural control systems, both acting as rate damping, have been designed and compared. They are both based on the reference model direct inverse scheme, but they differ each from the other for the identification of the inverse model: the first one is a MIMO feedforward two-layered neural network, while the; second one is a combination of three MISO feedforward two-layered neural networks connected in parallel. The strong dynamic cross-coupling, that characterizes the model, has enabled us to verify the actual MIMO capability of both the neural rate damping configurations. However the multi-MISO version has demonstrated to have a more robust adaptive ability.
Neural networks in medicine
ES2000-457
Neural networks approaches in medicine - a review of actual developments
T. Villmann
Neural networks approaches in medicine - a review of actual developments
T. Villmann
Abstract:
The utilization of neural network approaches have been rapidly increasing also in the area of medicine. Thereby both biomedical modelling as well as data analysis are objects of neural network applications. On the one hand side neural networks give the possibility for understanding of brain and cognitition processes. On the other hand the power of neural networks for data analysis, data visualization and knowledge discovery is used. In the following we will reflect some of the actual developments and problems when neural networks are applied in medical area.
The utilization of neural network approaches have been rapidly increasing also in the area of medicine. Thereby both biomedical modelling as well as data analysis are objects of neural network applications. On the one hand side neural networks give the possibility for understanding of brain and cognitition processes. On the other hand the power of neural networks for data analysis, data visualization and knowledge discovery is used. In the following we will reflect some of the actual developments and problems when neural networks are applied in medical area.
ES2000-452
A neural network architecture for automatic segmentation of fluorescence micrographs
T. Nattkemper, H. Wersing, W. Schubert, H. Ritter
A neural network architecture for automatic segmentation of fluorescence micrographs
T. Nattkemper, H. Wersing, W. Schubert, H. Ritter
Abstract:
A system for the automatic segmentation of fluorescence micrographs is presented. In a first step positions of fluorescent cells are detected by a fast learning neural network, which acquires the visual knowledge from a set of training cell-image patches selected by the user. Guided by the detected cell positions the system extracts in the second step the contours of the cells. For contour extraction a recurrent neural network model is used to approximate the cell shapes. Even though the micrographs are noisy and the fluorescent cells vary in shape and size, the system detects at minimum 95% of the cells.
A system for the automatic segmentation of fluorescence micrographs is presented. In a first step positions of fluorescent cells are detected by a fast learning neural network, which acquires the visual knowledge from a set of training cell-image patches selected by the user. Guided by the detected cell positions the system extracts in the second step the contours of the cells. For contour extraction a recurrent neural network model is used to approximate the cell shapes. Even though the micrographs are noisy and the fluorescent cells vary in shape and size, the system detects at minimum 95% of the cells.
ES2000-451
Boundary based movement correction of functional MR data using a genetic algorithm
G. Bao, J. C. Rajapakse
Boundary based movement correction of functional MR data using a genetic algorithm
G. Bao, J. C. Rajapakse
Abstract:
This paper describes a novel image registration method for movement correction of fMR time-series. It is important to align the fMR images in the time-series before time-dependent analyses. This registration method aligns the boundaries of brains extracted from the functional images. It uses a genetic algorithm to minimize the distance function obtained from the chamfer distance transform. The global search nature of genetic algorithm makes this method robust to the presence of local minima.
This paper describes a novel image registration method for movement correction of fMR time-series. It is important to align the fMR images in the time-series before time-dependent analyses. This registration method aligns the boundaries of brains extracted from the functional images. It uses a genetic algorithm to minimize the distance function obtained from the chamfer distance transform. The global search nature of genetic algorithm makes this method robust to the presence of local minima.
ES2000-455
A neural network approach to adaptive pattern analysis - the deformable feature map
A. Wismüller, F. Vietze, D.R. Dersch, K. Hahn, H. Ritter
A neural network approach to adaptive pattern analysis - the deformable feature map
A. Wismüller, F. Vietze, D.R. Dersch, K. Hahn, H. Ritter
Abstract:
In this paper, we present an algorithm that provides adaptive plasticity in function approximation problems: the deformable (feature) map (DM) algorithm. The DM approach reduces a class of similar function approximation problems to the explicit supervised one-shot training of a single data set. This is followed by a subsequent, appropriate similarity transformation which is based on a self-organized deformation of the underlying multidimensional probability distributions. After discussing the theory of the DM algorithm, we use a computer simulation to visualize its effects on a two-dimensional toy example. Finally, we present results of its application to the real-world problem of fully automatic voxel-based multispectral image segmentation, employing magnetic resonance data sets of the human brain.
In this paper, we present an algorithm that provides adaptive plasticity in function approximation problems: the deformable (feature) map (DM) algorithm. The DM approach reduces a class of similar function approximation problems to the explicit supervised one-shot training of a single data set. This is followed by a subsequent, appropriate similarity transformation which is based on a self-organized deformation of the underlying multidimensional probability distributions. After discussing the theory of the DM algorithm, we use a computer simulation to visualize its effects on a two-dimensional toy example. Finally, we present results of its application to the real-world problem of fully automatic voxel-based multispectral image segmentation, employing magnetic resonance data sets of the human brain.
ES2000-453
Automatic detection of clustered microcalcifications in digital mammograms using an SVM classifier
A. Bazzani, A. Bevilacqua, D. Bollini, R. Brancaccio, R. Carnpanini, N. Lanconelli, A. Riccardi, D. Romani, G. Zamboni
Automatic detection of clustered microcalcifications in digital mammograms using an SVM classifier
A. Bazzani, A. Bevilacqua, D. Bollini, R. Brancaccio, R. Carnpanini, N. Lanconelli, A. Riccardi, D. Romani, G. Zamboni
Abstract:
In this paper we investigate the performance of a Computer Aided Diagnosis (CAD) system for the detection of clustered microcalcifications in mammograms. Our detection algorithm consists on the combination of two different methods. The first one, based on difference-image techniques and gaussianity statistical tests, finds out the most obvious signals. The second one is able to discover more subtle microcalcifications by exploiting a multiresolution analysis by means of the wavelet transform. In the false-positive reduction step we separate false signals from microcalcifications by means of an SVM classifier. Our algorithm yields a sensitivity of 94.6% with 0.6 false positive cluster per image on the 40 images of the Nijmegen database.
In this paper we investigate the performance of a Computer Aided Diagnosis (CAD) system for the detection of clustered microcalcifications in mammograms. Our detection algorithm consists on the combination of two different methods. The first one, based on difference-image techniques and gaussianity statistical tests, finds out the most obvious signals. The second one is able to discover more subtle microcalcifications by exploiting a multiresolution analysis by means of the wavelet transform. In the false-positive reduction step we separate false signals from microcalcifications by means of an SVM classifier. Our algorithm yields a sensitivity of 94.6% with 0.6 false positive cluster per image on the 40 images of the Nijmegen database.
ES2000-454
A neuro-fuzzy approach as medical diagnostic interface
R. Brause, F. Friedrich
A neuro-fuzzy approach as medical diagnostic interface
R. Brause, F. Friedrich
Abstract:
In contrast to the symbolic approach, neural networks seldom are designed to explain what they have learned. This is a major obstacle for its use in everyday life. With the appearance of neuro-fuzzy systems which use vague, human-like categories the situation has changed. Based on the well-known mechanisms of learning for RBF networks, a special neuro-fuzzy interface is proposed in this paper. It is especially useful in medical applications, using the notation and habits of physicians and other medically trained people. As an example, a liver disease diagnosis system is presented.
In contrast to the symbolic approach, neural networks seldom are designed to explain what they have learned. This is a major obstacle for its use in everyday life. With the appearance of neuro-fuzzy systems which use vague, human-like categories the situation has changed. Based on the well-known mechanisms of learning for RBF networks, a special neuro-fuzzy interface is proposed in this paper. It is especially useful in medical applications, using the notation and habits of physicians and other medically trained people. As an example, a liver disease diagnosis system is presented.
ANN models and learning II
ES2000-9
Regularization in oculomotor control
J. A. Bullinaria, P. M. Riddell
Regularization in oculomotor control
J. A. Bullinaria, P. M. Riddell
Abstract:
In modelling the development of the oculomotor control system using neural networks, it is important to determine the appropriate cost function on which to train the models. Whilst blur and disparity are fairly obvious error components, choosing the regularization component is less easy. In this paper we explore the consequences of a number of the most reasonable possibilities and investigate the extent to which other factors may dominate their influence.
In modelling the development of the oculomotor control system using neural networks, it is important to determine the appropriate cost function on which to train the models. Whilst blur and disparity are fairly obvious error components, choosing the regularization component is less easy. In this paper we explore the consequences of a number of the most reasonable possibilities and investigate the extent to which other factors may dominate their influence.
ES2000-12
Limitations of hybrid systems
B. Hammer
Limitations of hybrid systems
B. Hammer
Abstract:
We examine the ability of combining symbolic and subsymbolic approaches by means of recursively;encoding and decoding structured data. We show;that encoding of symbolic data is possible in this;way - hence neural networks seem well suited for;control or classification in symbolic approaches - whereas decoding requires an increasing complexity of the decoding function - hence networks with this;dynamics are not adequate for producing structured data. Real labeled tree structures reject a smooth encoding in general.
We examine the ability of combining symbolic and subsymbolic approaches by means of recursively;encoding and decoding structured data. We show;that encoding of symbolic data is possible in this;way - hence neural networks seem well suited for;control or classification in symbolic approaches - whereas decoding requires an increasing complexity of the decoding function - hence networks with this;dynamics are not adequate for producing structured data. Real labeled tree structures reject a smooth encoding in general.
ES2000-13
Discriminative learning for neural decision feedback equalizers
E.D. Di Claudio, R. Parisi, G. Orlandi
Discriminative learning for neural decision feedback equalizers
E.D. Di Claudio, R. Parisi, G. Orlandi
Abstract:
In this work new Decision-Feedback (DF) Neural Equalizers (DFNE) are introduced and compared with classical DF equalizers and Viterbi demodulators. It is shown that the choice of an innovative cost functional based on the Discriminative Learning (DL) technique, coupled with a fast training paradigm, can provide neural equalizers that outperform standard DF equalizers (DFEs) at practical signal to noise ratio (SNR). In particular, the novel Neural Sequence Detector (NSD) is introduced, which allows to extend the concepts of Viterbi-like sequence estimation to neural architectures. Resulting architectures are competitive with the Viterbi solution from cost-performance aspects, as demonstrated in experimental tests.
In this work new Decision-Feedback (DF) Neural Equalizers (DFNE) are introduced and compared with classical DF equalizers and Viterbi demodulators. It is shown that the choice of an innovative cost functional based on the Discriminative Learning (DL) technique, coupled with a fast training paradigm, can provide neural equalizers that outperform standard DF equalizers (DFEs) at practical signal to noise ratio (SNR). In particular, the novel Neural Sequence Detector (NSD) is introduced, which allows to extend the concepts of Viterbi-like sequence estimation to neural architectures. Resulting architectures are competitive with the Viterbi solution from cost-performance aspects, as demonstrated in experimental tests.
ES2000-52
Neurocontrol of a binary distillation column
M.A. Torres, M.E. Pardo, J.M. Pupo, L. Boquete, R. Barea, L.M. Bergasa
Neurocontrol of a binary distillation column
M.A. Torres, M.E. Pardo, J.M. Pupo, L. Boquete, R. Barea, L.M. Bergasa
Abstract:
This paper deals with the control of a methanol-water distillation column using neural networks, setting up a multiloop system. The neural network used is a multi-layer perceptron type, trained off line by a gradient descent algorithm. Results show an improvement on the use of an algorithm based on a classic controller such as PID
This paper deals with the control of a methanol-water distillation column using neural networks, setting up a multiloop system. The neural network used is a multi-layer perceptron type, trained off line by a gradient descent algorithm. Results show an improvement on the use of an algorithm based on a classic controller such as PID
ES2000-53
E.O.G. guidance of a weelchair using spiking neural networks
R. Barea, L. Boquete, M. Mazo, E. Lopez, L.M. Bergasa
E.O.G. guidance of a weelchair using spiking neural networks
R. Barea, L. Boquete, M. Mazo, E. Lopez, L.M. Bergasa
Abstract:
In this paper we present a new architecture of spiking neural networks (SNNs) to control the movements of a wheelchair. In this case, to send different commands we have used electrooculography (EOG) techniques, so that, control is made by means of the ocular position (eye displacement into its orbit). Spatio-temporal coding that combines spatial constraints with temporal sequencing is of great interest to visual-like circuit model. Therefore, a neural network (SNN) is used to identify the eye model, therefore the saccadic eye movements can be detected and know where user is looking at. The system consists of a standard electric wheelchair with an on-board computer, sensors and graphical user interface running on a computer.
In this paper we present a new architecture of spiking neural networks (SNNs) to control the movements of a wheelchair. In this case, to send different commands we have used electrooculography (EOG) techniques, so that, control is made by means of the ocular position (eye displacement into its orbit). Spatio-temporal coding that combines spatial constraints with temporal sequencing is of great interest to visual-like circuit model. Therefore, a neural network (SNN) is used to identify the eye model, therefore the saccadic eye movements can be detected and know where user is looking at. The system consists of a standard electric wheelchair with an on-board computer, sensors and graphical user interface running on a computer.
Self-organizing maps for data analysis
ES2000-205
Self-Organizing Maps in data analysis - notes on overfitting and overinterpretation
J. Lampinen, T. Kostiainen
Self-Organizing Maps in data analysis - notes on overfitting and overinterpretation
J. Lampinen, T. Kostiainen
Abstract:
The Self-Organizing Map, SOM, is a widely used tool in exploratory data analysis. Visual inspection of the SOM can be used to list potential dependencies between variables, that are then validated with more principled statistical methods. In this paper we discuss the use of the SOM in searching for dependencies in the data. We point out that simple use of the SOM may lead to excessive number of false hypotheses. We formulate the exact probability density model for which the SOM training gives the Maximum Likelihood estimate and show how the model parameters (neighborhood and kernel width) can be chosen to avoid overfitting. The conditional distributions from the true density model offer a consistent way to quantify and test the dependencies between variables.
The Self-Organizing Map, SOM, is a widely used tool in exploratory data analysis. Visual inspection of the SOM can be used to list potential dependencies between variables, that are then validated with more principled statistical methods. In this paper we discuss the use of the SOM in searching for dependencies in the data. We point out that simple use of the SOM may lead to excessive number of false hypotheses. We formulate the exact probability density model for which the SOM training gives the Maximum Likelihood estimate and show how the model parameters (neighborhood and kernel width) can be chosen to avoid overfitting. The conditional distributions from the true density model offer a consistent way to quantify and test the dependencies between variables.
ES2000-204
Bootstrapping Self-Organizing Maps to assess the statistical significance of local proximity
E. de Bodt, M. Cottrell
Bootstrapping Self-Organizing Maps to assess the statistical significance of local proximity
E. de Bodt, M. Cottrell
Abstract:
One of the attractive features of Self-Organising Maps (SOM) is the so-called “topological preservation property”: observations that are close to each other in the input space (at least locally) remain close to each other in the SOM. In this work, we propose the use of a bootstrap scheme to construct a statistical significance test of the observed proximity among individuals in the SOM. While computer intensive at this stage, this work represents a first step in the exploration of the sampling distribution of proximities in the framework of the SOM algorithm.
One of the attractive features of Self-Organising Maps (SOM) is the so-called “topological preservation property”: observations that are close to each other in the input space (at least locally) remain close to each other in the SOM. In this work, we propose the use of a bootstrap scheme to construct a statistical significance test of the observed proximity among individuals in the SOM. While computer intensive at this stage, this work represents a first step in the exploration of the sampling distribution of proximities in the framework of the SOM algorithm.
ES2000-201
Evaluating SOMs using order metrics
A. P. Azcarraga
Evaluating SOMs using order metrics
A. P. Azcarraga
Abstract:
It has been shown that self-organized maps, when adequately trained with the set of integers 1 to 32, lay out real numbers in a 2D map in an ordering that is superior to any of the known 2D orderings, such as the Cantor-diagonal, Morton, Peano-Hilbert, raster-scan, row-prime, spiral, and random orderings. Two 2D order metrics (Average Direct Neighbor Distance and Average Unit Disorder) have been used to assess the quality of a map's 2D ordering. It is shown here that these same order metrics are useful in assessing the quality of the self-organization process itself. Based on these metrics, it can be determined whether the SOM has already adequately learned and whether the parameters used to train the SOM have been correctly specified. In applications like data analysis, where there is little clue as to the way the SOM is supposed to look like after training, it is important to be able to able to assess the quality of the self-organization process independent of the application.
It has been shown that self-organized maps, when adequately trained with the set of integers 1 to 32, lay out real numbers in a 2D map in an ordering that is superior to any of the known 2D orderings, such as the Cantor-diagonal, Morton, Peano-Hilbert, raster-scan, row-prime, spiral, and random orderings. Two 2D order metrics (Average Direct Neighbor Distance and Average Unit Disorder) have been used to assess the quality of a map's 2D ordering. It is shown here that these same order metrics are useful in assessing the quality of the self-organization process itself. Based on these metrics, it can be determined whether the SOM has already adequately learned and whether the parameters used to train the SOM have been correctly specified. In applications like data analysis, where there is little clue as to the way the SOM is supposed to look like after training, it is important to be able to able to assess the quality of the self-organization process independent of the application.
ES2000-202
Self-Organisation in the SOM with a finite number of possible inputs
J.A. Flanagan
Self-Organisation in the SOM with a finite number of possible inputs
J.A. Flanagan
Abstract:
Given a one dimensional SOM with a monotonically decreasing neighbourhood and an input distribution which is not Lebesque continuous, a set of sufficient conditions and a Theorem are stated which ensure probability one organisation of the neuron weights. This leads to a rule for choosing the number of neurons and width of the neighbourhood to improve the chances of reaching an organised state in a practical implementation of the SOM.
Given a one dimensional SOM with a monotonically decreasing neighbourhood and an input distribution which is not Lebesque continuous, a set of sufficient conditions and a Theorem are stated which ensure probability one organisation of the neuron weights. This leads to a rule for choosing the number of neurons and width of the neighbourhood to improve the chances of reaching an organised state in a practical implementation of the SOM.
ES2000-203
Topological map for binary data
M. Lebbah, F. Badran, S. Thiria
Topological map for binary data
M. Lebbah, F. Badran, S. Thiria
Abstract:
We propose a new algorithm using topological map on binary data. The usual Euclidean distance is replaced by binary distance measures, which take into account possible asymmetries of binary data. The method is illustrated on an example taken from literature. Finally an application from chemistry is presented. We show the efficiency of the proposed method when applied to high-dimensinal binary data.
We propose a new algorithm using topological map on binary data. The usual Euclidean distance is replaced by binary distance measures, which take into account possible asymmetries of binary data. The method is illustrated on an example taken from literature. Finally an application from chemistry is presented. We show the efficiency of the proposed method when applied to high-dimensinal binary data.
ES2000-206
Analytical comparison of the Temporal Kohonen Map and the Recurrent Self Organizing Map
M. Varsta, J. Heikkonen, J. Lampinen
Analytical comparison of the Temporal Kohonen Map and the Recurrent Self Organizing Map
M. Varsta, J. Heikkonen, J. Lampinen
Abstract:
The basic SOM is indifferent to the ordering of the input patterns. Real data, however, is often sequential in nature thus context of a pattern may significantly influence its correct interpretation. One simple SOM model that takes the context of a pattern into account is the Temporal Kohonen Map (TKM), which was modified into the Recurrent Self Organizing Map (RSOM). We show analytically and with experiments that the RSOM is a significant improvement over the TKM because the RSOM model allows simple derivation of a consistent update rule.
The basic SOM is indifferent to the ordering of the input patterns. Real data, however, is often sequential in nature thus context of a pattern may significantly influence its correct interpretation. One simple SOM model that takes the context of a pattern into account is the Temporal Kohonen Map (TKM), which was modified into the Recurrent Self Organizing Map (RSOM). We show analytically and with experiments that the RSOM is a significant improvement over the TKM because the RSOM model allows simple derivation of a consistent update rule.
Recurrent networks
ES2000-33
Local input-output stability of recurrent networks with time-varying weights
J.J. Steil
Local input-output stability of recurrent networks with time-varying weights
J.J. Steil
Abstract:
We present local conditions for input-output stability of recurrent neural networks with time-varying parameters introduced for instance by noise or on-line adaptation. The conditions guarantee that a network implements a proper mapping from time-varying input to time-varying output functions using a local equilibrium as point of operation. We show how to calculate necessary bounds on the allowed inputs to keep the network in the stable range and apply the method to an example of learning an input-output map implied by the chaotic Roessler attractor.
We present local conditions for input-output stability of recurrent neural networks with time-varying parameters introduced for instance by noise or on-line adaptation. The conditions guarantee that a network implements a proper mapping from time-varying input to time-varying output functions using a local equilibrium as point of operation. We show how to calculate necessary bounds on the allowed inputs to keep the network in the stable range and apply the method to an example of learning an input-output map implied by the chaotic Roessler attractor.
ES2000-36
An optimization neural network model with time-dependent and lossy dynamics
Z. Heszberger, J. Biro, E. Halasz
An optimization neural network model with time-dependent and lossy dynamics
Z. Heszberger, J. Biro, E. Halasz
Abstract:
The paper deals with continuously operating optimization neural networks with lossy dynamics. As the main feature of the neural model time-varying nature of neuron activation functions is introduced. The model presented is general in the sense that it covers the cases of neural networks for combinatorial optimization (Hopfield-like networks) and neural models for optimization problems with continuous decision variables. Besides the brief stability analysis of the proposed neural network we also show how to derive from it lossy versions of improved Hopfield neural models .
The paper deals with continuously operating optimization neural networks with lossy dynamics. As the main feature of the neural model time-varying nature of neuron activation functions is introduced. The model presented is general in the sense that it covers the cases of neural networks for combinatorial optimization (Hopfield-like networks) and neural models for optimization problems with continuous decision variables. Besides the brief stability analysis of the proposed neural network we also show how to derive from it lossy versions of improved Hopfield neural models .
ES2000-42
An algorithm for the addition of time-delayed connections to recurrent neural networks
R. Bone, M. Crucianu, J.-P. Asselin de Beauville
An algorithm for the addition of time-delayed connections to recurrent neural networks
R. Bone, M. Crucianu, J.-P. Asselin de Beauville
Abstract:
Recurrent neural networks possess interesting universal approximation capabilities, making them good candidates for time series modeling. Unfortunately, long term dependencies are difficult to learn if gradient descent algorithms are employed. We support the view that it is easier for these algorithms to find good solutions if one includes connections with time delays in the recurrent networks. The algorithm we present here allows one to choose the right locations and delays for such connections. As we show on two benchmark problems, this algorithm produces very good results while keeping the total number of connections in the recurrent network to a minimum.
Recurrent neural networks possess interesting universal approximation capabilities, making them good candidates for time series modeling. Unfortunately, long term dependencies are difficult to learn if gradient descent algorithms are employed. We support the view that it is easier for these algorithms to find good solutions if one includes connections with time delays in the recurrent networks. The algorithm we present here allows one to choose the right locations and delays for such connections. As we show on two benchmark problems, this algorithm produces very good results while keeping the total number of connections in the recurrent network to a minimum.
Time series prediction
ES2000-261
The K.U.Leuven competition data: a challenge for advanced neural network techniques
J.A.K. Suykens, J. Vandewalle
The K.U.Leuven competition data: a challenge for advanced neural network techniques
J.A.K. Suykens, J. Vandewalle
Abstract:
In this paper we shortly discuss the K.U. Leuven time-series prediction competition, which has been held in the framework of the International Workshop on Advanced Black-Box Techniques for Nonlinear Modeling, K.U.Leuven Belgium July 8-10 1998. The data are related to a 5-scroll attractor, generated from a generalized Chua's circuit. The time-series consists of a given set of 2000 data points, where the next 200 points are to be predicted. In total, 17 entries have been submitted. The winning contribution by McNames succeeds in making an accurate prediction over; a time horizon of about 300 points using a nearest trajectory method, which incorporates local modeling and cross-validation techniques. The competition data can serve as a challenging test-case for advanced nonlinear modelling techniques, including neural networks. The data are able to reveal shortcomings of many methods.
In this paper we shortly discuss the K.U. Leuven time-series prediction competition, which has been held in the framework of the International Workshop on Advanced Black-Box Techniques for Nonlinear Modeling, K.U.Leuven Belgium July 8-10 1998. The data are related to a 5-scroll attractor, generated from a generalized Chua's circuit. The time-series consists of a given set of 2000 data points, where the next 200 points are to be predicted. In total, 17 entries have been submitted. The winning contribution by McNames succeeds in making an accurate prediction over; a time horizon of about 300 points using a nearest trajectory method, which incorporates local modeling and cross-validation techniques. The competition data can serve as a challenging test-case for advanced nonlinear modelling techniques, including neural networks. The data are able to reveal shortcomings of many methods.
ES2000-255
Local model optimization for time series prediction
J. McNames
Local model optimization for time series prediction
J. McNames
Abstract:
Local models have emerged as one of the leading methods of chaotic time series prediction. However, the accuracy of local models is sensitive to the choice of user-specified parameters, not unlike neural networks and other methods. This paper describes a method of optimizing these parameters so as to minimize the leave-one-out cross-validation error. This approach reduces the burden on the user to pick appropriate values and improves the prediction accuracy.
Local models have emerged as one of the leading methods of chaotic time series prediction. However, the accuracy of local models is sensitive to the choice of user-specified parameters, not unlike neural networks and other methods. This paper describes a method of optimizing these parameters so as to minimize the leave-one-out cross-validation error. This approach reduces the burden on the user to pick appropriate values and improves the prediction accuracy.
ES2000-251
A multi-steap ahead prediction method based on local dynamic properties
G. Bontempi, M. Birattari
A multi-steap ahead prediction method based on local dynamic properties
G. Bontempi, M. Birattari
Abstract:
The task of forecasting a time series over a long horizon is commonly tackled by iterating one-step-ahead predictors. Despite the popularity that this approach gained in the prediction community, its design is still plagued by a number of important unresolved issues, the most important being the accumulation of prediction errors. We introduce a local method to learn one-step-ahead predictors with the aim of reducing the propagation of errors during the iteration. For each prediction, our method selects the structure of the local approximator using, in a local version, well-known results of dynamic system theory. Experimental results on two time series from the Santa Fe competition show that the technique is competitive with state-of-the-art forecasting methods.
The task of forecasting a time series over a long horizon is commonly tackled by iterating one-step-ahead predictors. Despite the popularity that this approach gained in the prediction community, its design is still plagued by a number of important unresolved issues, the most important being the accumulation of prediction errors. We introduce a local method to learn one-step-ahead predictors with the aim of reducing the propagation of errors during the iteration. For each prediction, our method selects the structure of the local approximator using, in a local version, well-known results of dynamic system theory. Experimental results on two time series from the Santa Fe competition show that the technique is competitive with state-of-the-art forecasting methods.
ES2000-252
Nonlinear prediction of spatio-temporal time series
U. Parlitz, C. Merkwirth
Nonlinear prediction of spatio-temporal time series
U. Parlitz, C. Merkwirth
Abstract:
A prediction scheme for spatio-temporal time series is presented that is based on reconstructed local states. As a numerical example the evolution of a Kuramoto-Sivashinsky equation is forecasted using previously sampled data.
A prediction scheme for spatio-temporal time series is presented that is based on reconstructed local states. As a numerical example the evolution of a Kuramoto-Sivashinsky equation is forecasted using previously sampled data.
ES2000-256
A Bayesian approach to combined neural networks forecasting
M.D. Out, W.A. Kosters
A Bayesian approach to combined neural networks forecasting
M.D. Out, W.A. Kosters
Abstract:
Suitable neural networks may act as experts for time series predictions. The naive prediction is in a Bayesian manner used as prior to steer the weighted combination of these experts.
Suitable neural networks may act as experts for time series predictions. The naive prediction is in a Bayesian manner used as prior to steer the weighted combination of these experts.
ES2000-257
Time series forecasting using CCA and Kohonen maps - application to electricity consumption
A. Lendasse, J. Lee, V. Wertz, M. Verleysen
Time series forecasting using CCA and Kohonen maps - application to electricity consumption
A. Lendasse, J. Lee, V. Wertz, M. Verleysen
Abstract:
A general-purpose useful parameter in time series forecasting is the regressor, corresponding to the minimum number of variables necessary to forecast the future values of the time series. If the models used are non linear, the choice of this regressor becomes very difficult. We will show a quasi-automatic method using Curvilinear Component Analysis to build it. This method will be applied to electric consumption of Poland.
A general-purpose useful parameter in time series forecasting is the regressor, corresponding to the minimum number of variables necessary to forecast the future values of the time series. If the models used are non linear, the choice of this regressor becomes very difficult. We will show a quasi-automatic method using Curvilinear Component Analysis to build it. This method will be applied to electric consumption of Poland.
ES2000-259
Financial predictions based on bootstrap-neural networks
A. Lombardi, A. Vicino
Financial predictions based on bootstrap-neural networks
A. Lombardi, A. Vicino
Abstract:
In this paper neural networks are applied to financial data in order to predict the daily price of the financial index LIFFE. Our attention is focused on the choice of the exogeneous variables and on the training of the network itself. The first problem is solved by using the pre-whitening method that provides information on which variables are the most relevant for our prediction. The latter problem is due to the fact that data referring to a far past cannot be used because of the non-stationarity of the financial indicators. This implies that the training set is relatively small and it is necessary to extract as much information as possible from recent data. The bootstrap approach is applied to the training set of the neural network to improve the predicition capabilities of the system. This results in better prediction performances even when a limited number of data is available.
In this paper neural networks are applied to financial data in order to predict the daily price of the financial index LIFFE. Our attention is focused on the choice of the exogeneous variables and on the training of the network itself. The first problem is solved by using the pre-whitening method that provides information on which variables are the most relevant for our prediction. The latter problem is due to the fact that data referring to a far past cannot be used because of the non-stationarity of the financial indicators. This implies that the training set is relatively small and it is necessary to extract as much information as possible from recent data. The bootstrap approach is applied to the training set of the neural network to improve the predicition capabilities of the system. This results in better prediction performances even when a limited number of data is available.
ES2000-253
On the use of the wavelet decomposition for time series prediction
S. Soltani
On the use of the wavelet decomposition for time series prediction
S. Soltani
Abstract:
This paper deals with the problem of nonlinear time series prediction. The method uses a couple of filters to decompose iteratively the series. This scheme leads to a time series which contains the slowest dynamics and a hierarchy of detail time series which contain intermediate, up to the highest, dynamics. The new series are then used for modeling and predicting. The result obtained on the Mackey-Glass chaotic series show the efficiency of this approach.
This paper deals with the problem of nonlinear time series prediction. The method uses a couple of filters to decompose iteratively the series. This scheme leads to a time series which contains the slowest dynamics and a hierarchy of detail time series which contain intermediate, up to the highest, dynamics. The new series are then used for modeling and predicting. The result obtained on the Mackey-Glass chaotic series show the efficiency of this approach.
ES2000-254
Chaotic time series prediction using the Kohonen algorithm
L. Monzon Benitez, A. Ferreira, D. I. Pedreira Iparraguirre
Chaotic time series prediction using the Kohonen algorithm
L. Monzon Benitez, A. Ferreira, D. I. Pedreira Iparraguirre
Abstract:
Deterministic nonlinear prediction is a powerful technique for the analysis and prediction of time series generated by nonlinear dynamical systems. In;this paper the use of a Kohonen network as a component of one deterministic nonlinear prediction algorithm is suggested. In order to evaluate the performance of the proposed algorithm, it was applied to the prediction of time series generated by two well known chaotic dynamical systems and the results were compared with those obtained using the Modified Method of Analogues with the same time series. The generated time series were corrupted by superimposed observational noise. The experimental results have shown that the Kohonen network can learn the neighborhood relations present in the reconstructed attractor of the time series and that good predictions can also be obtained with the proposed algorithm.
Deterministic nonlinear prediction is a powerful technique for the analysis and prediction of time series generated by nonlinear dynamical systems. In;this paper the use of a Kohonen network as a component of one deterministic nonlinear prediction algorithm is suggested. In order to evaluate the performance of the proposed algorithm, it was applied to the prediction of time series generated by two well known chaotic dynamical systems and the results were compared with those obtained using the Modified Method of Analogues with the same time series. The generated time series were corrupted by superimposed observational noise. The experimental results have shown that the Kohonen network can learn the neighborhood relations present in the reconstructed attractor of the time series and that good predictions can also be obtained with the proposed algorithm.
ES2000-260
Curve forecast with the SOM algorithm: using a tool to follow the time on a Kohonen map
P. Rousset
Curve forecast with the SOM algorithm: using a tool to follow the time on a Kohonen map
P. Rousset
Abstract:
To forecast a complete curve, we propose a method that consists in predicting from a rule based on a classification, which takes the present time class into account. This technique is simpler than a vectorial prediction and solves some problems of long term forecasting. A type of error, which belongs to this method, imposes to take care of it. The SOM and a tool of visualization that permits to follow the time on that kind of classification give us a way to control it. The application to the polish electrical consumption is presented.
To forecast a complete curve, we propose a method that consists in predicting from a rule based on a classification, which takes the present time class into account. This technique is simpler than a vectorial prediction and solves some problems of long term forecasting. A type of error, which belongs to this method, imposes to take care of it. The SOM and a tool of visualization that permits to follow the time on that kind of classification give us a way to control it. The application to the polish electrical consumption is presented.
ANN models and learning III
ES2000-24
Learning principal components in a contextual space
T. Voegtlin
Learning principal components in a contextual space
T. Voegtlin
Abstract:
Principal Components Analysis (PCA) consists in finding the orthogonal directions of highest variance in a distribution of vectors. In this paper, we propose to extract the principal components of a random vector that partially results from a previous PCA. We demonstrate that this contextual PCA provides an optimal linear encoding of temporal context. A recurrent neural network based on this principle is evaluated.
Principal Components Analysis (PCA) consists in finding the orthogonal directions of highest variance in a distribution of vectors. In this paper, we propose to extract the principal components of a random vector that partially results from a previous PCA. We demonstrate that this contextual PCA provides an optimal linear encoding of temporal context. A recurrent neural network based on this principle is evaluated.
ES2000-47
Training activation function in parametric classification
V. Colla, L.M. Reyneri, M. Sgarbi
Training activation function in parametric classification
V. Colla, L.M. Reyneri, M. Sgarbi
Abstract:
This work shows how to train the activation function in neuro-wavelet parametric modeling and how this improves performance in a number of modeling, classification and forecasting. A real example in the domain of pattern classification is presented.
This work shows how to train the activation function in neuro-wavelet parametric modeling and how this improves performance in a number of modeling, classification and forecasting. A real example in the domain of pattern classification is presented.
ES2000-7
Quantum iterative algorithm for image reconstruction problems
J.-i. Inoue
Quantum iterative algorithm for image reconstruction problems
J.-i. Inoue
Abstract:
Iterative algorithm based on quantum tunneling is proposed by making use of mean-field approximation. We apply our method to the problem of BW image reconstruction (IR). Its performance is investigated both analytically and numerically.
Iterative algorithm based on quantum tunneling is proposed by making use of mean-field approximation. We apply our method to the problem of BW image reconstruction (IR). Its performance is investigated both analytically and numerically.
ES2000-28
Quaternionic spinor MLP
S. Buchholz, G. Sommer
Quaternionic spinor MLP
S. Buchholz, G. Sommer
Abstract:
This paper introduces a novel quaternion--valued MLP--type network called the Quaternionic Spinor MLP (QSMLP). In contrast to another ecently proposed Quaternionic MLP it uses spinors in its propagation function. This allows very efficiently the processing of 3D vector data, which is demonstrated by experiments. The QSMLP is proven to be a universal approximator and a learning algorithm for it is derived.
This paper introduces a novel quaternion--valued MLP--type network called the Quaternionic Spinor MLP (QSMLP). In contrast to another ecently proposed Quaternionic MLP it uses spinors in its propagation function. This allows very efficiently the processing of 3D vector data, which is demonstrated by experiments. The QSMLP is proven to be a universal approximator and a learning algorithm for it is derived.
ES2000-29
Simplified neural architectures for symmetric boolean functions
B. Girau
Simplified neural architectures for symmetric boolean functions
B. Girau
Abstract:
The theoretical and practical framework of Field Programmable Neural Arrays has been defined to reconcile simple hardware topologies with complex neural architectures: FPNAs lead to powerful neural models whose original data exchange scheme allows to use hardware-friendly neural topologies. This paper addresses preliminary results in the study of the computation power of FPNAs. The computation of symmetric boolean functions is taken as a textbook example. The FPNA concept allows successive topology simplifications of standard neural models for such functions, so that the number of weights is greatly reduced with respect to previous works.
The theoretical and practical framework of Field Programmable Neural Arrays has been defined to reconcile simple hardware topologies with complex neural architectures: FPNAs lead to powerful neural models whose original data exchange scheme allows to use hardware-friendly neural topologies. This paper addresses preliminary results in the study of the computation power of FPNAs. The computation of symmetric boolean functions is taken as a textbook example. The FPNA concept allows successive topology simplifications of standard neural models for such functions, so that the number of weights is greatly reduced with respect to previous works.
Artificial neural networks for energy management systems
ES2000-504
Connectionist solutions for energy management systems
G. Joya
Connectionist solutions for energy management systems
G. Joya
Abstract:
In this paper a schematic survey on main tasks dealing with power system management as been accomplished. Each task has been defined and studied from a neural perspective. Moreover, a representative but non exhaustive analysis of recent reported works in the field is presented. Constitutive elements of a power system, such as generators, buses, lines, breakers, as well as most relevant magnitudes such as load, injection, active and reactive power flow, voltage and current, are introduced. The tasks which have been analyzed are Load Forecasting, Alarm Processing, Fault Diagnosis, State Estimation including Observability Analysis and Topology Assessment, Security Analysis, emphasizing Contingency and Transient Stability Analysis, and Operational Planning, including Expansion Planning, Unit Commitment and Economic Dispatch.
In this paper a schematic survey on main tasks dealing with power system management as been accomplished. Each task has been defined and studied from a neural perspective. Moreover, a representative but non exhaustive analysis of recent reported works in the field is presented. Constitutive elements of a power system, such as generators, buses, lines, breakers, as well as most relevant magnitudes such as load, injection, active and reactive power flow, voltage and current, are introduced. The tasks which have been analyzed are Load Forecasting, Alarm Processing, Fault Diagnosis, State Estimation including Observability Analysis and Topology Assessment, Security Analysis, emphasizing Contingency and Transient Stability Analysis, and Operational Planning, including Expansion Planning, Unit Commitment and Economic Dispatch.
ES2000-501
Stability assessment of electric power systems using growing neural gas and self-organizing maps
C. Rehtanz, C. Leder
Stability assessment of electric power systems using growing neural gas and self-organizing maps
C. Rehtanz, C. Leder
Abstract:
Liberalized competitive electrical energy markets need tools for real-time stability assessment to link the technical with the market issues. Analytical tools are available but time-consuming. Alternatively, knowledge based systems speed up the stability assessment but most of them need extensive and assessed training data. Unsupervised learning methods like Growing Neural Gas or Self-Organizing Maps use training situations and the information of stability sepa-rately. Doing this, the calculation of training data is less time consuming. The use of the two methods within a fully automated tool for stability assessment is discussed in this paper. Aspects of self-learning, quality of the assessment and application to real power systems are considered.
Liberalized competitive electrical energy markets need tools for real-time stability assessment to link the technical with the market issues. Analytical tools are available but time-consuming. Alternatively, knowledge based systems speed up the stability assessment but most of them need extensive and assessed training data. Unsupervised learning methods like Growing Neural Gas or Self-Organizing Maps use training situations and the information of stability sepa-rately. Doing this, the calculation of training data is less time consuming. The use of the two methods within a fully automated tool for stability assessment is discussed in this paper. Aspects of self-learning, quality of the assessment and application to real power systems are considered.
ES2000-34
Load forecasting dealing with medium voltage network reconfiguration
J.N. Fidalgo, J.A. Pecas Lopes
Load forecasting dealing with medium voltage network reconfiguration
J.N. Fidalgo, J.A. Pecas Lopes
Abstract:
Planing the operation in modern power systems requires suitable anticipation of load evolution at different levels of distribution network. Under this perspective, load forecasting performs an important task, allowing the optimization of investments and the adequate exploitation of existing distribution networks. This paper describes the models developed for current intensity forecasting at primary substation feeders. The main goal consists on defining a regression process characterized by good quality estimates of those future intensity values, based on historical database. Anticipation interval shall include from the next hour to one week in advance. The forecasting method shall also be adaptable to power network reconfiguration, whenever planned or not. In this work, artificial neural networks (ANN) were used as the basic regression tool. This paper describes used ANN as well as the premises that led to the implementation of selected forecasting models. At last, some illustrative results attained so far are presented, supporting the adequacy of adopted approach.
Planing the operation in modern power systems requires suitable anticipation of load evolution at different levels of distribution network. Under this perspective, load forecasting performs an important task, allowing the optimization of investments and the adequate exploitation of existing distribution networks. This paper describes the models developed for current intensity forecasting at primary substation feeders. The main goal consists on defining a regression process characterized by good quality estimates of those future intensity values, based on historical database. Anticipation interval shall include from the next hour to one week in advance. The forecasting method shall also be adaptable to power network reconfiguration, whenever planned or not. In this work, artificial neural networks (ANN) were used as the basic regression tool. This paper describes used ANN as well as the premises that led to the implementation of selected forecasting models. At last, some illustrative results attained so far are presented, supporting the adequacy of adopted approach.
ES2000-502
Application of MLP and stochastic simulations for electricity load forecasting in Russia
E. Savelieva, A. Kravetski, S. Chernov, V. Demyanov, V. Timonin, R. Arutyunyan, L. Bolshov, M. Kanevski
Application of MLP and stochastic simulations for electricity load forecasting in Russia
E. Savelieva, A. Kravetski, S. Chernov, V. Demyanov, V. Timonin, R. Arutyunyan, L. Bolshov, M. Kanevski
Abstract:
The work is devoted to an application of artificial neural network (multilayer perceptron) and conditional stochastic simulations to electricity load forecasting in Russia. One of the problems is missing data and some important weather parameters (wind, cloudiness, precipitation, historical information). This gives rise to rather large forecasting errors with complex statistical structure. Another problem deals with economic trends in the country during the last decade and its influence (sometimes contradictory) on electricity consumption. The methodological innovative aspect of the study is a use of geostatistical tools (variography) and simulations to characterise the expected variability of the results.
The work is devoted to an application of artificial neural network (multilayer perceptron) and conditional stochastic simulations to electricity load forecasting in Russia. One of the problems is missing data and some important weather parameters (wind, cloudiness, precipitation, historical information). This gives rise to rather large forecasting errors with complex statistical structure. Another problem deals with economic trends in the country during the last decade and its influence (sometimes contradictory) on electricity consumption. The methodological innovative aspect of the study is a use of geostatistical tools (variography) and simulations to characterise the expected variability of the results.
Learning in biological and artificial systems
ES2000-22
SpikeProp: backpropagation for networks of spiking neurons
S. M. Bohte, J. N. Kok, H. La Poutré
SpikeProp: backpropagation for networks of spiking neurons
S. M. Bohte, J. N. Kok, H. La Poutré
Abstract:
For a network of spiking neurons with reasonable post-synaptic potentials, we derive a supervised learning rule akin to traditional error-back-propagation, "SpikeProp" and demonstrate how to overcome the discontinuities introduced by thresholding. Using this learning algorithm, we demonstrate how networks of spiking neurons with biologically plausible time-constants can perform complex non-linear classification in fast temporal coding just as well as rate-coded networks. When comparing the (implicit) number of neurons required for the respective encodings, it is empirically demonstrated that temporal coding potentially requires significantly less neurons.
For a network of spiking neurons with reasonable post-synaptic potentials, we derive a supervised learning rule akin to traditional error-back-propagation, "SpikeProp" and demonstrate how to overcome the discontinuities introduced by thresholding. Using this learning algorithm, we demonstrate how networks of spiking neurons with biologically plausible time-constants can perform complex non-linear classification in fast temporal coding just as well as rate-coded networks. When comparing the (implicit) number of neurons required for the respective encodings, it is empirically demonstrated that temporal coding potentially requires significantly less neurons.
ES2000-15
Nonsynaptically connected neural nets
G. L. Aiello, P. Bach-y-Rita
Nonsynaptically connected neural nets
G. L. Aiello, P. Bach-y-Rita
Abstract:
Neural nets are generally considered to be connected synaptically. However, the majority of information transfer in the brain may not be by synapses. Nonsynaptic diffusion neurotransmission (NDN) may be a major mechanism for information transfer in the brain. In this paper, a model of a diffusive-only neural net, based on a ligand-receptor dynamics, is presented. A mechanism of active release of neurotransmitters as a response to binding, along with one of depletion of free ligands, make the net capable of spontaneous activity. States of thermodyamical equilibrium, and trajectory of the system in the phase space have been numerically determined. The results indicate the possibility of chaotic behavior. The relevance of the theoretical model to the study of some brain mass-sustained functions is discussed.
Neural nets are generally considered to be connected synaptically. However, the majority of information transfer in the brain may not be by synapses. Nonsynaptic diffusion neurotransmission (NDN) may be a major mechanism for information transfer in the brain. In this paper, a model of a diffusive-only neural net, based on a ligand-receptor dynamics, is presented. A mechanism of active release of neurotransmitters as a response to binding, along with one of depletion of free ligands, make the net capable of spontaneous activity. States of thermodyamical equilibrium, and trajectory of the system in the phase space have been numerically determined. The results indicate the possibility of chaotic behavior. The relevance of the theoretical model to the study of some brain mass-sustained functions is discussed.
ES2000-49
Establishing retinotopy by lateral-inhibition type homogeneous neural fields
W. A. Fellenz, J. G. Taylor
Establishing retinotopy by lateral-inhibition type homogeneous neural fields
W. A. Fellenz, J. G. Taylor
Abstract:
We study the topographic development of receptive fields by simulating the continuum field equations with learning on a two-dimensional lattice. The observed plasticity reveals a columnar organisation with spatial clustering of receptive field centres, and the development of oriented receptive fields, if certain conditions ar met.
We study the topographic development of receptive fields by simulating the continuum field equations with learning on a two-dimensional lattice. The observed plasticity reveals a columnar organisation with spatial clustering of receptive field centres, and the development of oriented receptive fields, if certain conditions ar met.