ESANN 2001 - proceedings

Abstract:
Abstract. One of the main issues in the research on time series is its prediction. Using a tapped-delay neural network we formulate the optimal network size from the signal correlation time. Then the biofeedback-driven neurocontrol for artificial human walking is developed with much detail on the signal preprocessing. Finally we indicate the need for a hierarchical network architecture to eliminate oscillatory effects inherent to handling the human motor control problem.

---

Abstract:
In conventional approaches for multi-class pattern recognition using Support Vector Machines (SVMs), each class is discriminated against all the other classes to build an SVM for that class. We propose a close-class-set discrimination method suitable for large class set pattern recognition problems. An SVM is built for each of the 145 Consonant-Vowel (CV) classes by discriminating that class against only a small number (about 15) of classes close to it phonetically. The method leads to about 17% reduction in the average number of support vectors per class with a decrease of only 4.4% in the recognition accuracy.

---

Abstract:
Automatic Revelance Determination (ARD) has been applied to multilayer perceptrons by inferring di erent regularization parameters for the input interconnection layer within the evidence framework. In this paper, this idea is extended towards Least Squares Support Vector Machines (LS-SVMs) for classification. Relating a probabilistic framework to the LS-SVM formulation on the rst level of Bayesian inference, the hyperparameters are inferred on the second level. Model comparison is performed on the third level in order to select the parameters of the kernel function. ARD is performed by introducing a diagonal weighting matrix in the kernel function. These diagonal elements are obtained by evidence maximization on the third level of inference. Inputs with a low weight value are less relevant and can be removed.

Abstract:
We present a neural network clustering approach to the analysis of dynamic cerebral contrast-enhanced perfusion magnetic resonance imaging (MRI) time-series. In contrast to conventional extraction of a few voxel-based perfusion parameters alone, neural network clustering does neither discard information contained in the complete signal dynamics time-series nor is its interpretation biased by the indicator-dilution theory of non-diffusible tracers, which may not be applicable under pathological conditions of a disrupted blood-brain barrier. We performed exploratory data analysis in patients with cerebrovascular disease. Minimal free energy vector quantization provided a self-organized segmentation of voxels w.r.t. fine-grained differences of signal amplitude and dynamics, thus identifying different vessel sizes, side asymmetries and local deficits of brain perfusion. We conclude that neural network clustering can provide a useful extension to the computation of conventional perfusion parameter maps. Thus, it can contribute to the analysis of cerebral circulation by non-invasive neuroimaging.

---

Abstract:
Morphological Neural Networks (MNN) have been proposed as an alternative neural computation paradigm. In this paper we explore the potential of Heteroassociative MNN (HMNN) for a vision based practical task, that of self-localization in a vision-based navigation framework for mobile robots. HMNN have a big potential for real time application because its recall process is very fast. We present some experimental results that illustrate the proposed approach.

---

Abstract:
This paper presents an adaptive procedure for the linear and non-linear separation of signals with non-uniform, symmetrical probability distributions, based on both simulated annealing and competitive learning methods by means of a neural network, considering the properties of the vectorial spaces of signals, and using a multiple linearization in the mixture space.

---

Abstract:
In this paper, we will focus on the problem of blind source separation for independent and identically distributed variables (iid). The problem may be stated as follows: we observe a linear (unknown) mixture of k iid variables (the sources), and we want to recover either the sources or the linear mapping. We give online stability conditions of the algorithm using the eigenvalues of the hessian matrix of the pseudo-likelihood matching our set of observations.

Abstract:
Algorithms, applications and hardware implementations of neural networks are not investigated in close connection. Researchers working in the development of dedicated hardware implementations develop simplifi ed versions of otherwise complex neural algorithms or develop dedicated algorithms: usually these algorithms have not been thoroughly tested on real-world applications. At the same time, many theoretically sound algorithms are not feasible in dedicated hardware, therefore limiting their success only to applications where a software solution on a general-purpose system is feasible. The paper focuses on the issues related to the hardware implementation of neural algorithms and architectures and their successful application to real world-problems.

---

Abstract:
While digital processors can solve problems in most application areas, in some fields their capabilities are very limited. A typical example is vision. Simple animals outperform super-computers in the realization of basic vision tasks. The limitations of conventional digital systems in this field can be overcome following a fundamentally different approach based on architectures closer to nature solutions. Retinas, the front end of biological vision systems, obtain their high processing power from parallelism, and consist of concurrent spatial distributions (on the focal plane aerea) of photoreceptors and basic analog processors with local connectivity and moderate accuracy. This can be implemented using an architecture with the following main components are: a) parallel processing through an array of locally-connected analog processors; b) a means of storing, locally, pixel-by-pixel, the intermediate computation results, and 3) stored on-chip programmability. When implemented as a mixed-signal VLSI chip, devices are obtained which are capable of image processing at rates of trillions of operations per second with very small size and low power consumption. This paper reviews the latest results on this type of chips and systems, and outlines the envisaged roadmap for these computers.

---

Abstract:
Probabilistic algorithms o er a means of computing that works with the grain of analogue hardware, rather than against it. This paper proposes the use of such an algorithm in applications where the advantages of analogue hardware are most likely to be realised.

---

Abstract:
In this paper we present a microelectronic implementation of a neural network of coupled oscillators that can segment black and white images. As an alternative to structures used in computer simulations where mathematical simplicity is more important, we used simple current mode astable multivibrators that can be easily implemented on silicon. Experimental results demonstrate the feasibility of this approach.

---

Abstract:
Support Vector Machines are gaining more and more acceptance thanks to their success in many real–world problems. We address in this work some issues related to their hardware implementations for identification and control of a thermal model of an extruder for injection molding process.

Abstract:
Sigmoidal or radial transfer functions do not guarantee the best generalization nor fast learning of neural networks. Families of parameterized transfer functions provide flexible decision borders. Networks based on such transfer functions should be small and accurate. Several possibilities of using transfer functions of different types in neural models are discussed, including enhancement of input features, selection of functions from a fixed pool, optimization of parameters of general type of functions, regularization of large networks with heterogeneous nodes and constructive approaches. A new taxonomy of transfer functions is proposed, allowing for derivation of known and new functions by additive or multiplicative combination of activation and output functions.

---

Abstract:
Approximation capabilities of single non-linear layer networks, that feature a single global minimum of the error function are addressed. Bases of different transfer functions are tested (Gaussian, sigmoidal, multiquadratics). These functions are orthogonalised in an incremental manner for training and restored back to the original basis for network deployment. Approximation results are given for a benchmark ECG signal. Results of incremental training with basis orthogonalisation are also shown for 2D approximations.

---

Abstract:
Neural networks use neurons of the same type in each layer. Such architecture cannot lead to data models of optimal complexity and accuracy. Networks with architectures (number of neurons, connections and type of neurons) optimized for a given problem are described here. Each neuron may implement transfer function of different type. Complexity of such networks is controlled by statistical criteria and by adding penalty terms to the error function. Results of numerical experiments on artificial data are reported.

---

Abstract:
Networks estimating probability density are usually based on radial basis function of the same type. Feature Space Mapping constructive network based on separable functions, optimizing type of the function that is added, is described. Small networks of such type may discover accurate representations of data. Numerical experiments on artificial and real datasets are reported.

Abstract:
In this paper, we present an empirical comparison among four different schemes of coding the outputs of a Multilayer Feedforward networks. Results are obtained for eight different classification problems from the UCI repository of machine learning databases. Our results show that the usual codification is superior to the rest in the case of using one output unit per class. However, if we use several output units per class we can obtain an improvement in the generalization performance and in this case the noisy codification seems to be more appropriate.

---

Abstract:
In this paper, we present the results of an experimental comparison among seven different weight initialization methods in twelve different problems. The comparison is performed by measuring the speed of convergence, the generalization capability and the probability of successful convergence. It is not usual to find an evaluation of the three properties in the papers on weight initialization. The training algorithm was Backpropagation (BP) with a hyperbolic tangent transfer function. We found that the performance can be improved with respect to the usual initialization scheme.

---

Abstract:
The neocognitron network is analysed from the point of view of the contribution of the different layers to the final classification. A variation to the neocognitron which gives improved performance is suggested. This variant combines the low level feature extraction capabilities of the initial layers with alternative classifiers such as LVQ and Class Based Means Clustering. This is shown to give performance which is superior to the either of those classifiers acting on their own, and to the neocognitron in its standard form on two different instances of the letter recognition problem.

---

Abstract:
In order to control a prostheses by means of biological nerve signals, a self-organizing map (SOM) has been used to classify nerve signals recorded by a regeneration type neurosensor. The trained SOM contains the information about the relation between the recorded nerve signal and the winning neuron of the SOM. Classes of nerve signals fired by defined axons can be found in cluster on the SOM. For controlling a prostheses, the clusters on the SOM must be assigned to an action of the prostheses. Since the medical stuff is usually not experienced to identify the clusters within the SOM we have developed Clusot, an algorithm that defines automatically clusters within SOMs. After a short introduction to the project of controlling a prostheses by nerve signals, we present the signal processing of the project. In this paper, we focus on the automatic detection of clusters within a trained SOM using Clusot. Clusot will be explained within the context of the project in question.

Abstract:
\N

---

Abstract:
Living creatures improve their adaptation capabilities to a changing World by means of two orthogonal processes: evolution and lifetime learning. Within Artificial Intelligence, both mechanisms inspired the development of non-orthodox problem solving tools, namely Genetic and Evolutionary Algorithms (GEAs) and Artificial Neural Networks (ANNs). Several locale search gradient-based methods have been developed for ANN training, with considerable success; however, in some situations, such procedures may lead to local minima. Under this scenario, the combination of evolution and learning techniques, may lead to better results (e.g., global optima). Comparative tests on several Machine Learning tasks attest this claim.

---

Abstract:
Multiple Layer Perceptron networks trained with backpropagation algorithm are very frequently used to solve a wide variety of real-world problems. Usually a gradient descent algorithm is used to adapt the weights based on a comparison between the desired and actual network response to a given input stimulus. All training pairs, each consisting of input vector and desired output vector, are forming a more or less complex multi-dimensional error surface during the training process. Numerous suggestions have been made to prevent the gradient descent algorithm from becoming captured in any local minimum when moving across a rugged error surface. This paper describes an approach to substitute it completely by a genetic algorithm. By means of some benchmark applications characteristic properties of both the genetic algorithm and the neural network are explained.

---

Abstract:
In sensorimotor behaviour often a great movement execution variability is combined with a relatively low error in reaching the intended goal. This phenomenon can especially be observed if the limb chain under regard has redundant degrees of freedom. Such a redundancy, however, is a pre-requisit of movement optimization, because without variability changes in movement execution are impossible. It is, therefore, suggested, that, given a fitness criterion, a related optimal movement trajectory can be learned by an genetic algorithm. However, precise reaching must also be learned. This requires to establish at least an internal inverse model of the (forward) "tool transformation" governing the physical behaviour of the limb chain. Learning of an inverse model can be performed best applying the so called auto-imitation algorithm, a non-supervised learning mechanism equivalent to (modified) Hebbian learning. The paper shows theoretically, how these two learning algorithms can be combined in motor learning, and exemplifies by simulation of a three-jointed arm confined in a plane, how the problem of combining goal invariance under motor variability with movement optimization can be solved practically in a biologically plausible manner.

---

Abstract:
A new evolutionary algorithm to design nearest neightbour classifiers is presented in this paper. Main design topics of this sort of classifiers are the number of prototypes used and their position. This algorithm is based on the evolution of a population of prototypes that try to achieve an equilibrium by nding the right size of the population and the position of each prototype in the environment, solving at the same time both design topics above. A biological point of view is given to explain most of the concepts introduced, as well as the operators used in evolution.

---

Abstract:
The evolution strategy with neighborhood attraction (EN) is a new combination of self-organizing maps (SOM) and evolution strategies (ES). It adapts the neighborhood relationship known from SOM to ES individuals to concentrate them around the optimum of the problem. In this paper, detailed investigations on the influence of one of the most important EN-operators - the neighborhood attraction - were performed on a variety of well-known optimization problems. It could be shown that the parameter setting for the neighborhood attraction has a very strong influence on the convergence velocity and the robustness of the EN, and suggestions for applicable parameter settings could be made.

---

Abstract:
A structural genetic algorithm is proposed to optimize the High Order Neural Network (HONN) architecture and the parameters of activation function. This work partitions the genes of chromosomes into control genes and parameter genes in a hierarchical form. Control genes binary encoded in the chromosome represent the activity of neurons, while real-valued parameter genes, represent the parameters of the sigmoid activation function. To verify the performance of our system, the HONN is used to approximate 2-D and 1-D functions.

---

Abstract:
Most processes in the real world are controlled by nonlinear models. This explains the interest of the scientific community in the development of new methods to estimate the parameters of nonlinear models that allow the modeling of such processes. In this article we propose a new method for the estimation of parameters for nonlinear problems using genetic algorithms with real encoding (RCGA). In these genetic algorithms we use a crossover operator based on confidence intervals, which uses information from the best individuals in the population. For the resolution of this kind of problems, this operator is remarkably robust and efficient when compared with other crossover operators used in RCGAs.

---

Abstract:
In this article, a real-coded genetic algorithm (GA) is proposed capable of simultaneously optimizing the structure of a system (number of inputs, membership functions and rules) and tuning the parameters that define the fuzzy system. A multideme GA system is used in which various fuzzy systems with different numbers of input variables and with different structures are jointly optimized. Communication between the different demes is established by the migration of individuals presenting a difference in the dimensionality of the input space of a particular variable. We also propose coding by means of multidimensional matrices of the fuzzy rules such that the neighborhood properties are not destroyed by forcing it into a linear chromosome. The effectiveness of the proposed approach is verified and is compared with other fuzzy, and neuro-fuzzy approaches in terms of the root mean squared error (RMSE).

Abstract:
The extended Kalman filter (EKF) is considered one of the most effective methods for both nonlinear state estimation and parameter estimation (e.g., learning the weights of a neural network). Recently, a number of derivative free alternatives to the EKF for state estimation have been proposed. These include the Unscented Kalman Filter (UKF) [1, 2], the Central Difference Filter (CDF) [3] and the closely related Divided Difference Filter (DDF) [4]. These filters consistently outperform the EKF for state estimation, at an equal computational complexity of O(L3). Extension of the UKF to parameter estimation was presented by Wan and van der Merwe in [5, 6]. In this paper, we further develop these techniques for parameter estimation and neural network training. The extension of the CDF and DDF filters to parameter estimation, and their relation to UKF parameter estimation is presented. Most significantly, this paper introduces efficient square-root forms of the different filters. This enables an O(L2) implementation for parameter estimation (equivalent to the EKF), and has the added benefit of improved numerical stability and guaranteed positive semi-definiteness of the Kalman filter covariances.

---

Abstract:
Texture is an important characteristic di erentiating objects or regions of interest in an image. A variety of approaches to texture analysis has previously been proposed; the approach in this paper is from the stochastic eld, whereby a two-dimensional Volterra model is used to represent a texture eld. Statistical moments are introduced as a means of determining the Volterra model generator for an unknown texture, with a view to using this model subsequently for recognition. However, an overdetermined set of equations results. These can be solved in the least squares sense using a conjugate gradient descent method with multiple restarts.

---

Abstract:
The weight perturbation learning algorithm was formerly developed by hardware designers for its friendly features in the perspective of the analog on-chip implementation. Therefore it has not been used for real-world applications but it has been verified only on test problems. To significantly increase its attitude for the on-chip implementation, we proposed a local learning rate adaptation technique, which anyway, increases also the performance. At the same time to demonstrate the efficiency of the weight perturbation algorithm, in this paper we report the results of the application of the proposed algorithm to the classification of remote-sensing images. Our results compare favorably with those reported in the literature and demonstrate the soundness of the proposed approach.

Abstract:
Many researchers are interesting in applying the neural networks methods to financial data. In fact these data are very complex, and classical methods do not always give satisfactory results. They need strong hypotheses which can be false, they have a strongly non-linear structures, and so on. But neural models must also be cautiously used. The black box aspect can be very dangerous. In this very simple paper, we try to indicate some specificity of financial data, to prevent some bad use of neural models.

---

Abstract:
Prediction of financial time series using artificial neural networks has been the subject of many publications, even if the predictability of financial series remains a subject of scientific debate in the financial literature. Facing this difficulty, analysts often consider a large number of exogenous indicators, which makes the fitting of neural networks extremely difficult. In this paper, we analyze how to aggregate a large number of indicators in a smaller number using -possibly nonlinear- projection methods. Nonlinear projection methods are shown to be equivalent to the linear Principal Component Analysis when the prediction tool used on the new variables is linear. The methodology developed in the paper is validated on data from the BEL20 market index.

---

Abstract:
Forecasting currency exchange rates are an important financial problem that is receiving increasing attention especially because of its intrinsic difficulty and practical applications. This paper presents improved neural network and fuzzy models used for exchange rate prediction. Several approaches including multi-layer perceprtons, radial basis functions, dynamic neural networks and neuro-fuzzy systems have been proposed and discussed. Their performances for one-step a-head predictions have been evaluated through a study, using real exchange daily rate values of the US Dollar vs. British Pound.

---

Abstract:
We discuss a framework for modeling the switching dynamics of a time series based on hidden Markov models (HMM) of prediction experts, here neural networks. Learning is treated as a maximum likelihood problem. In particular, we present an Expectation-Maximization (EM) algorithm for adjusting the expert parameters as well as the HMM transition probabilities. Based on this algorithm, we develop a heuristic that achieves a hard segmentation of the time series into distinct dynamical modes and the simultaneous specialization of the prediction experts on the segments. We present examples of the application of this algorithm to the segmentation of artificial and nancial time series.

Abstract:
Searching and retrieving information from the Web poses new issues that can be effectively tackled by applying machine learning techniques. In particular, the fast dynamics of the information available on the Internet requires new approaches for indexing; the huge amount of available data is hardly manageable by humans and, on the other hand, can provide large sets of examples for learning algorithms; nally, there is the need of new services like search tools optimized for a specific Web community or even for a single user. Thus, the main areas for the application of these methodologies are the classification of Web documents, the modeling of users' behavior, the personalization of search engines, the automatical extraction of information from Web pages, and the auto-organization of documents on the base of their contents. Many services used to access information on the Web, like Web directories, crawlers, search engines and recommender systems, can be improved by learning from data. This survey reports the main contributions in the application of learning-based techniques to Web searching.

---

Abstract:
Over the years the amount and range of electronic text stored on the WWW has expanded rapidly, overwhelming both users and tools designed to index and search the information. It is impossible to index the WWW dynamically at query time due to the sheer volume so the index must be pre-compiled and stored in a compact but incremental data structure as the information is ever-changing. Much of the text is unstructured so a data structure must be constructed from such text, storing associations between words and the documents that contain them. The index must be able to index fine-grained word-based associations and also handle more abstract concepts such as synonym groups. A search tool is also required to link to the index and enable the user to pinpoint their required information. We describe such a system we have developed in an integrated hybrid neural architecture and evaluate our system against the benchmark SMART system for retrieval accuracy :recall and precision.

---

Abstract:
We propose a method to automatically determine the relevance of the input dimensions of a learning vector quantization (LVQ) architecture during training. The method is based on Hebbian learning and introduces weighting factors of the input dimensions which are automatically adapted to the specifi c problem. The benefits are twofold: On the one hand, the incorporation of relevance factors in the LVQ architecture increases the overall performance of the classification and adapts the metric to the specific data used for training. On the other hand, the method induces a pruning algorithm, i.e. an automatic detection of the input dimensions which do not contribute to the overall classifi er. Hence we obtain a possibly more eÆcient classification and we gain insight to the role of the data dimensions.

---

Abstract:
In this paper, a novel visualization method for partitions of multidimensional data is presented. It can be used for characterization of a single partition or comparison of two partitions. The method finds the variables that best describe a partition or difference between partitions. It is especially useful when the data set size is large and there are many variables, i.e., the data dimension is high. The method has been implemented in a software tool prototype which is used in analysis of operational states of a paper machine. For simplicity, however, use of the method is here demonstrated using the well-known Iris data set.

---

Abstract:
The neural gas algorithm provides a method to cluster a data space via an adaptive lattice of neurons which captures the topology of the data space. We propose di erent methods to determine the relevance of the single data dimensions for the overall neural architecture. This enables us to perform input pruning for the unsupervised neural gas architecture. The methods are tested on various datasets.

Abstract:
An exact solution of a system of coupled differential equations describing the dynamics of a special class of winner-take-all networks is given. From the solution two properties of the short-term-memory traces are derived: (1) information preservation and (2) a discrimination measure. These properties justify a biologically inspired fault tolerant extension of the network using differentiating neurons.

---

Abstract:
The basic premise of this research is that deterministic chaos is a powerful mechanism for the storage and retrieval of information in the dynamics of artificial neural networks. Substantial evidence has been found in biological studies for the presence of chaos in the dynamics of natural neuronal systems [1-3]. Many have suggested that this chaos plays a central role in memory storage and retrieval [1,4-6]. Indeed, chaos offers many advantages over alternative memory storage mechanisms used in artificial neural networks. One is that chaotic dynamics are significantly easier to control than other linear or non-linear systems, requiring only small appropriately timed perturbations to constrain them within specific Unstable Periodic Orbits (UPOs). Another is that chaotic attractors contain an infinite number of these UPOs. If individual UPOs can be made to represent specific internal memory states of a system, then in theory a chaotic attractor can provide an infinite memory store for the system. In this paper we investigate the possibility that a network can self-select UPOs in response to specific dynamic input signals. These UPOs correspond to network recognition states for these input signals.

Abstract:
We discuss three aspects of modeling information extraction from visual data. Firstly, we discuss pre-processing issues in the context of stability and biological plausibility. Secondly, we discuss the problem of extraction of depth information from stereo data. Finally, we discuss the extraction of (almost) independent features from a data set. We use these three aspects of processing visual data to illustrate some of the successes and issues involved in using unsupervised learning with artificial neural networks on such data sets.

---

Abstract:
This paper illustrates how canonical correlation analysis can be used for designing efficient visual operators by learning. The approach is highly task oriented and what constitutes the relevant information is defined by a set of examples. The examples are pairs of images displaying a strong dependence in the chosen feature but are otherwise independent. Experimental results are presented illustrating the learning of local shift invariant orientation operators, representation of velocity, and image content invariant disparity operators.

---

Abstract:
In 1992 neurophysiologists [5] found an new type of cells in areas V1 and V2 of the monkey primary visual cortex,whic h they called grating cells. These cells respond vigorously to a grating pattern of appropriate orientation and periodicity. A few years later a computational model inspired by these .ndings was published [3]. The study of this paper is to model a grating cell operator that responds in a very similar way as these grating cells do. Three di.erent databases containing a total of 338 real world images of textures were applied to the operator. Based on these images,our .ndings were that grating cells respond best to repetitive alternating patterns of a speci.c orientation. These patterns are mostly human made structures,lik e buildings,fabrics,and tiles.

---

Abstract:
We show that neither linear nor static nonlinear operations in computational models are capable of precisely reproducing the observed characteristics of retina ganglion cells (RGCs) responding to dynamic stimuli. In particular, velocity tuning of single cells stimulated by uniform motion and the signaling of motion starts and stops by a RGC population are considered. In both cases, the consideration of a dynamic non-linear feedback loop originally introduced to explain contrast gain control effects brings the temporal properties of the model into agreement with experimental findings from multielectrode recordings.

---

Abstract:
An approach to symbolic contour extraction will be described that consists of three stages: enhancement, detection, and extraction of edges and corners. Edges and corners are enhanced by models of monkey cortical complex and end-stopped cells. Detection of corners and local edge maxima is performed by selection of local maxima in both edge and corner enhanced images. These maxima form the anchor points of a greedy contour following algorithm that extracts the edges. This algorithm is based on an idea of spatially linking neurons along the edge that will .re in synchrony to indicate an extracted edge. The extracted edges and detected corners represent the symbolic representation of the image. The advantage of the proposed model over other models is that the same low constant thresholds for corner and local edge maxima detection are used for di.erent images. Closed contours are guaranteed by the contour following algorithm to yield a fully symbolic representation which is more suitable for reasoning and recognition. In this respect our methodology is unique, and clearly di.erent from the standard edge detection methods.

---

Abstract:
We review the recently proposed method of Relevance Vector Machines which is a supervised training method related to Support Vector Machines. We also review the statistical technique of Canonical Correlation Analysis and its implementation in a Feature Space. We show how the technique of Relevance Vectors may be applied to the method of Kernel Canonical Correlation Analysis to gain a very sparse representation of a data set and discuss why such a representation may be beneficial to an organism.

Abstract:
This paper describes a method of supervised learning based on forward selection branching. This method improves fault tolerance by means of combining information related to generalization performance and fault tolerance. The method presented focuses on the evolutionary nature of the learning algorithm of Radial Basis Function Networks and employs optimization techniques to control the balance between the approximation error with and without faults. The technique developed is empirically analysed and provides a simple and efficient means of learning fault tolerance. This is illustrated by examples taken from different classification and function approximation problems.

---

Abstract:
This work deals with the use of pruning ANNs in conjunction with genetic algorithms for resolving nonlinear multicomponent systems based on oscillating chemical reactions. The singular analytical response provides by this chemical system after its perturbation was tted to a gaussian curve by least-square regression and the estimates were used as inputs to the ANNs. The proposed methodology was validated by the simultaneous determination of pyrogallol and gallic acid (two strong related phenol derivatives) in mixtures on the basis of their perturbation e ects on the classical Belousov-Zhabotinskii reaction. The trained network estimates concentrations of pyrogallol and gallic acid with a standard error of prediction for the testing set of ca. 4% and 5.7% respectively or 4.4%, 9% for di erent sets of train/test patterns. This result is much smaller than those provided by a classical parametric method such as non-linear regression.

---

Abstract:
In this paper, a new SOM competition algorithm is proposed for image recognition applications. The competition in this algorithm depends on a subset of most discriminate weights of the network codebooks. This indeed can reduce the required recognition time of one image. In addition, the competition is applied on the pixels corresponding to the object gray levels only, this allows recognizing complex images with different lighting conditions. Furthermore, to allow shift variations in the position of the input object, window-based competition is proposed. Where, different subset windows are selected from the input image, then the competition is applied between each window and window of the same size in the center of the codebook of all feature map neurons. The experimental results of the new algorithm showed good performance in recognizing gestures of complex images with variant object position while the normal SOM competition algorithm completely failed.

---

Abstract:
A novel approach is presented to implement continuous. Hopfield neural networks, which are modelled by a system of ordinary differential equations (ODEs). The simulation of a continuous network in a digital computer implies the discretization of the ODE, which is usually carried out by simply substituting the derivative by the difference, without any further theoretical justification. Instead, the numerical solution of the ODE is proposed. Among the existing numerical methods for ODEs, we have selected the modified trapezoidal rule. The Hamiltonian Cycle Problem is used as an illustrative example to compare the novel method to the standard implementation. Simulation results show that this "numerical neural technique" obtains valid solutions of the problem and it is more efficient than other simulation algorithms. This technique opens a promising way to optimization neural networks that could be competitive with nonlinear programming methods.

---

Abstract:
Multiplayer perceptrons or feed-forward networks are generally trained to represent functions or many-to-one (m-o) mappings. This creates a problem if the training data exhibits the property of many-to-many or almost many-many valued-ness because the model, which generated the data, was many-to-many. Therefore in this paper a modified feed-forward network and training algorithm is considered to represent a multi-valued mappings. The solution consists of adding another input to the standard feed-forward network and of modifying the training algorithm. This additional input will generally have no training values provided and an amended training algorithm is used to find its values. The modified feed-forward network and training method has been successfully applied both in representing the mapping implied by data generated by multivalued functions and in representing the mapping implied by data obtained from benchmark databases.

---

Abstract:
We develop improved risk bounds for function estimation with models such as single hidden layer neural nets, using a penalized least squares criterion to select the size of the model. These results show the estimator achieves the best order of balance between approximation error and penalty relative to the sample size. Bounds are given both for the case that the target function is in the convex hull C of a class Fi of functions of dimension d (determined through empirical l2 convering numbers) and for the case that the target is not in the convex hull.

---

Abstract:
We treat the Bayesian decision problem, mainly the two-category case. A three layered neural network, having a logistic output unit and a small number of hidden layer units, can approximate the a posteriori probability in L2-norm, without knowing the type of the probability distribution before learning, if the log ratio of the a posteriori probabilities is a polynomial of low degree as in the case of most familiar probability distributions. This is because the log ratio itself can be well approximated by a linear sum of outputs of the hidden layer units in L2-norm.

---

Abstract:
Hybrid HMM/MLP models are useful to model piecewise stationary non-linear time series. A popular way to estimate the parameters of such models is to use the E.M. algorithm thanks to the Baum and Welch, forward-backward, algorithm. In this paper, we study a convenient way to estimate the parameters thanks to differential optimization. This new method can dramatically improve the time of calculus for long time series.

---

Abstract:
Time varying environments or model selection problems lead to crucial dilemmas in identification and control science. In this paper, we propose a modular prediction scheme consisting in a mixture of expert architecture. Several Kalman filters are forced to adapt their dynamics and parameters to different parts of the whole dynamics of the system. The performances of this modular learning scheme are evaluated on a visual servoing problem: motion prediction of an object in a 3-D space for pursuing it with a 3 degree-of-freedom robot manipulator.

---

Abstract:
We investigate the use of an unsupervised artificial neural network to form a sparse representation of the underlying causes in a data set. By using fixed lateral connections that are derived from the Rectified Generalised Gaussian distribution, we form a network that is capable of identifying multiple cause structure in visual data and grouping similar causes together on the output response of the network. We show that the network may be used to form local spatiotemporal filters in response to real images contained in video data.

---

Abstract:
Aim of the present article is to show the results obtained from the application of neuro-fuzzy methodology in the solution of agriculture problems like the Bactrocera Oleae (olive fly) infestation in the Liguria region olive grows. The research is focused to create an informatic decisional instrument to support experts in the applications of Integrated Pest Management strategies against the Bactrocera Oleae infestation. The system will suggest types of treatments for each monitored farm in order to optimize the quality of the olive oil and improve the economic and environmental impact of these treatments. Statistical and forecast analyses on data sets referred to agronomic case studies, like the growth of olive fly, are actually made using standard and model approaches like analytical; these dates instead present characteristics (big variability and non-linearity) which make them complex to be treat mathematically. Agronomic research needs to introduce new analysis techniques of taken dates and information, for example neuro-fuzzy methodologies that allow a large use of infestation dates with a good flexibility degree.