Kernel Machines
http://www.kernel-machines.org
These are the search results for the query, showing results 481 to 495.
Computation with infinite neural networks http://www.kernel-machines.org/publications/Williams98b No publisheradmin2007-01-31T10:09:03ZArticle ReferenceGaussian processes for regression http://www.kernel-machines.org/publications/WilRas96 No publisheradmin2007-01-31T10:09:03ZIncollection ReferenceComputing with infinite networks http://www.kernel-machines.org/publications/Williams97 No publisheradmin2007-01-31T10:09:04ZInproceedings ReferenceGaussian Processes for Bayesian Classification via Hybrid Monte Carlo http://www.kernel-machines.org/publications/BarWil97 No publisheradmin2007-01-31T10:09:04ZInproceedings ReferenceA Geometric Approach to Train Support Vector Machines http://www.kernel-machines.org/publications/YanAhu00 No publisheradmin2007-01-31T10:09:05ZInproceedings ReferenceSparse Greedy Gaussian Process Regression http://www.kernel-machines.org/publications/SmoBar01 No publisheradmin2007-01-31T10:09:05ZInproceedings ReferenceKernel Principal Component Regression with EM Approach to Nonlinear Principal Components Extraction. http://www.kernel-machines.org/publications/RJA00 In kernel based methods such as Support Vector Machines, Kernel PCA, Gaussian Processes or Regularization Networks the computational requirements scale as O(n^3) where n is the number of training points. In this paper we investigate Kernel Principal Component Regression (KPCR) with the Expectation Maximization approach in estimating of the subset of p principal components (p < n) in a feature space defined by a positive definite kernel function. The computational requirements of the method are O(pn^2). Moreover, the algorithm can be implemented with memory requirements O(p^2)+O((p+1)n)). We give the theoretical description explaining how by the proper selection of a subset of non-linear principal components desired generalization of the KPCR is achieved. On two data sets we experimentally demonstrate this fact. Moreover, on a noisy chaotic Mackey-Glass time series prediction the best performance is achieved with p << n and experiments also suggests that in such cases we can also use significantly reduced training data sets to estimate the non-linear principal components. The theoretical relation and experimental comparison to Kernel Ridge Regression and epsilon-insensitive Support Vector Regression is also given.No publisheradmin2007-01-31T10:09:05ZTechreport ReferenceA note on the decomposition methods for support vector regression http://www.kernel-machines.org/publications/LiaLinLin01 The dual formulation of support vector regression involves with two closely related sets of variables. When the decomposition method is used, many existing approaches use pairs of indices from these two sets as the working set. Basically they select a base set first and then expand it so that all indices are pairs. This makes the implementation different from that for support vector classification. In addition, a larger optimization sub-problem has to be solved in each iteration. In this paper from different aspects we demonstrate that there are no needs to do so. In particular we show that directly using this base set as the working set leads to similar convergence (number of iterations). Therefore, not only the program can be simpler, with a smaller working set and similar number of iterations, it can also be more efficient.No publisheradmin2007-01-31T10:09:06ZTechreport ReferenceLearning and Soft Computing, Support Vector Machines, Neural Networks and Fuzzy Logic Models http://www.kernel-machines.org/publications/Kecman01 This is the first textbook that provides a thorough, comprehensive and unified introduction to the field of learning from experimental data and soft computing. Support vector machines (SVMs) and neural networks (NNs) are the mathematical structures, or models, that underlie learning, while fuzzy logic systems (FLS) enable us to embed structured human knowledge into workable algorithms. The book assumes that it is not only useful, but necessary, to treat SVMs, NNs, and FLS as parts of a connected whole. The theory and algorithms are illustrated by 47 practical examples, as well as by 155 problem sets and simulated experiments. This approach enables the reader to develop SVMs, NNs, and FLS in addition to understanding them. The book also presents three case studies on: NNs based control, financial time series analysis, and computer graphics. A solutions manual and all of the MATLAB programs needed for the simulated experiments are available. Learning and Soft Computing provides a clearly organized book focusing on a broad range of algorithms and it is aimed at senior undergraduate students, graduate students and practicing researchers and scientists who want to use and develop SVMs, NNs and/or FL models rather than simply study them. The book is reach in graphical presentations (268 illustrations). The insight obtained through the simulation experimenting and graphical presentation of the results enables faster and better understanding of initially 'difficult' matrix-vectorial notations present in the field. This, we hope, will enable self-study too.No publisheradmin2007-01-31T10:09:07ZBook ReferenceIncremental and Decremental Support Vector Machine Learning http://www.kernel-machines.org/publications/CauPog01 An on-line recursive algorithm for training support vector machines, one vector at a time, is presented. Adiabatic increments retain the Kuhn-Tucker conditions on all previously seen training data, in a number of steps each computed analytically. The incremental procedure is reversible, and decremental ``unlearning'' offers an efficient method to exactly evaluate leave-one-out generalization performance. Interpretation of decremental unlearning in feature space sheds light on the relationship between generalization and geometry of the data.No publisheradmin2007-01-31T10:09:08ZInproceedings ReferenceA Bayesian Committee Machine http://www.kernel-machines.org/publications/Tresp00 The Bayesian committee machine (BCM) is a novel approach to combining estimators which were trained on different data sets. Although the BCM can be applied to the combination of any kind of estimators the main foci are Gaussian process regression and related systems such as regularization networks and smoothing splines for which the degrees of freedom increase with the number of training data. Somewhat surprisingly, we find that the performance of the BCM improves if several test points are queried at the same time and is optimal if the number of test points is at least as large as the degrees of freedom of the estimator. The BCM also provides a new solution for online learning with potential applications to data mining. We apply the BCM to systems with fixed basis functions and discuss its relationship to Gaussian process regression. Finally, we also show how the ideas behind the BCM can be applied in a non-Bayesian setting to extend the input dependent combination of estimators.No publisheradmin2007-01-31T10:09:08ZArticle ReferenceThe Generalized Bayesian Committee Machine http://www.kernel-machines.org/publications/Tresp00b In this paper we introduce the Generalized Bayesian Committee Machine (GBCM) for applications with large data sets. In particular, the GBCM can be used in the context of kernel based systems such as smoothing splines, kriging, regularization networks and Gaussian process regression which —for computational reasons— are otherwise limited to rather small data sets. The GBCM provides a novel and principled way of combining estimators trained for regression, classification, the prediction of counts, the prediction of lifetimes and other applications which can be derived from the exponential family of distributions. We describe an online version of the GBCM which only requires one pass through the data set and only requires the storage of a matrix of the dimension of the number of query or test points. After training, the prediction at additional test points only requires resources dependent on the number of query points but is independent of the number of training data. We confirm the good scaling behavior using real and experimental data sets.No publisheradmin2007-01-31T10:09:09ZInproceedings ReferenceMixtures of Gaussian Processes http://www.kernel-machines.org/publications/Tresp01 We introduce the mixture of Gaussian processes (MGP) model which is useful for applications in which the optimal bandwidth of a map is input dependent. The MGP is derived from the mixture of experts model and can also be used for modeling general conditional probability densities. We discuss how Gaussian processes —in particular in form of Gaussian process classification, the support vector machine and the MGP model— can be used for quantifying the dependencies in graphical models.No publisheradmin2007-01-31T10:09:09ZInproceedings ReferenceGeneralization properties of finite-size polynomial support vector machines http://www.kernel-machines.org/publications/RisGor00 No publisheradmin2007-01-31T10:09:09ZArticle ReferenceOn the influence of the kernel on the generalization ability of support vector machines http://www.kernel-machines.org/publications/Steinwart01 In this article we study the generalization abilities of several classifiers of support vector machine type. Our considerations are based on an investigation of certain approximation properties of the used kernels which also gives a new insight into the role of kernels in these and other algorithms. For deterministic supervisors we derive estimates on the generalization performance which are asymptotically sharper than all known results. Moreover, for supervisors which are corrupted by a certain kind of noise we show that the support vector approach yields acceptable generalization results.No publisheradmin2007-01-31T10:09:10ZTechreport Reference