Mehr lesen
Informationen zum Autor Vladimir CherKassky, PhD, is Professor of Electrical and Computer Engineering at the University of Minnesota. He is internationally known for his research on neural networks and statistical learning. Filip Mulier, PhD, has worked in the software field for the last twelve years, part of which has been spent researching, developing, and applying advanced statistical and machine learning methods. He currently holds a project management position. Klappentext An interdisciplinary framework for learning methodologies-now revised and updatedLearning from Data provides a unified treatment of the principles and methods for learning dependencies from data. It establishes a general conceptual framework in which various learning methods from statistics, neural networks, and pattern recognition can be applied-showing that a few fundamental principles underlie most new methods being proposed today in statistics, engineering, and computer science.Since the first edition was published, the field of data-driven learning has experienced rapid growth. This Second Edition covers these developments with a completely revised chapter on support vector machines, a new chapter on noninductive inference and alternative learning formulations, and an in-depth discussion of the VC theoretical approach as it relates to other paradigms.Complete with over one hundred illustrations, case studies, examples, and chapter summaries, Learning from Data accommodates both beginning and advanced graduate students in engineering, computer science, and statistics. It is also indispensable for researchers and practitioners in these areas who must understand the principles and methods for learning dependencies from data. Zusammenfassung An interdisciplinary framework for learning methodologies, covering statistics, neural networks, and fuzzy logic, Learning from Data provides a unified treatment of the principles and methods for learning dependencies from data. Inhaltsverzeichnis PREFACE. NOTATION. 1 Introduction. 1.1 Learning and Statistical Estimation. 1.2 Statistical Dependency and Causality. 1.3 Characterization of Variables. 1.4 Characterization of Uncertainty. 1.5 Predictive Learning versus Other Data Analytical Methodologies. 2 Problem Statement, Classical Approaches, and Adaptive Learning. 2.1 Formulation of the Learning Problem. 2.1.1 Objective of Learning. 2.1.2 Common Learning Tasks. 2.1.3 Scope of the Learning Problem Formulation. 2.2 Classical Approaches. 2.2.1 Density Estimation. 2.2.2 Classification. 2.2.3 Regression. 2.2.4 Solving Problems with Finite Data. 2.2.5 Nonparametric Methods. 2.2.6 Stochastic Approximation. 2.3 Adaptive Learning: Concepts and Inductive Principles. 2.3.1 Philosophy, Major Concepts, and Issues. 2.3.2 A Priori Knowledge and Model Complexity. 2.3.3 Inductive Principles. 2.3.4 Alternative Learning Formulations. 2.4 Summary. 3 Regularization Framework. 3.1 Curse and Complexity of Dimensionality. 3.2 Function Approximation and Characterization of Complexity. 3.3 Penalization. 3.3.1 Parametric Penalties. 3.3.2 Nonparametric Penalties. 3.4 Model Selection (Complexity Control). 3.4.1 Analytical Model Selection Criteria. 3.4.2 Model Selection via Resampling. 3.4.3 Bias-Variance Tradeoff. 3.4.4 Example of Model Selection. 3.4.5 Function Approximation versus Predictive Learning. 3.5 Summary. 4 Statistical Learning Theory. 4.1 Conditions for Consistency and Convergence of ERM. 4.2 Growth Function and VC Dimension. 4.2.1 VC Dimension for Classification and Regression Problems. 4.2.2 Examples of Calculating VC Dimension. 4.3 Bounds on t...
Inhaltsverzeichnis
Preface.
Notation.
1. Introduction.
1.1 Learning and Statistical Estimation.
1.2 Statistical Dependency and Causality.
1.3 Characterization of Variables.
1.4 Characterization of Uncertainty.
References.
2. Problem Statement, Classical Approaches, and Adaptive Learning.
2.1 Formulation of the Learning Problem.
2.2 Classical Approaches.
2.3 Adaptive Learning: Concepts and Inductive Principles.
2.4 Summary.
References.
3. Regularization Framework.
3.1 Curse and Complexity of Dimensionality.
3.2 Function Approximation and Characterization of Complexity.
3.3 Penalization.
3.4 Model Selection (Complexity Control).
3.5 Summary.
References.
4. Statistical Learning Theory.
4.1 Conditions for Consistency and Convergence of ERM.
4.2 Growth Function and VC-Dimension.
4.3 Bounds on the Generalization.
4.4 Structural Risk Minimization.
4.5 Comparisons of Model Selection for Regression.
4.6 Measuring the VC-dimension.
4.7 Summary and Discussion.
References.
5. Nonlinear Optimization Strategies.
5.1 Stochastic Approximation Methods.
5.2 Iterative Methods.
5.3 Greedy Optimization.
5.4 Feature Selection, Optimization, and Statistical Learning Theory .
5.5 Summary.
References.
6. Methods for Data Reduction and Dimensionality Reduction.
6.1 Vector Quantization.
6.2 Dimensionality Reduction: Statistical Methods.
6.3 Dimensionality Reduction: Neural Network Methods.
6.4 Methods for Multivariate Data Analysis.
6.5 Summary.
References.
7. Methods for Regression.
7.1 Taxonomy: Dictionary versus Kernel Representation.
7.2 Linear Estimators .
7.3 Adaptive Dictionary Methods.
7.4 Adaptive Kernel Methods and Local Risk Minimization.
7.5 Empirical Studies.
7.6 Combining Predictive Models.
7.7 Summary.
References.
8. Classification.
8.1 Statistical Learning Theory Formulation.
8.2 Classical Formulation.
8.3 Methods for Classification.
8.4 Combining Methods and Boosting.
8.5 Summary.
References.
9. Support Vector Machines.
9.1 Motivation for margin-based loss.
9.2 Margin-based loss, robustness and complexity control.
9.3 Optimal Separating Hyperplane .
9.4 High-Dimensional Mapping and Inner Product Kernels.
9.5 Support Vector Machine for Classification.
9.6 Support Vector Implementations.
9.7 Support Vector Machine for Regression.
9.8 SVM Model Selection.
9.9 SVM vs regularization approach.
9.10 Single-class SVM and novelty detection.
9.11 Summary and discussion.
References.
10. Non-Inductive Inference and Alternative Learning Formulations.
10.1 Sparse High-Dimensional Data.
10.2 Transduction .
10.3 Inference Through Contradictions.
10.4 Multiple Model Estimation.
10.5 Summary.
References.
Appendix A: Review of Nonlinear Optimization.
Appendix B: Eigenvalues and Singular Value Decomposition.
Index.
Bericht
"The authors have succeeded in summarizing some of the recent trends and future challenges in different learning methods, including enabling technologies and some interesting practical applications." ( Computing Reviews, May 22, 2008)
