Maximum Penalized Likelihood Estimation - 1: Maximum Penalized Likelihood Estimation

This book is intended for graduate students in statistics and industrial mathematics, as well as researchers and practitioners in the field. We cover both theory and practice of nonparametric estimation. The text is novel in its use of maximum penalized likelihood estimation, and the theory of convex minimization problems (fully developed in the text) to obtain convergence rates. We also use (and develop from an elementary view point) discrete parameter submartingales and exponential inequalities. A substantial effort has been made to discuss computational details, and to include simulation studies and analyses of some classical data sets using fully automatic (data driven) procedures. Some theoretical topics that appear in textbook form for the first time are definitive treatments of I.J. Good's roughness penalization, monotone and unimodal density estimation, asymptotic optimality of generalized cross validation for spline smoothing and analogous methods for ill-posed least squares problems, and convergence proofs of EM algorithms for random sampling problems.

List of contents

Parametric Maximum Likelihood Estimation.- Parametric Maximum Likelihood Estimation in Action.- Kernel Density Estimation.- Maximum Likelihood Density Estimation.- Monotone and Unimodal Densities.- Choosing the Smoothing Parameter.- Nonparametric Density Estimation in Action.- Convex Minimization in Finite Dimensional Spaces.- Convex Minimization in Infinite Dimensional Spaces.- Convexity in Action.

Summary

This book is intended for graduate students in statistics and industrial mathematics, as well as researchers and practitioners in the field. We cover both theory and practice of nonparametric estimation. The text is novel in its use of maximum penalized likelihood estimation, and the theory of convex minimization problems (fully developed in the text) to obtain convergence rates. We also use (and develop from an elementary view point) discrete parameter submartingales and exponential inequalities. A substantial effort has been made to discuss computational details, and to include simulation studies and analyses of some classical data sets using fully automatic (data driven) procedures. Some theoretical topics that appear in textbook form for the first time are definitive treatments of I.J. Good's roughness penalization, monotone and unimodal density estimation, asymptotic optimality of generalized cross validation for spline smoothing and analogous methods for ill-posed least squares problems, and convergence proofs of EM algorithms for random sampling problems.

Additional text

From the reviews:

"…A highly readable and appealing book…In a world of dry prose, this book is a refreshing change…The book is enjoyable to read, which alone merits praise." Journal of the American Statistical Association

"This is a theoretical work, but the authors always keep the practical aspect in mind. Algorithmic issues are treated with great care. In fact, very interesting chapters, demonstrating the main techniques at work on simulated and real data, complement the theoretical treatment. The monograph is highly recommended for teaching advanced courses on nonparametric statistics. This book is a must for anyone who is serious about nonparametric curve estimation." SIAM Reviews

"The basic tools needed are introduced in the book itself, proofs are complete, partly using the many exercises which are added. The text contains an impressive list of references. … the variety of ideas and approaches is also an advantage of the book since one can learn quite different approaches and techniques from it. … Even specialists may find some new aspects. Certainly the book belongs to the bookshelf of researchers and advanced students being interested in the subject." (Ulrich Stadtmüller, Metrika, July, 2003)

"Throughout the book, applications and the practical performance of the theoretical results are studied. … The book provides a good and up-to-date introduction to nonparametric density estimation. One of its main strengths is giving overviews and motivations of the general ideas before moving on to the technicalities. This, together with the in-action chapters, makes it an excellent text-book for graduate students in statistics, as well as practitioners in the field." (Pia Veldt Larsen, Journal of the Royal Statistical Society Series A: Statistics in Society, Vol. 157 (2), 2004)

"The selection of important topics has been made with excellent taste. The authors’ entertaining style of writing, rare in mathematical texts, makes the book a pleasure to read. The authors are never afraid of giving their opinion explicitly on the beauty, difficulty, and importance of the discussed issues. … The monograph is highly recommended for teaching advanced courses on nonparametric statistics. This book is a must for anyone who is serious about nonparametric curve estimation." (Gábor Lugosi, SIAM Review, Vol. 45 (2), 2003)

"This well written book gives a nice mathematical treatment of parametric and nonparametric maximum likelihood estimation, mainly in the context of density estimation. In addition to these main parts there is a final section on convexity and optimization. … This broader and unifying view is indeed an asset compared to earlier monographs on the above mentioned topics." (Jan Beirlant, Mathematical Reviews, Issue 2002 j)

"The mathematical level is quite high, but most of the required tools, like martingales, exponential inequalities, Fourier analysis, Banach spaces, etc. are explained in the text. An interesting feature of the book is also that each part ends with an ‘in action’ chapter in which the estimation procedures are put to work and small sample performance is discussed. The book can be used for classes and seminars, particularly because of the presence of numerous exercises and tasks." (N. D. C. Veraverbeke, Short Book Reviews, Vol. 22 (1), 2002)

Report

From the reviews:
"...A highly readable and appealing book...In a world of dry prose, this book is a refreshing change...The book is enjoyable to read, which alone merits praise." Journal of the American Statistical Association
"This is a theoretical work, but the authors always keep the practical aspect in mind. Algorithmic issues are treated with great care. In fact, very interesting chapters, demonstrating the main techniques at work on simulated and real data, complement the theoretical treatment. The monograph is highly recommended for teaching advanced courses on nonparametric statistics. This book is a must for anyone who is serious about nonparametric curve estimation." SIAM Reviews
"The basic tools needed are introduced in the book itself, proofs are complete, partly using the many exercises which are added. The text contains an impressive list of references. ... the variety of ideas and approaches is also an advantage of the book since one can learn quite different approaches and techniques from it. ... Even specialists may find some new aspects. Certainly the book belongs to the bookshelf of researchers and advanced students being interested in the subject." (Ulrich Stadtmüller, Metrika, July, 2003)
"Throughout the book, applications and the practical performance of the theoretical results are studied. ... The book provides a good and up-to-date introduction to nonparametric density estimation. One of its main strengths is giving overviews and motivations of the general ideas before moving on to the technicalities. This, together with the in-action chapters, makes it an excellent text-book for graduate students in statistics, as well as practitioners in the field." (Pia Veldt Larsen, Journal of the Royal Statistical Society Series A: Statistics in Society, Vol. 157 (2), 2004)
"The selection of important topics has been made with excellent taste. The authors' entertaining style of writing, rare in mathematical texts, makes the book a pleasure to read. The authors are never afraid of giving their opinion explicitly on the beauty, difficulty, and importance of the discussed issues. ... The monograph is highly recommended for teaching advanced courses on nonparametric statistics. This book is a must for anyone who is serious about nonparametric curve estimation." (Gábor Lugosi, SIAM Review, Vol. 45 (2), 2003)
"This well written book gives a nice mathematical treatment of parametric and nonparametric maximum likelihood estimation, mainly in the context of density estimation. In addition to these main parts there is a final section on convexity and optimization. ... This broader and unifying view is indeed an asset compared to earlier monographs on the above mentioned topics." (Jan Beirlant, Mathematical Reviews, Issue 2002 j)
"The mathematical level is quite high, but most of the required tools, like martingales, exponential inequalities, Fourier analysis, Banach spaces, etc. are explained in the text. An interesting feature of the book is also that each part ends with an 'in action' chapter in which the estimation procedures are put to work and small sample performance is discussed. The book can be used for classes and seminars, particularly because of the presence of numerous exercises and tasks." (N. D. C. Veraverbeke, Short Book Reviews, Vol. 22 (1), 2002)

Product details

• Authors: P P Eggermont, P P B Eggermont, P. P. B. Eggermont, P.P.B. Eggermont, Paulus P. B. Eggermont, V N LaRiccia, V. N. LaRiccia, V.N. LaRiccia, Vincent N Lariccia, Vincent N. LaRiccia
• Publisher: Springer, Berlin
• Languages: English
• Product format: Hardback
• Released: 25.07.2001
• EAN: 9780387952680
• ISBN: 978-0-387-95268-0
• No. of pages: 512
• Weight: 878 g
• Illustrations: XVIII, 512 p.
• Sets: Maximum Penalized Likelihood Estimation; Maximum Penalized Likelihood Estimation
• Series: Springer Series in Statistics; Springer Texts in Statistics; Springer Texts in Statistics; Springer Series in Statistics; Springer Statistics
• Subject: Natural sciences, medicine, IT, technology > Mathematics > Probability theory, stochastic theory, mathematical statistics