BayesianMethodsforNonlinearClassificationandRegressionPDF清晰版

Bayesian Methods for Nonlinear Classification and Regression PDF 清晰版David G T Denison et alWILEY SERIES IN PROBABILITY AND STATISTICSEstablished by WaLter A shewhart and samuel s wilksEditors: Peter Bloomfield, Noel A. C. Cressie, Nicholas 1. Fisher,lain M. Johnstone, J.B. Kadane, Louise M. Ryan, David W. ScottAdrian F M. Smith, Jozef L. teugelsEditors Emeritus: Vic Barnett, Ralph A. Bradley, J. Stuart Hunter,David g. KendallA complete list of the titles in this series appears at the end of this volumeBayesian Methods for NonlinearClassification and regressionDavid g. T Denison and christopher C. holmesImperial College of science, Technology and Medicine, UKBani k. mallickTexas a&M University, USAAdrian F.m. smithQueen Mary, University of london, UKJohn WILEY SONS, LtDCopyright O 2002 John Wiley Sons Ltd,. The Atrium, Southern Gate, Chichester,West Sussex Po19 8SQ. EnglandTelephone(+4)1243779777Email (for orders and customer service enquiries): cS-books(@wiley. co ukVisitourHomePageonwww.wileyeurope.comorwww.wiley.comReprinted February 2004, March 2005All Rights Reserved. No part of this publication may be reproduced, stored in a retrievalsystem or transmitted in any form or by any means, electronic, mechanical, photocopyingrecording, scanning or otherwise, except under the terms of the Copyright, Designs andPatents Act 1988 or under the terms of a licence issued by the Copyright Licensing AgencyLtd, 90 Tottenham Court Road, London WIT 4LP, UK, without the permission in writing ofthe Publisher. Requests to the Publisher should be addressed to the permissions departmentJohn Wiley Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex Po198SQEngland or emailed to permreq@wiley. co uk, or faxed to (+44)1243770571This publication is designed to provide accurate and authoritative information in regard tothe subject matter covered. It is sold on the understanding that the Publisher is not engaged inrendering professional services. If professional advice or other expert assistance is required,the services of a competent professional should be soughtOther wiley Editorial oficesJohn wiley sons Inc, 111 River Street, Hoboken NJ 07030. USAJossey-Bass, 989 Market Street, San francisco, Ca94103-174l, USAWiley-VCH Verlag GmbH, Boschstr. 12. D-69469 Weinheim, GermanyJohn Wiley Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australiaohn Wiley Sons(Asia )Pte Ltd, 2 Clementi Loop #02-01, Jin Xing Distripark, Singapore129809John Wiley Sons Canada ltd, 22 Worcester road, Etobicoke, Ontario, Canada m9W IlBritish Library Cataloguing in Publication Dataa catalogue record for this book is available from the british LibraryISBN 0471 490369(Hardback)IsBn 9780 471 49036 4(Hardback)Produced from files supplied by the authors, typeset by t&T Productions Ltd, LondonContentsPrefaceAcknowledgements1 Introduction1.1 Regression and Classification1.2 Bayesian Nonlinear methods1.2.1Aapproximatingfunctions1.2.2 The 'best'model1.2.3 Bayesian methods1.3 Outline of the book2 Bayesian Modelling2.1 Introduction2.2 Data Modelling455999902.2. 1 The representation theorem for classification2.2.2 The general representation theorem2.2.3 Bayes'Theorem2.2.4 Modelling with predictors122.3 Basics of Regression modelling142.3. 1 The regression problem143.2 Basis function models for the regression function142.4 The Bayesian Linear Model152. 4. 1 The priors162.4.2 The likelihood172.4.3 The posterior172.5 Model Comparison182.5.1 Bayes’ factors192.5.2 Occam's razor202.5.3 Lindleys paradox22CONTENTS2.6 Model selection242.6.1 Searching for models257 Model averaging282.7.1 Predictive inference282.7.2 Problems with model selection302.7.3 Other work on model averaging2.8 Posterior Samplin312.8.1The Gibbs sampler2.8.2 The Metropolis-Hastings algorithm342.8.3 The reversible jump algorithm362.8.4 Hybrid sampling392.8.5 Convergence402. 9 Further reading412.10Prbl3 Curve Fitting453.1 Introduction453.2 Curve Fitting Using Step Functions463.2. 1 Example: Nile discharge d463.3 Curve Fitting with Splines513.3.1MeiS-Hastings sampler533. 2 Gibbs sampling563.3.3 Example: Great BReef dat573.3.4 Monitoring convergence of the sampler6033. 5 Default curve fitting633.4 Curve Fitting using wavelets3.4.1 Wavelet shrinkage3.4.2 Bayesian wavelets703.5 Prior elicitation723.5.1 The model prior3.5.2 Prior on the model parameters3.5.3 The prior on the coefficients3.5.4 The prior on the regression variance823.6 Robust Curve fitting823. 6. 1 Modelling with a heavy-tailed error distribution833.6.2 Outlier detection models3.7 Discussion8838Further Reading893.9 Problems914 Surface Fitting954.1 Introduction95CONTENTS4.2 Additive models954.2. 1 Introduction to additive modelling954.2.2 Ozone data example984.2.3 Further reading on bayesian additive models994.3 Higher-Order splines1004.3. 1 Truncated linear splines4.4 High-Dimensional Regression1024.4.1 Extending to higher dimension1024.4.2 The bwise model1034.4.3 The bars model1034.4.4 Piecewise linear models1104.4.5 Neural network models1154.5 Time Series analysis1194.5.1 The baYStar model12l4.5.2 Example: Wolfs sunspots data1224.5.3 Chaotic Time Series1244.6 Further Reading1264.7 Problems1265 Classification Using Generalised Nonlinear Models1295.1 Introduction1295.2 Nonlinear models for Classification1305.2.1 Classification1305.2.2 Auxiliary variables method for classification1325.3 Bayesian MARS for Classification1365.3.1 Multiclass classification1375.4 Count Data1385.4.1 Example: Rongelap island dataset1405.5 The Generalised Linear Model framework14l5.5.1 Bayesian generalised linear models1445.5.2Log-concavity1445.6 Further reading1455.7 Problems1466 Bayesian Tree Models1496.1 Introduction6.1.1 Motivation for trees1506. 1.2 Binary-tree structure1506.2 Bayesian Trees1526.2. 1 The random tree structure1526.2.2 Classification trees1536.2.3 Regression trees1556.2.4 Prior on trees156CONTENTS6.3 Simple trees1586.3.1 Stum1596.3.2 A Bayesian splitting criterion1606.4 Searching for large trees1616.4.1 The sampling algorith16l6.4.2 Problems with samplin6.4.3 Improving the generated'sample1656.5 Classification USing Bayesian Trees1666.5.1 The Pima Indian dataset1666.5.2 Selecting trees from the sample1676.5.3 Summarising the output1676.5.4 Identifying good trees6.6D1706.7 Further Reading1746. 8 Problems1757 Partition models1777.1 Introduction1777.2 One-Dimensional Partition models1797.2.1 Changepoint models7.3 Multidimensional partition models1847.3.1 Tessellations1847.3.2 Marginal likelihoods for partition models1867. 3. 3 Prior on the model structure1877.3.4Computational strategy1887.4 Classification with Partition Models1887.4. 1 Speech recognition dataset1887.5 Disease Mapping with Partition Models1917.5Introduction197.5.2 The disease mapping problem1927.5.3 The binomial model for disease risk1927.5.4 The poisson model for disease risk1937.5.5 Example: leukaemia incidence data1937.5.6 Convergence assessment1957.5.7 Posterior inference for the leukaemia data1977.6 Discussion997.7 Further reading2037. 8 Problems2068 Nearest-Neighbour Models2098.1 Introduction2098.2 Nearest-Neighbour Classification209CONTENTS8.3 Probabilistic Nearest Neighbour8.3.1 Formulation8.3.2 Implementation2138.4Exampl48.4.1 Ripley's simulated data2148.4.2 Arm tremor data2168.4.3 Lancing Woods data2178.5 Discussion2198.6 Further Reading2209 Multiple response models2211 Introduction2219.2 The Multiple Response Model2219.3 Conjugate Multivariate Linear regression2229.4 Seemingly Unrelated Regressions2239.4.1 Prior on the basis function matrix2269.5 Computational details2279.5.1 Updating the parameter vector 82279.6 Examples2289.6. 1 Vector autoregressive processes2299.6.2 Multiple curve fitting2309.7 Discussion234Appendix a Probability Distributions237Appendix b Inferential Processes239B I The linear model240B 2 Multivariate Linear model241B3 Exponential-Gamma Model242B. 4 The Multinomial-Dirichlet Model243B 5 Poisson-Gamma Model244B 6 Uniform-Pareto model245References247Index265Author index271