A Course in Large Sample Theory

A Course in Large Sample Theory, 大样本的性质，作者是Thoms S.FergusonContentsPrefacePart 1 Basic ProbabilityModes of ConvergencePartial Converses to Theorem 12345Convergence in Law3839Laws of Large NumbersCentral Limit Theorems26Part 2 Basic Statistical Large Sample Theory37Slutsky Theorems67890Functions of the Sample MomentsThe Sample correlation Coefficient51Pearsons Chi-Square56Asymptotic Power of the Pearson Chi-Square Test61Part 3 Special Topics67Stationary m-Dependent Sequences6912Some rank statistics7513Asymptotic Distribution of Sample Quantiles87Asymptotic Theory of Extreme Order Statistics94Asymptotic Joint Distributions of Extrema101Part 4 Efficient Estimation and Testing10516A Uniform Strong Law of Large Numbers107Strong Consistency of Maximum-Likelihood Estimates112Contents18Asymptotic Normality of the Maximum-LikelihoodEstimate11919The Cramer-Rao Lower bound12620Asymptotic Efficiency13321Asymptotic Normality of Posterior Distributions22Asymptotic Distribution of the likelihood ratioTest Statistic14423Minimum Chi-Square Estimates15124General Chi-Square Tests163Appendix: solutions to the exercises172References236Index239PrefaceThe subject area of mathematical statistics is so vast that in undergraduatecourses there is only time enough to present an overview of the materialIn particular, proofs of theorems are often omitted, occasionally with areference to specialized material, with the understanding that proofs willbe given in later, presumably graduate, courses. Some undergraduate textscontain an outline of the proof of the central limit theorem, but othertheorems useful in the large sample analysis of statistical problems areusually stated and used without proof. Typical examples concern topicssuch as the asymptotic normality of the maximum likelihood estimate theasymptotic distribution of Pearsons chi-square statistic, the asymptoticdistribution of the likelihood ratio test, and the asymptotic normality ofthe rank-sum test statisticBut then in graduate courses, it often happens that proofs of theorems areassumed to be given in earlier, possibly undergraduate, courses, or proofs aregiven as they arise in specialized settings. Thus the student never learns in ageneral methodical way one of the most useful areas for research in statisticslarge sample theory, or as it is also called, asymptotic theory. There is ameed for a separate course in large sample theory at the beginning graduatelevel. It is hoped that this book will help in filling this needA course in large sample theory has been given at uCla as the secondquarter of our basic graduate course in theoretical statistics for about twentyyears. The students who have learned large sample theory by the route givenin this text can be said to form a large sample. Although this course is givenin the Mathematics Department, the clients have been a mix of graduatestudents from various disciplines. Roughly 40% of the students have beenfrom Mathematics, possibly 30% from Biostatistics, and the rest fromBiomathematics, Engineering, Economics, Business, and other fields. ThePrefacestudents generally find the course challenging and interesting, and have oftencontributed to the improvement of the course through questions, suggestionsand, of course, complaintsBecause of the mix of students, the mathematical background requiredfor the course has necessarily been restricted In particular, it could not beassumed that the students have a background in measure-theoretic analysisor probability. However, for an understanding of this book, an under-graduate couise in analysis is needed as well as a good undergraduate coursein mathematical statisticsStatistics is a multivariate discipline. Nearly, every useful univariate problem has important multivariate extensions and applications. For this reasonnearly all theorems are stated in a multivariate setting. Often the statementof a multivariate theorem is identical to the univariate version but when it isnot, the reader may find it useful to consider the theorem carefully in onedimension first, and then look at the examples and exercises that treat problems in higher dimensionsThe material is constructed in consideration of the student who wants tolearn techniques of large sample theory on his/her own without the benefitof a classroom environment. There are many exercises, and solutions to allexercises may be found in the appendix. For use by instructors, other exer-cises,without solutions, can be found on the web page for the course,athttp://www.stat.uclaedu/courses/graduate/m276b/.Each section treats a specific topic and the basic idea or central result ofthe section is stated as a theorem. There are 24 sections and so there are 24theorems. The sections are grouped into four parts. In the first part, basicnotions of limits in probability theory are treated including laws of largenumbers and the central limit theorem. In the second part, certain basic toolsin statistical asymptotic theory such as Slutsky's Theorem and Cramer'sTheorem, are discussed and illustrated, and finally used to derive the asymptotic distribution and power of Pearson's chi-square. In the third part,certain special topics are treated by the methods of the first two parts, suchas some time series statistics, some rank statistics. and distributions ofquantiles and extreme order statistics. The last part contains a treatment ofstandard statistical techniques including maximum likelihood estimationthe likelihood ratio test, asymptotic normality of Bayes estimates, andminimum chi-square estimation. Parts 3 and 4 may be read independentlyThere is easily enough material in the book for a semester course In a quartercourse, some material in parts 3 and 4 will have to be omitted or skimmedI would like to acknowledge a great debt this book owes to lucien leCam not only for specific details as one may note in references to him inthe text here and there, but also for a general philosophic outlook on thePrefacesubject. Since the time I learned the subject from him many years ago,hehas developed a much more general and mathematical approach to thesubject that may be found in his book, Le cam(1986)mentioned in the referencesRudimentary versions of this book in the form of notes have been inexistence for some 20 years, and have undergone several changes in computersystems and word processors. I am indebted to my wife, Beatriz, for cheerfully typing some of these conversions. Finally, I am indebted to my students,too numerous to mention individually. Each class was distinctive and eachclass taught me something new so that the next year's class was taughtsomewhat differently than the last. If future students find this book helpfulthey also can thank these students for their contribution to making it understandableThomas S. Ferguson, april 1996Basic Probability TheoryModes of ConvergenceWe begin by studying the relationships among four distinct modes ofconvergence of a sequence of random vectors to a limit. All convergencesare defined for d-dimensional random vectors. For a random vectorX=(X1,., XdER, the distribution function of X, defined for x(x1,…,xd)∈Rd, is denoted by Fx(x)=P(xX≤x)=P(K1≤x1,…,Xd sxd). The Euclidean norm of x=(xi,., xd)eR is denoted byxI=(xi+.+x2)/2.Let X,X, X2,... be random vectors with valuesInRDEFINITION 1. X, converges in law to X,X-oxx)→F2(x)X if F(asn=o0, for all points x at which Fx(x)is continuous.Convergence in law is the mode of convergence most used in thefollowing chapters It is the mode found in the Central Limit Theorem andis sometimes called convergence in distribution, or weak convergenceEXAMPLE 1. We say that a random vector X e r is degenerate at a pointcER if P(X=c)=1. Let X, R be degenerate at the point 1/n,forn=1, 2,... and let X Er be degenerate at 0. Since 1/n converges tozero as n tends to infinity, it may be expected that X, X. This may beseen by checking Definition 1. The distribution function of X, is Fr(x)H u/m, o(x), and that of X is Fx(x)=lro, o(), where IA()denotes theindicator function of the set A (i.e, IA(x) denotes 1 if x A, and 0otherwise). Then Fx(x)- Fx(x) for all x except x=0, and for x=0 wehave Fx0=0+ Fx(0)=1. But because Fx(x) is not continuous atx=0, we nevertheless have X,- X from Definition 1. This shows theA Course in Large Sample Theoryneed, in the definition of convergence in law, to exclude points x at which(x is not continuousDEFINITION 2.X, converges in probability to X,X,X, if for everyε>0,P{Xn-X>e}→0asn→∞DEFINITION 3. For a real number r>0, x, converges in the rth mean to xX,fE|X,-X|→0asn→∞DEFINITION 4. X, converges almost surely to x,XnX, ifPlimnxn=x=1Almost sure convergence is sometimes called convergence with probabiity 1(wp. 1)or strong convergence. In statistics, convergence in the rthmean is most useful for2, when it is called convergence in quadraticnean and is written x. gm,X. The basic relationships are as follows.THEOREM 1a)XnX→Xn→X.b)Xn→ X for some r>0→Xn→X)XX→XTheorem 1 states the only universally valid implications between thevarious modes of convergence, as the following examples showEXAMPLE 2. To check convergence in law, nothing needs to be knownabout the joint distribution of X and X, whereas this distribution must bedefined to check convergence in probability. For example, if X1,X2are independent and identically distributed (ii. d. )normal random variables, with mean 0 and variance 1, then X,Xi, yet X,+ XiEXAMPLE 3. Let Z be a random variable with a uniform distribution onthe interval (0,1),Z E 2(0, 1), and let X:=1,X2=Ir0, 1/2)(Z),X3=i1/2Z)x4=l14(2,x5=ln1/4,1/2(Z),…, In general,if2+ m, where 0 sm< 2k and k20, then Xn=Im2-k, (m +12-k(Z)Then Xn does not converge for any ZE [0, 1), so X, +0. Yet X,0for all!r>0 and X→0.