Extreme-Value Moment Goodness-of-Fit Tests

Theodore P. Hill^{1} School of Mathematics, Georgia Institute of Technology Atlanta, GA 30332 USA

Victor Perez-Abreu^{2} Centro de Investigacíon en Matemáticas Guanajuato, Gto 36000 Mexico

Abstract A general goodness-of-fit test for scale-parameter families of distributions is intro-

duced which is based on quotients of expected sample minima. The test is independent of the mean of the distribution, and, in applications to testing for exponentiality of data, compares favorably to other goodness-of-fit tests for exponentiality based on the empirical distribution function, regression methods and correlation statistics. The new minimalmoment method uses ratios of easily-calculated, unbiased, strongly consistent U-statistics, and the general technique can be used to test many standard composite null hypotheses such as exponentiality, normality or uniformity (as well as simple null hypotheses).

Key Words and Phrases: Goodness-of-fit, scale-parameter families, U-statistics, exponential family, composite exponential hypothesis, minimal moments, extreme values. AMS 1991 subject classification: 62G10

1 Research partially supported by NSF Grant 95-03375.

2 Research partially supported by CONACYT Grant 0328PE9506.

§1. Introduction

Deciding whether given data comes from a particular class of probability distributions is one of the basic problems of statistics. For example, given a random sample X_{1}, X_{2}, …, X_{n} from an unknown distribution F, it is often desired to test for exponentiality of the data, i.e., to test the null hypothesis

(1)  H_{0} : F ∈ {exp(β) : β > 0},

(where exp(β) denotes the exponential distribution with density βe^{−βx} for x > 0).

Alternatively, it may be desired to test for uniformity, normality, or other general properties of the underlying distribution, and many goodness-of-fit tests for such composite null hypotheses are available (cf. D’Agostino and Stephens (1986)).

It is the purpose of this article to introduce a general test for scale-parameter families, that is, to test the null hypothesis H_{0} : F ∈ F, where F = {F_{β}, β > 0} is a family of distributions satisfying F_{β}(x) = F_{1}(βx) for all β > 0. Such scale-parameter families include the class of exponential distributions in (1), as well as many other families such as uniform (F_{β} ∼ U[0, β]), and normal (F_{β} ∼ N(0, β^{2})). (Note that neither {Poisson(β)} nor {N(β, 1)} are scale-parameter families; scale-invariance says that the random variable X has law in the family if and only if βX also does for all β > 0.)

The test described below is based on the expected value of sample maxima or minima, for which classical U-statistics are standard estimators, and is based on a representation theorem of Hoeffding, on the linearity of expectations of sample extrema, and on the classical U-statistic strong law of large numbers and law of iterated logarithm.

§2. Extreme-Value Moments

Throughout this article, X, X_{1}, X_{2}, … are iid random variables with finite mean EX and distribution F. The minimal and maximal moments of F are

m̂_{k}(F) = E(X_{1} ∧ ⋯ ∧ X_{k})

m^{∨}_{k}(F) = E(X_{1} ∨ X_{2} ∨ ⋯ ∨ X_{k}),

where k is a positive integer, a ∧ b = min{a, b}, a ∨ b = max{a, b}, and E(·) denotes expectation with respect to the underlying probability. Thus, for example, m̂_{1}(F) = m^{∨}_{1}(F) = EX; and m̂_{1}(F) = m̂_{2}(F) iff F is degenerate (Dirac point mass). Also note that in contrast to the classical moment (E(X^{k})) framework, finiteness of EX implies that of both m̂_{k} and m^{∨}_{k} for all k, both {m̂_{k}} and {m^{∨}_{k}} are o(k) with decreasing o(1) difference sequences {|m̂_{k} − m̂_{k+1}|}, (cf. Hill and Spruill (1994), Lemma 2.2) and both are linear

(2) if G = law(aX + b), then m̂_{k}(G) = am̂_{k}(F) + b and m^{∨}_{k}(G) = am^{∨}_{k}(F) + b for all k ∈ IN.

A well-known result of Hoeffding (cf. Pollack (1973)) which plays an essential role in this article is that

(3) the sequences {m̂_{k}(F)} and {m^{∨}_{k}(F)} each determine F (and vice versa).

(Much more is true: even Müntz subsequences {k_{j} : Σ^{1}/_{k_{j}} = ∞} of {m̂_{k}} and {m^{∨}_{k}} (Hill and Spruill (1994)) and moment sequences of non-extremal order statistics (Pollack (1973) also determine F.)

Hoeffding’s result (3) and easy calculations imply, for example, that

(4)  m̂_{k}(F) = \frac{1}{βk} for all k ∈ IN iff F ∼ exp(β);

and

(5)  m^{∨}_{k}(F) = \frac{βk}{k + 1} for all k ∈ IN iff F ∼ U(0, β).

(On the other hand, {m^{∨}_{k}} for N(0, 1) are known in closed form only for k = 1, 2, 3, 4, 5 (cf. David (1981).)

There are many statistics which can be used to estimate m̂_{k}(F) and m^{∨}_{k}(F) from a random sample X_{1}, …, X_{n}; the standard one for m̂_{k} is the U-statistic

(6) M̂_{k}(X_{1}, …, X_{n}) := \frac{(n − k)!}{n!} ∑ min\{X_{i_{1}}, …, X_{i_{k}} : \{i_{j}\} distinct, 1 ≤ i_{j} ≤ n\}

which is both an unbiased and (by Hoeffding’s (1961) strong law of large numbers for U-statistics) a strongly consistent estimator for m̂_{k}, that is,

(7)  M̂_{k}(X_{1}, …, X_{n}) → m̂_{k}(F)  a.s. as n → ∞.

In fact with a second moment hypothesis as well (Var X < ∞), central limit and law of iterated logarithm results also hold for convergence of M̂_{k} to m_{k} (Serfling (1980, pp. 191)).

The L-statistic corresponding to M̂_{k} and m̂_{k}

∑_{i=1}^{n} i^{k}X_{i:n}/ ∑_{i=1}^{n} i^{k}

(cf. Gilat and Hill (1992)) is essentially equivalent to M̂_{k}, with corresponding consistency, CLT and LIL properties.

We observe that a useful computing formula for (6) is

(8)  M̂_{k}(X_{1}, …, X_{n}) := \frac{1}{\binom{n}{k}} ∑_{i=1}^{n−k+1}(n − ik − 1)X_{i:n}.

§3. Quotient Extreme-Value Moment Goodness-of-Fit Tests

The purpose of this section is to use the linearity of the extremal moments (2), Hoeffding’s representation property of the extremal moments (3), and the U-statistic strong law of large numbers (7) to introduce a goodness-of-fit test for scale-parameter families which is based on quotients of the extreme moments and U-statistics. For simplicity only positive variables and the minimal moments {m̂_{k}} will be addressed, and the extension to general variables and to the maximal moments {m^{∨}_{k}} is left to the interested reader.

For each k ∈ IN, let

q_{k}(F) := m̂_{k}(F)/m̂_{k+1}(F)

and

(9)  q_{k}(X_{1}, …, X_{n}) := M̂_{k}(X_{1}, …, X_{n})/M̂_{k+1}(X_{1}, …, X_{n}).

The next theorem gives an extreme-moment characterization of scale-parameter families which is the basis for the test statistics Q_{p,F} defined below.

Theorem 3.1. Suppose X_{1}, X_{2}, … are iid with distribution F, and F = {F_{β}} is a scaleparameter family of (integrable) random variables. Then the following are equivalent:

(i) F = F_{β} for some β > 0; (ii) q_{k}(F) = q_{k}(F_{1}) for all k ∈ IN; (iii) q_{k}(X_{1}, …, X_{n}) → q_{k}(F_{1}) a.s. as n → ∞, for all k ∈ IN.

Proof. “(i) ⇒ (ii).” Suppose F = F_{β} for some β > 0. Then by (2), q_{k}(F) = q_{k}(F_{β}) = βm̂_{k}(F_{1})/βm̂_{k+1}(F_{1}) = q_{k}(F_{1}) for all k ∈ IN.

“(ii) ⇒ (i).” Suppose q_{k}(F) = q_{k}(F_{1}) =: q_{k} for all k ∈ IN, and let μ = m̂_{1}(F) > 0 and λ = m̂_{1}(F_{1}) > 0. Then by the definition of {q_{k}}, m̂_{k+1}(F) = μ/ ∏_{j=1}^{k} q_{j} and m̂_{k+1}(F_{1}) = λ/ ∏_{j=1}^{k} q_{j}, so m̂_{k}(F) = \frac{μ}{λ} m̂_{k}(F_{1}) for all k ∈ IN, which by linearity (2) and Hoeffding’s representation theorem (3) implies that F is in the scale-parameter family containing F_{1}, so F = F_{β} for some β > 0.

“(i) ⇒ (iii).” Suppose F = F_{β} for some β > 0, and fix k ∈ IN. By the U-statistic strong law of large numbers (7), q_{k}(X_{1}, …, X_{n}) → q_{k}(F_{β}) a.s. n → ∞. But by linearity (2) and the definition of {q_{k}}, q_{k}(F_{1}) = q_{k}(F_{β}).

“(iii) ⇒ (ii).” Immediate by (7) and the definition of {q_{k}}.

Corollary 3.2. (Exponential Family Case). Let X_{1}, X_{2}, … be iid with finite positive mean and with distribution F. Then the following are equivalent:

(i) F ∼ exp(β) for some β > 0; (ii) q_{k}(F) = (k + 1)/k for all k ∈ IN; (iii) q_{k}(X_{1}, …, X_{n}) → (k + 1)/k a.s. for all k ∈ IN; (iv) {(m̂_{k}(F))^{−1}} is linear in k.

Proof. Immediate from Theorem 3.1 and the {m̂_{k}} for exponentials in (4).

Test statistics {Q_{p,F} : 1 ≤ p ≤ ∞} will now be defined which measure the discrepancy between the sample quotients (of k-minima) and the true quotients of the scale-parameter family containing F. The distances used are the standard ℓ_{p} distances, and the quotients are simply those of the classical U-statistics estimators (6) of the k-minima {m_{k}(F)}.

Here ∥(x_{1}, …, x_{m})∥_{p} denotes the usual ℓ_{p} norm (∑_{i=1}^{m} |x_{i}|^{p})^{1/p} for 1 ≤ p < ∞, and the sup norm max{|x_{1}|, …, |x_{m}|} for p = ∞. The composite null hypothesis for the random sample to be from the given scale-parameter family (e.g. (1)) will then be accepted iff the discrepancy Q_{p,F} is sufficiently small.

Let {a_{n}} be an increasing sequence of positive integers with a_{n} = o((n/ log log n)^{1/2}).

Definition 3.3. Let F = {F_{α}} be a scale-parameter family, and let X_{1}, X_{2}, …, X_{n} be iid with finite positive mean, and with distribution F. For each p, 1 ≤ p ≤ ∞, let

(10)  Q_{p,F_{1}}(X_{1}, …, X_{n}) := ∥q⃗_{n}(X_{1}, …, X_{n}) − q⃗_{n}(F_{1})∥_{p},

where q⃗_{n}(X_{1}, …, X_{n}) = (q_{1}(X_{1}, …, X_{n}), …, q_{a_{n}}(X_{1}, …, X_{n})) and q⃗_{n}(F_{1}) = (q_{1}(F_{1}), …, q_{a_{n}}(F_{1})).

Thus the statistic (10) of a random sample of size n is the ℓ_{p} distance between the first a_{n} quotients of minimal moments of the sample, and those of the distribution. Thus for a_{n} = [n^{1/3}]+6 (as was used in the simulation below), if n = 20 the first 9 moment-quotients are compared, and if n = 50 the first 10 are compared.

Theorem 3.4. Let X_{1}, X_{2}, … be positive iid random variables with distribution F, let F = {F_{β}} be a scale-parameter family, and let 1 ≤ p ≤ ∞. If Q_{p,F_{1}}(X_{1}, …, X_{n}) → 0 a.s., then F ∈ F. Conversely, if F_{1} has finite variance, then for all F ∈ F, Q_{p,F_{1}}(X_{1}, …, X_{n}) → 0 a.s.

Proof. Fix p, 1 ≤ p ≤ ∞.

“⇒” By way of contradiction, suppose F ∉ F. Then by Theorem 3.1 there exists an m ∈ IN, and d > 0 so that |q_{m}(F) − q_{m}(F_{1})| = d. By (7) and the definition of {q_{m}}, q_{m}(X_{1}, …, X_{n}) → q_{m}(F) a.s. as n → ∞, so lim sup_{n→∞} Q_{p,F_{1}}(X_{1}, …, X_{n}) ≥ |q_{m}(F) − q_{m}(F_{1})| = d > 0 a.s.

Conversely, suppose that F_{1} has finite variance σ^{2}. Since the {q_{k}} are scale invariant, assume without loss of generality that F = F_{1}. If σ^{2} = 0, then F is the class of (positive) constant (Dirac measure) distributions, and trivially q_{k}(X_{1}, …, X_{n}) = q_{k}(F_{β}) = 1 a.s. for all k and β, so Q_{p,F_{1}}(X_{1}, …, X_{n}) ≡ 0 a.s. If σ^{2} > 0, then since 0 < Var(X_{1}) < ∞ if and only if 0 < Var(min{X_{1}, …, X_{k}}) < ∞, it follows from Serfling’s law of iterated logarithm for U-statistics (Serfling (1980, Theorem C, pp. 191)), using kernel h(x_{1}, …, x_{k}) = min{x_{1}, …, x_{k}}, that

\frac{M̂_{k}(X_{1}, …, X_{n}) − m̂_{k}(F_{1})}{\sqrt{\frac{log log n}{n}}} = O(1) a.s.

Hence

q_{k}(X_{1}, …, X_{n}) = \frac{M̂_{k}(X_{1}, …, X_{n})}{M̂_{k+1}(X_{1}, …, X_{n})}

= \frac{m̂_{k}(F_{1}) + O (\sqrt{(log log n)/n})}{m̂_{k+1}(F_{1}) + O (\sqrt{(log log n)/n})}

= \frac{m̂_{k}(F_{1}) + O (\sqrt{(log log n)/n})}{m̂_{k+1}(F_{1})} a.s.

Since m̂_{k+1}(F_{1}) > 0, this implies that

(11)  |q_{k}(X_{1}, …, X_{n}) − q_{k}(F_{1})| = O (\sqrt{(log log n)/n})  a.s.

Since

∥q⃗_{n}(X_{1}, …, X_{n}) − q⃗_{n}(F_{1})∥_{p} ≤ ∑_{k=1}^{a_{n}} |q_{k}(X_{1}, …, X_{n}) − q_{k}(F_{1})|,

it follows from (11) that

Q_{p,F_{1}}(X_{1}, …, X_{n}) = O (a_{n}\sqrt{(log log n)/n}) a.s.,

and the conclusion then follows since a_{n} = o ((n/ log log n)^{1/2}).

Corollary 3.5. (Exponential Family Case). Let X_{1}, X_{2}, … be iid with distribution F, and let F = {exp(β) : β > 0}. Then for all 1 ≤ p ≤ ∞,

( ∑_{k=1}^{[n^{1/3}]} |q_{k}(X_{1}, …, X_{n}) − (k + 1)/k|^{p})^{1/p} → 0 a.s. iff F is exp(β) for some β > 0.

Proof. Immediate from Theorem 3.4 and (4).

Analogs of Corollary 3.5 to other standard scale-parameter families are also easy to construct. In case the extremal moments {m̂_{k}} or {m^{∨}_{k}} are not available in closed form (as in the Gaussian case), they may easily be approximated by Monte Carlo simulation using the U-statistic (6).

§4. Application to Tests for Exponentiality

In this section, application of the above ideas is given for testing whether data comes from an exponential distribution exp(β) for some β > 0; that is, for testing the null hypothesis in (1). Although there are various other tests for exponentiality, such as the Pearson chi-square test and modern adaptations, empirical distribution function tests and regression tests, and tests based on the classical sample moments (cf. D’Agostino and Stephens (1986) Chapter 10 for an extensive treatise of tests for exponentiality and Baringhaus and Henze (1991) for more recent tests), the new Q_{p,F} test is robust (through choice of p and numbers of moment-quotients compared), is easy to implement, has limiting distribution which is parameter free, and is based on classical U-statistics about which a great deal is known. Since Poisson processes are characterized by exponential interarrival times, these tests may also be used for testing whether given point process data comes from a Poisson process. Analogs of the table in this section to Q_{p,F} tests for uniformity, normality and other scale-parameter families are calculated similarly.

Comparisons of the Q_{p,F} test with other standard tests for exponentiality are given in the following table against standard alternatives to the exponential distribution including types of both increasing (IFR) and decreasing failure rates (DFR) (recall the exponential has constant failure rate). The alternatives used (cf. D’Agostino and Stephens (1986), pp. 452–453) are the IFR types χ_{4}^{2}, U(0, 1), Weib(1.5) and |N|, and DFR types χ_{1}^{2}, Weib(0.8), LN, |C|, and exp(1), where:

Weib(γ) is the Weibull distribution with density, f(x) = γx^{γ−1} exp(−x^{γ}), x > 0; |N| is the distribution of Y = |N| where N is N(0, 1); LN refers to the lognormal density f(x) = const · exp(−(log x)^{2}/2), x > 0; |C| is the distribution of Y = |C| where C is standard Cauchy; exp(1) is mean-1 exponential distribution.

Standard tests against which the Q_{p,F} test is compared in Table 1 below include the Anderson-Darling test A^{2}, the Cramer-vonMises test W^{2} and the regression test W_{S} of Stephens, details of which can be found in D’Agostino and Stephens (1986).

(TABLE 1 HERE)

¿From our power study we conclude that for large sample sizes the proposed tests Q_{p,F} behave well against a large selection of alternatives and are competitive with other well known goodness of fit tests for exponentiality. For small samples, the statistics Q_{p,F} (especially Q_{2,F}) behave very well against DFR alternatives, but poorly against IFR distributions. It is expected that there will be the opposite situation if maxima are used instead of minima.

Remark. Corollary 3.2 ((i)⇔(iv)) suggests an alternative test for exponentiality: estimate m̂_{k} with the U-statistics M̂_{k} in (6), and then test M̂_{k}^{−1} for linearity in k. A similar linearitybased goodness-of-fit test for Poisson data was obtained by Nakamura and Perez-Abreu (1993).

§5. Remarks

a) The same ideas can be used to test for simple null hypotheses by just comparing the sample k-extrema with true expected extrema. For example, the null hypothesis H_{0} : F = exp(1) should be accepted iff the sample k-extrema {M̂_{k}(X_{1}, …, X_{n})} are close to the expected k-extrema {1/k} (cf. (4)). Although the above statistics Q_{p,F} were designed to test for scale-parameter families only, a similar statistic can test for (finite-moment) families in which both scale and location are unknown, since by linearity (2),

(m̂_{k+1} − m̂_{k})/(m̂_{k+2} − m̂_{k+1})

is independent of both scale and location.

b) The statistics Q_{p,F} described above give equal weight to proximity to all extremal moments, but perhaps nonuniform weights (e.g., more weight on proximity of means than higher expected extrema) will lead to tests which are more powerful against standard alternatives; this has not yet been studied by the authors.

Table 1: Estimated Powers of Q_{p,F} Test for Exponentiality at 5% Level.

n = 20, 4000 trials, a_{20} = 9

Q_{1,F} Q_{2,F} Q_{∞,F} A^{2} W^{2} W_{S}

Alternatives

IFR

χ_{4}^{2}  0.108  0.080  0.088  0.474  0.486  0.344
U(0, 1)  0.076  0.132  0.264  0.634  0.672  0.690
Weib(1.5)  0.076  0.074  0.096  0.454  0.483  0.399
|N|  0.025  0.020  0.027  0.185  0.218  0.169

DFR

χ_{1}^{2}  0.796  0.783  0.656  0.705  0.520  0.361
Weib(0.8)  0.339  0.358  0.314  0.273  0.198  0.193
LN  0.047  0.089  0.111  0.146  0.158  0.169
|C|  0.571  0.682  0.718  0.631  0.625  0.678
exp(1)  0.048  0.045  0.045  0.049  0.048  0.047
n = 50, 4000 trials, a_{50} = 10
Q_{1,F} Q_{2,F} Q_{∞,F} A^{2} W^{2} W_{S}

Alternatives

IFR

χ_{4}^{2}  0.825  0.840  0.780  0.919  0.903  0.900
U(0, 1)  0.485  0.924  0.969  0.986  0.984  1.000
Weib(1.5)  0.701  0.814  0.819  0.898  0.892  0.956
|N|  0.145  0.281  0.347  0.435  0.470  0.750

DFR

χ_{1}^{2}  0.978  0.976  0.926  0.961  0.891  0.631
Weib(0.8)  0.558  0.612  0.559  0.502  0.416  0.312
LN  0.296  0.199  0.178  0.360  0.310  0.302
|C|  0.885  0.952  0.959  0.935  0.934  0.948
exp(1)  0.047  0.053  0.050  0.053  0.054  0.080

c) Although much is known about the distribution of sample extrema (cf. Resnick (1987)) and about convergence and limiting distributions of the U-statistics {M̂_{k}}, the approximate and limiting distributions of Q_{p,F} (ratios of M̂_{k}/M̂_{k+1}) are not known to the authors.

d) The statistics Q_{p,F} have several additional advantages. First, in many applications such as those involving time to failure, the sample minima (and their quotients, which measure the advantage of adding an additional component to the system) are natural objects of independent interest.

Second, they are (via Theorem 3.4) one of the only goodness-of-fit tests for composite scale-parameter null hypotheses which are consistent against any finite-moment alternative to the null hypothesis. And third, since they depend only on existence of a finite first moment, they may prove useful in testing distributions such as stable laws which do not have finite variance.

e) As pointed out to us by Michael Stephens, the quotient test described above could be useful in the presence of censoring, since the minimum, for example, is insensitive to deletion of high values of the sample. In addition, an even more censoring-robust test could be easily designed using a statistic based on q_{c}, ..., q_{c+d} instead of q_{1}, ..., q_{d+1}, since the moments {m̂_{k}/ˆ ,k ≥ c}, also determine the distribution uniquely.

f) Finally, since the U-statistic strong law of large numbers also holds for large classes of dependent random variables such as ergodic stationary sequences (Aaronson, et al. (1996)), the same Q_{p,F} statistical tests can be used for many applications in which the data X_{1}, X_{2}, … are not necessarily independent.

The first author wishes to thank CIMAT and the Free University of Amsterdam for their hospitality during visits in which some of this research was done. The authors wish to thank Lisa Bloomer and Armando Dominguez for their assistance in running the simulations used to obtain the data in Table 1.

References

Aaronson, J., Burton, R., Dehling, H., Gilat, D., Hill, T. and Weiss, B. (1996) Strong laws

for L- and U-statistics. Trans. Amer. Math. Soc. 348, 2845–2866. Baringhaus, L. and Henze, N. (1991) A class of consistent tests for exponentiality based

on the empirical Laplace transform. Ann. Inst. Statist. Math. 43, 551-564. D’Agostino, R. and Stephens, M. (1986) Goodness of Fit Techniques, Marcel Dekker, New

York. David, H. (1981) Order Statistics, 2nd Ed., Wiley, New York. Gilat, D. and Hill, T. (1992) One-sided refinements of the strong law of large numbers and

the Glivenko-Cantelli Theorem, Ann. Probab. 20, 1213–1221. Hill, T. and Spruill, M. (1994) On the relationship between convergence in distribution

and convergence of expected extremes. Proc. Amer. Math. Soc. 121, 1235–1243. Hoeffding, W. (1961) The strong law of large numbers for U-statistics, Univ. N. Carolina,

Institute of Stat. Mimeo Series 302. Nakamura, M. and Perez-Abreu, V. (1993) Use of an empirical probability generating

function for testing a Poisson model. Canad. J. Statistics 21, 149–156. Pollack, M. (1973) On equal distributions. Ann. Statist. 1, 180–182. Resnick, S. (1987) Extreme Values, Regular Variation and Point Processes, Springer, New

York. Serfling, R. (1980) Approximation Theorems of Mathematical Statistics, Wiley, New York.

e-mail addresses:

T. P. Hill  hill@math.gatech.edu
V. Perez-Abreu pabreu@fractal.cimat.mx

Prof. S. Amari Editorial Office AISM, The Institute of Statistical Mathematics 4-6-7 Minami-Azabu, Minato-Ku Tokyo 106 JAPAN

Dear Professor Amari: Enclosed are four (4) copies of a manuscript “Extreme-value moment goodness-of-fit

tests,” joint with V. Perez-Abreu, which we would like to submit for consideration for publication in the Annals of the Institute of Statistical Mathematics.

Thank you for your time and consideration. Sincerely yours, Theodore P. Hill Enclosure TPH:ar