Confidence Bound Calculation and Prediction in the Fraction-of-Time Probability Framework

Jacek Leśkow

CIMAT, Guanajuato

and

Antonio Napolitano

Dipartimento di Ingegenria Elettronica e delle Telecomunicazioni Università di Napoli Federico II, Napoli, Italy

Abstract

The aim of this work is to introduce the concept of the confidence bound via the fraction-of-time probability approach. This approach allows to use information coming from only one realization of the phenomena under study while the observation time is increasing. The construction of the empirical and a large-sample confidence interval is presented and its implications for quantile prediction is studied.

1 Introduction

An alternative signal analysis framework that does not resort to random processes is the fractionof-time (FOT) probability approach [4]. In such an approach, statistical parameters are defined through infinite-time averages of signals (i.e., single functions of time) rather than ensemble averages of random processes.

The adoption of the FOT probability approach is motivated by the fact that in several applications, e.g. time series in finance, signal processing, and climatology, multiple realizations of a stochastic process are not at disposal of the experimenter. Rather, there is only one realization that can be assumed to be observed for an increasing-length time interval.

The differences between the FOT and stochastic approach have a variety of implications in the study of properties of inferential techniques involved. For example, in the FOT probability framework a natural way to define estimators is through considering finite-time averages of the same quantities involved in the infinite-time averages. Thus, the kind of convergence of the estimators to be considered as the time of the observation approaches infinity is the convergence of the sequence of the finite-time averages, that is, ‘pointwise,’ in the ‘temporal mean-square sense’ [15], or in the ‘sense of generalized functions (distributions)’ [11]. On the contrary, in the stochastic process framework, the convergence must be demonstrated in the mean-square sense, almost everywhere sense or in the weak convergence sense.

The FOT probability approach was first introduced in [15] with reference to time-invariant statistics of ordinary functions of time. Later, it was developed in [3] and [8], and then extended to the case of distributions (generalized functions) in [11] . Moreover, in [16] an isometric isomorfism (Wold isomorfism) between a stationary stochastic process and the Hilbert space generated by a single sample path was singled out and a rigorous link between the FOT probability and the stochastic process frameworks in the stationary case was established. The case of time-variant FOT statistics of almost-cyclostationary (ACS) time-series was widely treated in [4], [5] with reference to the second-order statistics and in [7], [13] for the higher-order statistics. Finally, the Wold isomorfism was extended to the case of cyclostationary time-series in [9]. A further development in the FOT probability theory for nonstationary signals was very recently presented in [10]. In that paper a more general class of nonstationary time-series called the generalized almost-cyclostationary (GACS) time-series has been introduced and characterized.

The apper is organized as follows. In Section 2, general concepts and definitions of the FOT probability framework are reviewed. Section 3 is devoted to construction of the confidence bounds using the FOT probability. The problem of predicting quantiles (without the independence and same distribution assumption) is presented in Section 4. Finally, simulation experiment to corroborate the proposed predictors are described in Section 5. Conclusions are drawn in Section 6.

2 General definitions

Let us consider a real-valued function x(·) that is Lebesgue measurable on the real axis ℝ.

Definition 2.1 The empirical fraction-of-time probability distribution function F_{T} (t; ξ; x) of x(u) observed on the time interval [t, t + T] is defined as

F_{T} (t; ξ; x) =^{△} meas {u ∈ [t, t + T] : x(u) ≤ ξ} ; meas {u ∈ [t, t + T]}
= \frac{1}{T} ∫_{t}^{t+T} U(ξ − x(u)) du ,  (1)

where meas{·} denotes the Lebesgue measure and

U(t) =^{△} \{ \begin{matrix}1 & t ≥ 0, \\ 0 & elsewhere. \end{matrix}  (2)

It represents the proportion of time where x(u) ≤ ξ while u ∈ [t, t + T].

In this section we assume that the function x(t) is fixed, therefore we will drop it from the notation whenever possible.

It can be easily shown that the function F_{T} (t; ξ) defined in (1) is a valid cumulative distribution function. In fact, it assumes values in the set [0, 1], is a nondecreasing function of ξ and

lim_{ξ→−∞} F_{T} (t; ξ) = 0 ,  (3a)
lim_{ξ→+∞} F_{T} (t; ξ) = 1 ,  (3b)
lim_{ξ→ξ_{0}+} F_{T} (t; ξ) = F_{T} (t; ξ_{0}) .  (3c)

From the cumulative distribution function (1) the corresponding probability density function can be defined as

f_{T} (t; ξ) =^{△} \frac{d}{dξ} F_{T} (t; ξ)
= \frac{1}{T} ∫_{t}^{t+T} δ(ξ − x(u)) du ,  (4)

where the derivation operation and the second equality must be intended in the sense of generalized functions (distributions).

The cumulative distribution function (1) and the probability density function (4) can be used to express statistical functions of the signal x(t). For example, the expected value of the distribution F_{T} is given by

E {x(u), u ∈ [t, t + T]} =^{△} ∫_{−∞}^{+∞} ξf_{T} (t; ξ) dξ
= ∫_{−∞}^{+∞} ξ \frac{1}{T} ∫_{t}^{t+T} δ(ξ − x(u)) du dξ
= \frac{1}{T} ∫_{t}^{t+T} ∫_{−∞}^{+∞} ξδ(ξ − x(u)) dξ du
= \frac{1}{T} ∫_{t}^{t+T} x(u) du
≡ ⟨x(u)⟩_{u∈[t,t+T]} ,  (5)

that is, it is coincident with the time average of the function x(u) in the interval [t, t + T]. In the derivation of (5) it has been assumed that the order of limit and integration operation can be interchanged and the sampling property of the Dirac delta function has been exploited.

A natural question for the function F_{T} (t; ξ) in Definition 2.1 is to study its asymptotic behaviour when T → ∞.

Definition 2.2. Assume that lim_{T →∞} F_{T} (t; ξ) exists. The (limit) fraction-of-time probability distribution function F (t; ξ) is defined as

F (t; ξ) =^{△} lim_{T →∞} F_{T} (t; ξ)

= lim_{T →∞} \frac{1}{T} ∫_{t}^{t+T} U(ξ − x(u)) du
= lim_{T →∞} \frac{1}{T} ∫_{0}^{T} U(ξ − x(t + t′)) dt′ .  (6)

The above function can be interpreted as the proportion of time in which the values of x(t), when t ranges in ℝ, are less or equal to ξ.

The limit in Definition 2.2 exists for a large class of functions, for example for x(·) nonnegative and increasing to infinity when t does so. For a complete mathematical treatment of the functions for which the limit in (6) exists the reader is referred to the paper of Urbanik [14]. The two cases that will be treated in some cases in this paper are the almost periodic functions [1] and stepwise functions.

Fact 2.3. Assume that for each t > 0 the limit in (6) exists. Then it does not depend on t.

Proof. We have the following obvious equality:

\frac{1}{T} ∫_{t}^{t+T} U(ξ − x(u)) du − \frac{1}{T} ∫_{0}^{T} U(ξ − x(u)) du

= \frac{1}{T} ∫_{T}^{t+T} U(ξ − x(u)) du − \frac{1}{T} ∫_{0}^{t} U(ξ − x(u)) du.  (7)

Taking T → ∞ we get the desired result. The same result can be found if t < 0. The Fact 2.3. allows us to drop t as the argument of F (t; ξ).

Note that the limit function F (ξ) is in turn a valid cumulative distribution function since the same properties of F_{T} (t; ξ) hold. Thus, the (limit) probability density function can be defined as

f(ξ) =^{△} \frac{d}{dξ} F (ξ)
= lim_{T →∞} \frac{1}{T} ∫_{t}^{t+T} δ(ξ − x(u)) du .  (8)

In the following , we will turn our attention to the specific form of the FOT distribution for stepwise function.

Definition 2.4. The function x_{s}(t) is called stepwise if it is of the form

x_{s}(t) = ∑_{k=0}^{∞} a_{k}I(t − kT_{s})  (9)

where a_{k} are finite constants and the function I(·) is the indicator function of the interval [0, T_{s}].

Fact 2.5. The empirical fraction of time distribution F_{T} (t; ξ) of the stepwise function x_{s} has the following form:

F_{T} (t; ξ) = \frac{T_{s}}{T} ∑_{k=r+1}^{p−1} U(ξ − a_{k}) + R(t, T; ξ) ,  (10)

where, for each t > 0, p = [\frac{T +t}{T_{s}}], r = [\frac{t}{T_{s}}], [b] being the integer part of b and R(t, T; ξ) tends to zero when T → ∞.

Proof. We have that

F_{T} (t; ξ) =^{△} \frac{1}{T} ∫_{t}^{T +t} U(ξ − x_{s}(u)) du
= \frac{1}{T} ∫_{t}^{(r+1)T_{s}} U(ξ − x_{s}(u)) du + \frac{1}{T} ∫_{(r+1)T_{s}}^{pT_{s}} U(ξ − x_{s}(u)) du

+\frac{1}{T} ∫_{pT_{s}}^{t+T} U(ξ − x_{s}(u)) du

= \frac{T_{s}}{T} ∑_{k=r+1}^{p−1} U(ξ − a_{k}) + R(t, T; ξ).  (11)

where

R(t, T; ξ) =^{△} \frac{1}{T} ∫_{t}^{(r+1)T_{s}} U(ξ − x_{s}(u)) du + \frac{1}{T} ∫_{pT_{s}}^{t+T} U(ξ − x_{s}(u)) du
= \frac{(r + 1)T_{s} − t}{T} U(ξ − a_{r}) + \frac{t + T − pT_{s}}{T} U(ξ − a_{p}).  (12)

Now, it easy to see that for each t > 0 and ξ ∈ ℝ it results |R(t, T; ξ)| ≤ \frac{T_{s}}{T} and this proves the result.

Theorem 2.6. Assume that the coefficients a_{k} belong to the finite alphabet {A_{1}, …, A_{M}} with A_{1} < ⋯ < A_{M} and the limit FOT probabilities p_{i} =^{△} Prob{a_{k} = A_{i}} exist. Then the limit function F exists.

Proof. For ξ < A_{1} the function F is obviously zero. For A_{1} ≤ ξ < A_{2}, accounting for (11), it results

F_{T} (t, ξ) = \frac{T_{s}}{T} ∑_{k=r+1}^{p−1} U(ξ − a_{k}) + R(t, T; ξ)

= \frac{1}{p − r − 2}\{number of a_{k} = A_{1} , k ∈ \{r + 1, …, p − 1\}\} + R(t, T; ξ) . (13)

Thus, in the limit for T → ∞ one has

F (ξ) = p_{1} , A_{1} ≤ ξ < A_{2} .  (14)

For A_{i} ≤ ξ < A_{i+1} it results

F_{T} (t, ξ) = \frac{1}{p − r − 2}\{number of a_{k} = A_{1} , k ∈ \{r + 1, …, p − 1\}\}

+\frac{1}{p − r − 2}\{number of a_{k} = A_{2} , k ∈ \{r + 1, …, p − 1\}\}

+ ⋯

+\frac{1}{p − r − 2}\{number of a_{k} = A_{i} , k ∈ \{r + 1, …, p − 1\}\} + R(t, T; ξ) (15)

and, hence, in the limit for T → ∞ one has

F (ξ) = ∑_{j=1}^{i} p_{j} , A_{i} ≤ ξ < A_{i+1} .  (16)

Remark. The FOT distributions F_{T} and F can be understood in the following sense. First, we have a Lebesgue measurable function x(t). This enables us to construct a probability measure P_{T} corresponding to the observations of {x(t) : 0 ≤ t ≤ T }. The measure P_{T} is defined on ℝ equipped with the σ-field generated by the sets {t ∈ ℝ : x(t) ≤ c; 0 ≤ t ≤ T, c ∈ ℝ}. The natural link between P_{T} and F_{T} is established via the equality

P_{T} {x(t) ≤ c} =^{△} F_{T} (c),

where F_{T} (c) can be calculated from the Definition 2.1. Identical simple considerations can be done to illicit the link between the FOT distribution F and the measure P obtained as the limit of P_{T} for T → ∞, provided that such limits exist.

Assume now that we have at our disposal N real valued measurable functions x_{1}, …, x_{N} and let us fix them for the remainder of this paragraph.

Definition 2.6. The Nth-order FOT joint cumulative distribution function is defined as:

F (ξ_{1}, …, ξ_{N}, τ_{1}, …, τ_{N}) =^{△} lim_{T →∞} \frac{1}{T} ∫_{t}^{t+T} ∏_{i=1}^{N} U(ξ_{i} − x_{i}(u + τ_{i})) du .  (17)

The function above can be interpreted as the fraction of time where jointly x_{1}(u + τ_{1}) ≤ ξ_{1}, ⋯, x_{N}(u + τ_{N}) ≤ ξ_{N} for u in ℝ and τ_{1}, …, τ_{N} fixed. It is easy to see that the function defined in (17) is a distribution function since it assumes values in [0, 1], is right-continuous, non-decreasing with respect to each variable ξ_{i}. Moreover, the function defined in (17) verifies the consistency conditions, that is, the joint cumulative distribution function of any subset of time-series x_{n}(t + τ_{n}) with n ∈ M ⊂ {1, …, N} can be obtained from (17) in the limit when the ξ_{n} such that n is not in M is set to infinity.

From (17) it follows that the Nth-order FOT joint probability density function for x_{1}, …, x_{N} is given by

f(ξ_{1}, ..., ξ_{N}, τ_{1}, …, τ_{N}) =^{△} \frac{∂^{N}}{∂ξ_{1} ⋯ ∂ξ_{N}} F (ξ_{1}, ..., ξ_{N}, τ_{1}, …, τ_{N})

= lim_{T →∞} \frac{1}{T} ∫_{0}^{T} ∏_{i=1}^{N} δ(ξ_{n} − x(t + τ_{i} + t′)) dt′ .  (18)

The Nth-order joint probability density functions allows to evaluate the joint statistical functions of the functions x_{1}(t + τ_{1}), ⋯, x_{N}(t + τ_{N}). In particular, the expected value of the product x_{1}(t + τ_{1})x_{2}(t + τ_{2}) is given by

E {x(t + τ_{1})x(t + τ_{2})}

=^{△} ∫_{−∞}^{+∞} ∫_{−∞}^{+∞} ξ_{1}ξ_{2}f(ξ_{1}, ξ_{2}, τ_{1}, τ_{2}) dξ_{1} dξ_{2}
= ∫_{−∞}^{+∞} ∫_{−∞}^{+∞}ξ_{1}ξ_{2} lim_{T →∞} \frac{1}{T} ∫_{t}^{t+T} δ(ξ_{1} − x_{1}(u + τ_{1}))δ(ξ_{2} − x_{2}(u + τ_{2})) du dξ_{1} dξ_{2}
= lim_{T →∞} \frac{1}{T} ∫_{t}^{t+T} ∫_{−∞}^{+∞}ξ_{1}δ(ξ_{1} − x_{1}(u + τ_{1})) dξ_{1} ∫_{−∞}^{+∞}ξ_{2}δ(ξ_{2} − x_{2}(u + τ_{2})) dξ_{2} du
= lim_{T →∞} \frac{1}{T} ∫_{t}^{t+T} x_{1}(u + τ_{1})x(u + τ_{2}) du
≡ ⟨x_{1}(t′ + τ)x_{2}(t′)⟩_{t′} ,  (19)

which is conincident with the temporal cross-correlation function between x_{1}(t) and x_{2}(t). In the derivation of (19), it has been assumed that the order of limit and integration operations can be interchanged and in the last equality t′ = u + τ_{2} and τ = τ_{1} − τ_{2} have been set.

Equations (5) and (19) show the duality existing between the FOT probability approach and the classical stochastic (stationary) process approach. In the former, the statistical functions are defined through infinite-time averages of a single function of time (or a lag product of more functions of time), while in the latter the analogous statistical functions are defined as ensemble averages of a stationary stochastic process (or a lag product of more stationary stochastic processes). When the stochastic process is also ergodic, then the functions defined in both the approaches are coincident, provided that the function of time is a nonpathological sample path of the ergodic stationary stochastic process.

It is worthwhile to underline that the functions (6), (8), (17), and (18) do not depend on the variable t and, hence, are suitable to describe the statistics of signals for which a stationary model is assumed.

The almost-cyclostationary case

Let us consider N real-valued measurable functions x_{1}(t), …, x_{N}(t). If the set Γ_{τ,ξ} of all the frequencies of the additive sinewave components contained in the function

U_{x}(1t + τ , ξ) =^{△} ∏_{n=1}^{N} U(ξ_{n} − x_{n}(t + τ_{n}))  (20)

is countable for each of the column vectors τ =^{△} [τ_{1}, ..., τ_{N}]^{T} ∈ ℝ^{N} and ξ =^{△} [ξ_{1}, ⋯, ξ_{N}]^{T} ∈ ℝ^{N}, then the time-series are said to be jointly generalized almost-cyclostationary in the strict sense [10]. In (20), 1 =^{△} [1, …, 1]^{T} and x =^{△} [x_{1}(t + τ_{1}), …, x_{N}(t + τ_{N})]^{T}. Moreover, if the union over all τ ∈ ℝ^{N} and ξ ∈ ℝ^{N} of the sets Γ_{τ,ξ} is a countable set Γ, then the above functions are jointly almost-cyclostationary in the strict sense. Furthermore, it is shown in [6] that the function

F (1t + τ , ξ) =^{△} E^{{α}} {U_{x}(1t + τ , ξ)}
= ∑_{γ∈Γ} ⟨U_{x}(1v + τ , ξ) e^{−j2πγv}⟩_{v} e^{j2πγt}  (21)

is a valid cumulative distribution function for each fixed value of t and τ. In (21), ⟨·⟩_{v} denotes the bilateral time average operator with respect to v and E^{{α}}{·} is the almost-periodic component extraction operator, that is, the operator that extracts all the additive sinewave components in the function specified as its argument.

The Nth-order derivative (in the sense of generalized functions) of the cumulative distribution function is defined as

f(1t + τ , ξ) =^{△} \frac{∂^{N}}{∂ξ_{1} ⋯ ∂ξ_{N}} F (1t + τ , ξ)

= E^{{α}} {∏_{n=1}^{N} δ(ξ_{n} − x_{n}(t + τ_{n}))}  (22)

and turns out to be a valid probability density function for each fixed value of t and τ.

The set Γ contains at least one element γ = 0. If γ = 0 is the only element of Γ, then the N functions are said to be jointly stationary in the strict sense. If all γ’s different from zero are integer multiples of a value γ_{0}, then the N functions are said to be jointly cyclostationary in the strict sense with the period 1/γ_{0}. If the set Γ contains incommensurate elements, then the N functions are said to be jointly almost-cyclostationary in the strict sense. Analogous definitions can be given with reference to a single signal (i.e., x_{1}(t) ≡ ⋯ ≡ x_{N}(t) ≡ x(t)).

Let us consider the special case of a single signal x(t). If the function x(t) is almost-periodic then the function (20) is also almost periodic and hence it coincides with its almost-periodic component (21). Therefore, the probability density function (22) can be expressed as

f(1t + τ , ξ) = ∏_{n=1}^{N} δ(ξ_{n} − x(t + τ_{n})) ,  (23)

that is, the almost-periodic functions are the deterministic time-series in the FOT probability framework. From this property it follows that the almost-periodic extraction operator E^{{α}}{·} extracts the deterministic component of its argument, that is, it plays the same role played by the expectation operation in the stochastic process approach. Thus, it can be shown that the almost-periodic function

ℛ(1t + τ) =^{△} E^{{α}} {∏_{n=1}^{N} x_{n}(t + τ_{n})}
= ∑_{α∈A} ℛ^{α}(τ)e^{j2παt} ,  (24)

which is called the temporal moment function, is a valid moment function, that is, it can be expressed as (see [7])

ℛ(1t + τ) = ∫_{ℝ^{N}} ∏_{n=1}^{N} ξ_{n}f(1t + τ , ξ) dξ .  (25)

In (24), A ⊆ Γ and the functions

ℛ^{α}(τ) =^{△} ⟨∏_{n=1}^{N} x_{n}(t + τ_{n})e^{−j2παt}⟩_{t}  (26)

are referred to as the cyclic temporal cross-moment functions (CTCMFs).

The set A of the α such that the CTCMF is different form zero can be possibly empty. If α = 0 is the only element of A, then the N functions are said to be jointly stationary in the wide sense. If all α’s different from zero are integer multiples of a number α_{0}, then the N functions are said to be jointly cyclostationary in the wide sense with period 1/α_{0}. If the set A contains incommensurate elements, then the functions are said to be jointly almost-cyclostationary in the wide sense. Analogous definitions can be given with reference to a single function.

It can be shown that if g[t; ξ] is an almost-periodically time-varying real-valued function of ξ ∈ ℝ^{N}, it results that

E^{{α}} {g[t; x_{1}(t + τ_{1}), …, x_{N}(t + τ_{N})]} = ∫_{ℝ^{N}} g[t; ξ]f(1t + τ , ξ) dξ,  (27)

which is called the fundamental theorem of the almost-periodic component extraction [4], [5], [6].

Finally, it is worth emphasizing that two ore more functions are said to be statistically independent in the FOT probability sense if their joint probability density function factorizes as the product of the marginal probability density functions [4], [5]. Consequently, any almost-periodic function (wich is a deterministic signal in the FOT probability framework) is statistically independent on any other function including itself (see [6]).

3 Confidence bounds in the FOT probability approach

The problem of finding confidence bounds in the FOT probability approach does not differ from the analogous problem in the classical stochastic approach. In this section we assume the following definition of quantile:

Definition 3.1. Let F be an arbitrary distribution function. Then the α-quantile q_{α} of F is defined as :

q_{α} = inf{s ∈ ℝ : F (s) ≥ α} ,  (28)

where 0 < α < 1. Similarly, we define the inverse F^{−1} of a given distribution function F as F^{−1}(u) = inf{s ∈ ℝ : F (s) ≥ u} for u ∈ (0, 1).

For more details see e.g. [12], Chapter 1.

From the previous section it was clear that the calculation of the empirical FOT probability distribution depended on t, the starting point of the observation. To simplify the notation in this Section we will assume that t is fixed and we will drop it from the notation. The Definition 3.1 can be applied to both empirical FOT distribution and the FOT distribution. For the empirical FOT distribution F_{T} an α-quantile q_{α}^{T} is such a number that the proportion of time that x(u) ≤ q_{α}^{T} while u ∈ [t, t + T] is equal to or greater than α. Similar interpretation can be derived for the limiting FOT distribution F. The question of finding q_{α}^{T} corresponds in finance to finding a Value-at-Risk for a given time series of returns x(·) on a given asset, or, in signal theory, to finding an empirical threshold corresponding to a given false alarm rate α.

Definition 3.2. The FOT confidence interval [L, U] at the confidence level α is defined as the interval between L = \frac{α}{2}-quantile and U = (1 − \frac{α}{2})-quantile of the FOT distribution F.

Similarly, we have

Definition 3.3. The emprical FOT confidence interval [L_{T} , U_{T}] at the confidence level α is defined as the interval between L_{T} = \frac{α}{2}-quantile and U_{T} = (1 − \frac{α}{2})-quantile of the empirical FOT distribution F_{T}.

The calculation of the FOT quantile follows the same lines as in the ’classical case’. For more information, see e.g. [12].

The central result of this section is the following

Theorem 3.4. Assume that the significance level α for the calculation of the quantile q_{α} of the limit FOT distribution F is a continuity point of F^{−1}. Then the empirical quantile q_{α}^{T} is a consistent estimate of q_{α}.

Proof. We use the well-known results regarding the weak convergence of the distribution functions (see [12], p. 19). Since we know (via the construction of FOT distribution F) that F_{T} converges to F then from the quoted result it follows that q_{α}^{T} → q_{α}.

To analyze a situation when F^{−1} may have some discontinuties, (e.g., in the desired significance level α) consider the following:

Example 3.5. Let x_{s}(t) be a stepwise function in the sense of the Definition 2.4. Assume also, that for each k, a_{k} ∈ {A_{1}, …, A_{M}} with A_{1} < ⋯ < A_{M}, that is coefficients a_{k} come from a finite set. Denote now p_{i} = F (A_{i}) − F (A_{i−1}), where F is the FOT distribution of x_{s}(·). Obviously, F is stepwise in this case and so is F^{−1}. By convention, assume also that F (A_{0}) = 0 and that neither of p_{i} is zero. The calculation of the empirical quantile can be accomplished using the following

Quantile estimation algorithm.

Step 1. For each 1 ≤ i ≤ M, accounting for (11), calculate the empirical probabilities p_{i}^{T} =^{△} F_{T} (A_{i}) − F_{T} (A_{i−1}) directly using the observations of the function x_{s}(·) in the interval [t, t + T], since x_{s}(t) = a_{k} for t ∈ [kT_{s}, (k + 1)T_{s}].

Step 2. Find the integer k such that

∑_{i=1}^{k} p_{i}^{T} < α ≤ ∑_{i=1}^{k+1} p_{i}^{T} .

Step 3. Put the empirical α-quantile q_{α}^{T} = A_{k+1}.

It should not be overlooked that in the algortihm above the probabilities p_{i}^{T} as well as F_{T} depend on the starting point t.

We have the following

Corollary 3.6. The empirical quantile q_{α}^{T} calculated via the above algorithm is consistent, i.e. q_{α}^{T} → q_{α}.

Proof. If the selected significance level α is a continuity point of F^{−1} then we can simply apply Theorem 3.4. If this is not the case, then q_{α} is a point where F has a discontinuity as well. In our case (stepwise function x_{s}(t) as in Example 3.5) this means that there exists such k that

α = ∑_{i=1}^{k} p_{i}.

On the other hand we know that for each i and each fixed t, p_{i}^{T} → p_{i} therefore we get the desired result.

Remark 3.7. It is easy to see that the quantile estimation algorithm presented above is a simple application of the Definition 3.1. Therefore, the calculation of the quantile in a general situation (arbitraty x) will follow similar routine.

4 Prediction of quantiles in the FOT probability approach

In this section we would like to present a method of prediction of the quantile using the FOT probability approach. The general assumption is that we observe a function x(u) on the interval [t, t + T]. Then, the basic question is:

Having observed the function until time t + T, how to predict its empirical FOT quantile at time t + T + δ ?

The answer to the question above will be based entirely on the previously presented method of calculating quantiles and empirical FOT distributions. In our opinion, the FOT approach has a lot of advantages over the classical, stochastic methods. First of all, no assumption on independence of observations or stationarity of distributions is necessary.

This Section will start with the presentation of a method of predicting quantile for a case when the observed signal x_{s} is stepwise function in the sense of Definition 2.4 with a_{k} ∈ {A_{1}, …, A_{k}}. We will call this case a finite alphabet case. Then, in the latter part of this Section we will put forward a quantile calculation for any continuous signal x(t).

4.1 Finite alphabet case

Let us first note the following useful fact:

Fact 4.1. Assume that t and T are fixed and assume that the function x_{s}(t) is stepwise plus a_{k} ∈ {A_{1}, … A_{M}} with A_{1} < ⋯ < A_{M} – the finite alphabet for the coefficients a_{k}. Let the integers p and r have the same meaning as in Fact 2.5. Therefore

p_{i}^{T} = F_{T} (A_{i}) − F_{T} (A_{i−1})
= \frac{T_{s}}{T} ∑_{k=r+1}^{p−1} [U(A_{i} − a_{k}) − U(A_{i−1} − a_{k})]

+\frac{(r + 1)T_{s} − t}{T} [U(A_{i} − a_{r}) − U(A_{i−1} − a_{r})]

+\frac{t + T − pT_{s}}{T} [U(A_{i} − a_{p}) − U(A_{i−1} − a_{p})] .  (29)

Observe that the last two terms of the equality (29) are of the order T^{−1}.

Proof. The desired results follows from (11) and (12).

In the formula (29) we use the convention that U(A_{0} − x) = 0 for any x.

The method of predicting quantile is based on the study of behaviour of probabilities p_{i}^{T} when the length of observation changes from T to T + δ. Obviously, the precision of the prediction depends on how large the horizon δ is. Here we will assume that δ = T_{s}, that is, the horizon of the prediction is equal to the length of the window. In finance, this corresponds e.g., to constructing a one-day predictor of Value-at-Risk, when the data are gathered daily and possible values of the return x(·) come from a finite set. With that assumption in mind we have the following

Fact 4.2. Let t, T and x_{s} be as in Fact 4.1. Assume also that the time horizon δ of the prediction is equal to T_{s}. Therefore, accountig for (11) it results

F_{T +T_{s}}(ξ) = \frac{T_{s}}{T + T_{s}} ∑_{k=r+1}^{p} U(ξ − a_{k}) + R(t, T + T_{s}; ξ)

= \frac{T}{T + T_{s}} F_{T} (ξ) − \frac{t + T − (p + 1)T_{s}}{T + T_{s}} U(ξ − a_{p}) + \frac{t + T − pT_{s}}{T + T_{s}} U(ξ − a_{p+1}) (30)

and, hence,

p_{i}^{T +T_{s}} = F_{T +T_{s}}(A_{i}) − F_{T +T_{s}}(A_{i−1})
= \frac{T}{T + T_{s}} p_{i}^{T} + R_{1}(t, T; i) + R_{2}(t, T; i) ,  (31)

where

R_{1}(t, T; i) =^{△} −\frac{t + T − (p + 1)T_{s}}{T + T_{s}} [U(A_{i} − a_{p}) − U(A_{i−1} − a_{p})]  (32)

and

R_{2}(t, T; i) =^{△} \frac{t + T − pT_{s}}{T + T_{s}} [U(A_{i} − a_{p+1}) − U(A_{i−1} − a_{p+1})] .  (33)

Note that in (31) for t = rT_{s} and t + T = pT_{s} it results R_{2}(t, T; i) = 0 and the term R_{1}(t, T; i) cannot be evaluated since it depends on the unobservable value a_{p}. If pT_{s} < t + T < (p + 1)T_{s}, then R_{1}(t, T; i) can be evaluated whereas R_{2}(t, T; i) cannot since it depends on the unobservable value a_{p+1}.

Corollary 4.3. From Fact 4.2 it follows that for each 1 ≤ i ≤ M and for t, T and x_{s} such as in Fact 4.1 we have

∑_{i=1}^{k} p_{i}^{T +T_{s}} = \frac{T}{T + T_{s}} ∑_{i=1}^{k} p_{i}^{T} + ∑_{i=1}^{k} [R_{1}(t, T; i) + R_{2}(t, T; i)] ,  (34)

where, accountig for the fact that for arbitrary a_{q}

∑_{i=1}^{k} [U(A_{i} − a_{q}) − U(A_{i−1} − a_{q})] = \{ \begin{matrix}1 & A_{1} ≤ a_{q} ≤ A_{k} \\ 0 & otherwise \end{matrix}  (35)

it results

∑_{i=1}^{k} |R_{n}(t, T; i)| ≤ \frac{T_{s}}{T + T_{s}} , n = 1, 2 .  (36)

Moreover, (34) can be written as

∑_{i=1}^{k} p_{i}^{T +T_{s}} = A(T, T_{s}; k) + ϵ B(t, T, T_{s}) ,  (37)

where

A(T, T_{s}; k) = \{ \begin{matrix}\frac{T}{T + T_{s}} ∑_{i=1}^{k} p_{i}^{T} & t = rT_{s} , t + T = pT_{s} , \\ \frac{T}{T + T_{s}} ∑_{i=1}^{k} p_{i}^{T} + ∑_{i=1}^{k} R_{1}(t, T; i) & otherwise \end{matrix}  (38)

and

B(t, T, T_{s}) = \{ \begin{matrix}\frac{T_{s}}{T + T_{s}} & t = rT_{s} , t + T = pT_{s} , \\ \frac{t + T − pT_{s}}{T + T_{s}} & otherwise \end{matrix}  (39)

in which the value of ϵ ∈ {0, 1} depends on the unobservable symbol.

The problem of finding the quantile q_{α}^{T +T_{s}} consists in calculating the integer k such that

∑_{i=1}^{k} p_{i}^{T +T_{s}} < α ≤ ∑_{i=1}^{k+1} p_{i}^{T +T_{s}}  (40)

and to put q_{α}^{T +T_{s}} = A_{k+1}. However, on the basis of the observations in [t, t + T] we cannot calculate the probabilities p_{i}^{T +T_{s}} but only p_{i}^{T}. Therefore, the problem of predicting the quantile is calculating a predictor q̂_{α}^{T +T_{s}} only using observations from the interval [t, t + T]. Consider the following

Quantile prediction algorithm for stepwise functions

Step 1 For each i ∈ {1, …, M} calculate the empirical probabilities p_{i}^{T}.

Step 2 Make the following procedure:

for ϵ′ ∈ {0, 1}

for ϵ′′ ∈ {0, 1}

find k such that

A(T, T_{s}; k) + ϵ′B(t, T, T_{s}) < α ≤ A(T, T_{s}; k + 1) + ϵ′′B(t, T, T_{s}) (∗)

if disequality (∗) is true then

q̂_{α}^{T +T_{s}}(ϵ′, ϵ′′) = A_{k+1}

end

end

end

In the above algorithm there are four cases to consider: ϵ′ = ϵ′′ (two cases) and ϵ′ ≠ ϵ′′ (two cases). Therefore the proposed prediction algorithm provides at least one and at most four possible predicted values for the quantile q_{α}.

In that context, a natural question arises: Given a certain precision θ, how large T is sufficient to claim that the quantile stabilizes at that precision, that is |q_{α}^{T +T_{s}} − q_{α}^{T} | ≤ θ ? This question has a simple answer for stepwise functions x_{s} as in Fact 4.1.

Remark 4.4. Assume that we would like to predict a quantile q_{α}^{T +T_{s}} with a given precision θ, 0 < θ < 1. Then it suffices to take the sample from the signal of the length T such that

T > \frac{4 − θ}{θ}

and put q_{α}^{T +T_{s}} =^{△} q_{α}^{T}, where q_{α}^{T} is a quantile calculated according to the algorithm presented in Section 3.

Proof. It suffices to see that, according to Corollary 4.3

| ∑_{i=1}^{k}(p_{i}^{T +T_{s}} − p_{i}^{T})| ≤ \frac{4T_{s}}{T + T_{s}} .

Therefore, at a precision θ the quantile calculation algorithm for the distribution F_{T +T_{s}} will not be changed if we replace ∑_{i=1}^{k} p_{i}^{T +T_{s}} by ∑_{i=1}^{k} p_{i}^{T}. This argument can be seen more clearly noting that solving the inequality (*) in the quantile prediction algorithm for a significance level α is equivalent to finding a quantile from the empirical fraction-of-time probability F_{T} on the level α − θ, where θ is sufficiently small. This last observation is true do to the property of left-continuity of F^{−1}.

4.2 Continuous functions

In this part we will analyze the quantile prediction algorithm for any continuous function x. As it was seen in the previous subsection, the quantile prediction algorithm produces a (finite) set of values for the predictor q̂_{α}^{T +T_{s}}. It is easy to deduce that in the case of the continuous function x we will end up with the interval of possible predictors. This intution will be clarified in this subsection. We start with the technical result, convienient while calculating the predicted quantile.

Fact 4.5. Assume that the function x generating the FOT distributions F_{T} and F is continuous. Assume also that the time horizon δ is arbitrary and let F_{T +δ} be the empirical fraction of time distribution generated by x on the interval [t, t + T + δ]. Then

F_{T +δ}(ξ) = \frac{T}{T + δ} F_{T} (ξ) + \frac{1}{T + δ} ∫_{t+T}^{t+T +δ} U(ξ − x(u)) du.  (41)

Proof of this Fact is standard and follows from Fact 2.5.

It is easy to see that the second term of the right hand side of (41) is smaller than \frac{δ}{T +δ}. In another words,

F_{T +δ}(ξ) ∈ [\frac{T}{T + δ} F_{T} (ξ), \frac{T}{T + δ} (F_{T} (ξ) + \frac{δ}{T})].

This observation allows us to formulate the following Quantile prediction algorithm for continuous functions

Step 1. Given the observation of the continuous function x calculate the F_{T}. Step 2. Calculate

q̂_{α}^{T +δ}(1) = inf\{ξ : \frac{T}{T + δ} F_{T} (ξ) + \frac{δ}{T + δ} ≥ α\}

and

q̂_{α}^{T +δ}(2) = inf\{ξ : \frac{T}{T + δ} F_{T} (ξ) ≥ α\}.

It is easy to see that the above algorithm produces the interval of predictions of the form [q̂_{T +δ}(1), q̂_{T +δ}(2)] and that the length of this interval, \frac{δ}{T +δ} converges to zero while T → ∞ for any fixed time horizon δ. This yields the following

Remark 4.6. Assume that we would like to predict a quantile q_{α}^{T +δ} with a given precision θ, 0 < θ < 1. Then it suffices to take the sample from the signal of the length T such that

T > \frac{δ · (1 − θ)}{θ}

and put q_{α}^{T +δ} =^{△} q_{α}^{T}, where q_{α}^{T} is a quantile calculated according to the algorithm presented in Section 3.

Appendix A

In this Appendix the convergence in the temporal mean-square sense of time averages and integrals is discussed. A more comprehensive treatment can be found in [15] for stationary time-series and in [2] for almost-cyclostationary time series. The more general convergence in the sense of distributions (generalized functions) of statistical functions defined starting from a single time-series is treated in [11].

In this paper, unless otherwise indicated, all the time averages are assumed to exist and to be convergent in the temporal mean-square sense (t.m.s.s.), that is, given a time-series z(t) and defined

z_{β}(t)_{T} =^{△} \frac{1}{T} ∫_{t−T/2}^{t+T/2} z(u) e^{−j2πβu} du ,  (A.1)
z_{β} =^{△} lim_{T →∞} \frac{1}{T} ∫_{−T/2}^{+T/2} z(u) e^{−j2πβu} du ,  (A.2)

it is assumed that

lim_{T →∞} z_{β}(t)_{T} = z_{β} (t.m.s.s.) , ∀β ∈ ℝ  (A.3)

that is,

lim_{T →∞} ⟨|z_{β}(t)_{T} − z_{β}|^{2}⟩_{t} = 0 , ∀β ∈ ℝ .  (A.4)

It can be shown that for a finite-power time-series z(t) (i.e., with ⟨|z(t)|^{2}⟩_{t} < ∞) the set B =^{△} {β ∈ ℝ : z_{β} ≠ 0} is countable, the series ∑_{β∈B} |z_{β}|^{2} is summable [2], and from (A.3) it follows that

lim_{T →∞} ∑_{β∈B} z_{β}(t)_{T} e^{j2πβt} = ∑_{β∈B} z_{β} e^{j2πβt} (t.m.s.s.) .  (A.5)

Note that the right-hand side in (A.5) is just the almost-periodic component contained in the time-series z(t).

It is worth to underline that, in the FOT probability framework, the variance of the estimators is defined in terms of the almost-periodic component extraction operation, which plays the same role played by the statistical expectation in the stochastic process framework [4], [5]. Thus, unlike the stochastic process framework, where the variance accounts for fluctuations of the estimates over the ensemble of sample paths, in the FOT probability framework the variance accounts for the fluctuations of the estimates in the time parameter, e.g., the central point of the finite-length time-series segment adopted for the estimation. Therefore, the assumption that the estimator asymptotically approaches the true value (the infinite-time average) in the mean-square sense is just equivalent to the statement that the variance of the estimator in the FOT probability sense approaches zero as the collect time approaches infinity. That is, for such time-series, estimates obtained by using different time segments are asymptotically independent of the central point of the segments.

References

[1] Besicovitch, “Almost-periodic functions,” 1932.

[2] W.A. Brown, “On the theory of cyclostationary signals,” Ph. D. dissertation, Dep. Elect. Eng. Comput. Sci., Univ. California, Davis, 1987.

[3] H. Furstenberg, “Stationary processes and prediction theory,” Princeton University Press, Princeton, NJ, 1960.

[4] W.A. Gardner, Statistical Spectral Analysis: A Nonprobabilistic Theory. Prentice Hall, Englewood Cliffs, NJ, 1987.

[5] W.A. Gardner, “An introduction to cyclostationary signals,” Chapter 1 in Cyclostationarity in Communications and Signal Processing, W.A. Gardner, Ed., pp. 1-90, IEEE Press, New York, 1994.

[6] W.A. Gardner and W.A. Brown, “Fraction-of-time probability for time-series that exhibit cyclostationarity,” Signal Processing, vol. 23, pp. 273-292, June 1991.

[7] W.A. Gardner and C.M. Spooner, “The cumulant theory of cyclostationary time-series, Part I: Foundation,” IEEE Trans. Signal Processing, vol. 42, pp. 3387-3408, December 1994.

[8] E.M. Hofstetter, “Random processes,” Chapter 3 in The Mathematics of Physics and Chemistry, vol. II, H. Margenau and G.M. Murphy eds., Princeton, NJ: D. Van Nostrand Co., 1964.

[9] H.L. Hurd and T. Koski, “On the Wold isomorfism and Wold decomposition for prediction of cyclostationary sequences,” in Proc. of Workshop on Cyclostationary Processes, Noisy Le Grand, F, July 1996.

[10] L. Izzo and A. Napolitano, “The higher-order theory of generalized almost-cyclostationary time-series,” IEEE Trans. Signal Processing, vol. 46, pp. 2975-2989, November 1998.

[11] E. Pfaffelhuber, “Generalized harmonic analysis for distributions,” IEEE Trans. Inform. Theory, vol. IT-21, pp. 605-611, November 1975.

[12] R.D. Reiss, Approximate Distributions of Order Statistics, with applications to nonparametric statistics. Springer Series in Statistics, 1989.

[13] C.M. Spooner and W.A. Gardner, “The cumulant theory of cyclostationary time-series, Part II: Development and applications,” IEEE Trans. Signal Processing, vol. 42, pp. 3409-3429, December 1994.

[14] K. Urbanik, “Effective processes in the sense of H. Steinhaus,” Studia Mathematica, XVII, 1958.

[15] N. Wiener, “Generalized harmonic analysis, ” Acta Mathematica, vol. 55, pp. 117-258, 1930.

[16] H.O.A. Wold, “On prediction in stationary time series,” Ann. Math. Statist., vol. 19, pp.558-567, 1948.