Discretely observed diffusions: classes of estimating functions and small Δ−optimality

Martin Jacobsen^{∗} Department of Theoretical Statistics University of Copenhagen 5 Universitetsparken 2100 Copenhagen Ø Denmark

November 11, 1998

Abstract

Ergodic diffusions in several dimensions, depending on an unknown multivariate parameter are considered. For estimation, when the diffusion is observed only at finitely many equidistant timepoints, unbiased estimating functions leading to consistent and asymptotically Gaussian estimators are used. Different types of estimating functions are discussed and the concept of small Δ−optimality is introduced to help select good estimating functions. Explicit criteria for small Δ−optimality are given. Also some exact optimality conditions are presented as well as, for one-dimensional diffusions, methods for improving estimators using time reversibility.

Keywords and Phrases: estimating functions of type: simple, martingale, transition dependent, reversible; asymptotic normality of estimators; minimizing asymptotic covariances.

∗ MaPhySto – Centre for Mathematical Physics and Stochastics, funded by a grant from the Danish National Research Foundation.

Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 The model; setup and notation . . . . . . . . . . . . . . . . . . . . . . 4 3 Classes of estimation functions . . . . . . . . . . . . . . . . . . . . . . . 9 4 Asymptotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5 Optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 6 Time reversal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 7 Small Δ−optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

1. Introduction

The main purpose of this paper is to discuss criteria for choosing good estimating functions when estimating the parameters of an ergodic diffusion that is observed only at discrete points in time. While in principle one should of course use maximum-likelihood, in practice this is difficult because only in a few cases are the transition densities of the diffusion available in closed analytic form. Thus the existing methods for maximum-likelihood, see Pedersen [13] and Aït-Sahalia [1], rely on approximations of the transition densities and the methods prove quite computer intensive. (Pedersen uses numerical approximations based on iterations of the Gaussian transition densities emanating from the Euler scheme, while Aït-Sahalia, using a specific transformation of the diffusion, is able to obtain accurate theoretical approximations based on (transformed) Hermite function expansions).

Because of the difficulty in performing accurate maximum-likelihood, much research has focused on finding alternatives in the form of various types of unbiased estimating functions, but again here there may be practical problems because explicit analytic expressions are not always available.

The inspiration for the present paper came from following the work of Michael Sørensen and his colleagues, notably Mathieu Kessler, and was initiated by the desire for finding explicit estimating functions beyond the class of simple ones discussed by Kessler [9]. This led first to what proved merely a rediscovery of the class of explicit, transition dependent estimating functions presented by Hansen and Scheinkman [7], but eventually also to the study of optimality criteria presented below.

The basic model, involving only ergodic diffusions observed at equidistant timepoints distance Δ apart, and the basic assumptions together with the notation is presented in Section 2. In Section 3 we give an overview of the different types of estimating functions known from the literature, and in Section 4 survey the asymptotic theory (consistency, asymptotic normality) for the estimators obtained from the estimating functions. Section 6 treats the special results that arise from the time-reversibility of one-dimensional ergodic diffusions, while optimality is discussed in Sections 5 and 7. The main results are the conditions for small Δ−optimality presented in Theorems 7.5 and 7.8: while exact optimality (i.e. minimizing the asymptotic covariance for an estimator within a given class) leads to results that are difficult to implement, see Section 5, it turns out that it is possible to give explicit, easily verifiable conditions on flows of estimating functions that ensure nearly efficient estimation if the discrete observations of the diffusion are close together. (A flow is simply a sufficiently smooth family of estimating functions, one for each Δ > 0). The conditions are flexible enough that there are many choices for small Δ−optimal flows. A drawback is of course that a flow that performs well for small Δ, even though the estimator is always consistent and asymptotically Gaussian, may behave poorly for large Δ, and in the last part of Section 7 we make some preliminary suggestions on how to resolve this difficulty.

2. The model; setup and notation

Consider a statistical model for a d−dimensional diffusion process X, solving the stochastic differential equation

dX_{t} = b_{θ} (X_{t}) dt + σ_{θ} (X_{t}) dB_{t}  (2.1)

with some initial condition X_{0} = U. Here B is a standard d−dimensional Brownian motion and b and σ are known functions, b_{θ}(x), σ_{θ}(x) of an unknown p−dimensional parameter θ and x ∈ ℝ^{d×1}, b taking values in ℝ^{d×1} and σ matrix-valued, σ_{θ}(x) ∈ ℝ^{d×d}.

Notation. Throughout the paper, column vectors are denoted x, θ etc., row vectors are written as transposed columns, x^{T} , θ^{T} etc. Indexes i, j are reserved for objects relating to the state space D ⊂ ℝ^{d} of the diffusion, indexes k, l for objects relating to the parameter space Θ ⊂ ℝ^{p}. We write x_{i} for the coordinates of x, θ_{k} for the coordinates of θ, but for processes or the functions b, σ, where time, respectively θ, appears as a subscript, use superscripts to designate the coordinates, X^{i}, b_{θ}^{i}, σ_{θ}^{ij}. Thus writing (2.1) coordinatewise,

dX_{t}^{i} = b_{θ}^{i} (X_{t}) dt + ∑_{j=1}^{d} σ_{θ}^{ij} (X_{t}) dB_{t}^{j}, X_{0}^{i} = U^{i}.

Formally, the diffusion X is defined on a filtered measurable space (Ω, F, F_{t}) with U F_{0}−measurable. We shall assume that for any θ ∈ Θ, and any probability ν on ℝ^{d}, there is a probability measure P_{θ}^{ν} on (Ω, F) with respect to which the σ−algebra F_{0} and the Brownian motion B are independent, and such that for the prescribed θ−value, (2.1) has a unique strong solution (in particular the law of the solution is unique) with ν the distribution of U. Subject to P_{θ}^{ν}, X is then a time-homogeneous diffusion with transition probabilities that do not depend on ν and depend on the diffusion coefficient σ_{θ} through

C_{θ} := σ_{θ}σ_{θ}^{T}

only (with ^{T} denoting matrix transposition). Apart from these assumptions of a rather general nature, we shall need some more specific assumptions:

Assumption 1. The parameter θ belongs to an open subset Θ ⊂ ℝ^{p}, and for each θ ∈ Θ, the diffusion X is ergodic, i.e. admits of a uniquely determined invariant distribution μ_{θ}: subject to P_{θ}^{μ_{θ}}, X is strictly stationary. It is also assumed that the range of X does not depend on θ: there is an open connected subset D ⊂ ℝ^{d} such that for all θ, ν,

P_{θ}^{ν} ⋂_{t≥0} (X_{t} ∈ D) = 1,

and μ_{θ}(D ∩ O) > 0 for all open O with the interior of D ∩ O non-empty.

Remark. The assumption about D not depending on θ is made of course for statistical reasons – the mere fact that an observation X_{t_{0}} belongs to some subset of D must not contain information about θ. We have assumed for convenience that D is open, but in principle one could allow for diffusions with accessible reflecting boundaries. In that case, typically (but not always), (2.1) is not sufficient to describe X and a complete description involves local time along the boundary. The assumption D connected is there simply to ensure that the invariant measure is uniquely determined. The reader is reminded that for d = 1, Assumption 1 amounts to the following when σ_{θ}(x) > 0 for all x: D = ]l, r[ for some −∞ ≤ l < r ≤ ∞ and if S_{θ} is a scale function with derivative (with respect to x)

S′_{θ}(x) = exp (−2 ∫_{x_{0}}^{x} \frac{b_{θ}(y)}{σ_{θ}^{2}(y)} dy)  (2.2)

for some x_{0} ∈ D, then S_{θ}(l) = −∞, S_{θ}(r) = ∞; furthermore, k_{θ} := ∫_{l}^{r} κ_{θ}(x) dx < ∞ where

κ_{θ} = 2/ (σ_{θ}^{2}S′_{θ})  (2.3)

is the density of the speed measure matching the scale S_{θ} and μ_{θ} has density κ_{θ}(x)/k_{θ}.

If d ≥ 2 not too much is known about the range D of the solution to (2.1), although there are of course well known conditions ensuring that D = ℝ^{d}.

Notation. With ν a probability on D, we write ν(f) for ∫_{D} f dν. The Lebesgue density of the invariant distribution of μ_{θ} will also be denoted μ_{θ}. If ν = μ_{θ} we write P_{θ}^{μ} instead of P_{θ}^{μ_{θ}} and if ν = ε_{x}, simply P_{θ}^{x}. We shall write E_{θ}^{ν} for expectations with respect to P_{θ}^{ν} and π_{t,θ} for the transition operators for X when θ is the parameter, p_{t,θ}(x, y) for the transition densities. Thus, for functions f integrable with respect to p_{t,θ}(x, y) dy,

E_{θ}^{ν} (f(X_{s+t}) |X_{s} = x) = π_{t,θ}f(x) = ∫_{D} dy p_{t,θ}(x, y)f(y)

for all s, x, ν. Finally, Q_{t,θ} will denote the joint distribution of (X_{s}, X_{s+t}) under P_{θ}^{μ} (for any s) and q_{t,θ} its density,

q_{t,θ}(x, y) = μ_{θ}(x)p_{t,θ}(x, y).

Assumption 2. The following assumptions are made on the drift and diffusion coefficients b and σ from (2.1): for all θ ∈ Θ, b_{θ}(x), C_{θ}(x) are continuous in x, and for each x ∈ D, b_{θ}(x), C_{θ}(x) are continuously differentiable in θ. Also C_{θ}(x) > 0 for all θ, x.

Notation. C_{θ}(x) is a symmetric, positive semidefinite d × d−matrix. If A_{1}, A_{2} are two symmetric matrices, we write A_{1} ≥ A_{2} if A_{1} − A_{2} is positive semidefinite and A_{1} > 0 if A_{1} is symmetric and positive definite.

Throughout the paper the following notation is used for differentiation: if a(r) = (a_{1}(r), …, a_{q}(r)) ∈ ℝ^{q} is a differentiable function of a q′−dimensional variable r = (r_{1}, …, r_{q′}), ∂_{r}a(r) denotes the q × q′−dimensional matrix of partial derivatives (∂_{r}a)_{ij} = \frac{∂a_{i}}{∂r_{j}} evaluated at r. Thus, if q = 1, ∂_{r}a(r) is a q′−dimensional row vector, and for general q, the i′th row of ∂_{r}a(r) is ∂_{r}a_{i}(r). If differentiation is with respect to θ, we use a dot, ^{·}, instead of the symbol ∂. Thus (cf Assumption 2),

\begin{matrix}b^{·}_{θ} (x) ∈ ℝ^{d×p}, & b^{·}_{θ,ik} (x) = \frac{∂b_{θ}^{i}(x)}{∂θ_{k}} \\ C^{·}_{θ} (x) ∈ ℝ^{d^{2}×p}, & C^{·}_{θ,(ij)k} (x) = \frac{∂C_{θ}^{ij}(x)}{∂θ_{k}}. \end{matrix}

Introduce the differential operator associated with the infinitesimal generator for the transition semigroup (π_{t,θ})_{t≥0} for the diffusion X,

A_{θ}f(x) = ∑_{i=1}^{d} b_{θ}^{i}(x)\frac{∂}{∂x_{i}} f(x) + \frac{1}{2} ∑_{i,j=1}^{d} C_{θ}^{ij}(x)\frac{∂^{2}}{∂x_{i}∂x_{j}} f(x)

acting on the space C^{2}(D) of functions f : D → ℝ that are twice continuously differentiable.

If f ∈ C^{2}(D), by Itô’s formula

df(X_{t}) = A_{θ}f(X_{t}) dt + ∂_{x}f(X_{t})σ_{θ}(X_{t}) dB_{t}  (2.4)

(with the three ‘factors’ in the last term of dimensions 1 × d, d × d, d × 1 respectively). The last term defines a local martingale, which is a true martingale if

E_{θ}^{μ}∂_{x}f(X_{t})C_{θ}(X_{t})(∂_{x}f)^{T} (X_{t}) = ∫_{D} dx μ_{θ}(x)∂_{x}f(x)C_{θ}(x)(∂_{x}f)^{T} (x) < ∞, (2.5)

hence it follows that if f ∈ L^{1} (μ_{θ}) ∩ C^{2}(D), A_{θ}f ∈ L^{1} (μ_{θ}) and (2.5) holds, then

μ_{θ}(A_{θ}f) = 0,  (2.6)

which is the equation used to determine μ_{θ}.

Here we shall be concerned with a L^{2}−generator for X. Recall that π_{t,θ} : L^{2} (μ_{θ}) → L^{2} (μ_{θ}) , so we have a semigroup of operators on L^{2} (μ_{θ}), and then define the domain D_{θ} for the L^{2}−generator A_{θ} as the subspace comprising all f ∈ L^{2} (μ_{θ}) ∩ C^{2}(D) for which A_{θ}f ∈ L^{2} (μ_{θ}) and (2.5) holds, and for such f, define A_{θ}f = A_{θ}f. It is well known that D_{θ} is dense in L^{2} (μ_{θ}) . Clearly for any θ, D_{θ} contains the space DC consisting of the f ∈ C^{2}(D) with compact support. More generally, D_{θ} ⊃ DB_{θ}, the space of f ∈ C^{2}(D) such that f and A_{θ}f are bounded: to verify this, the problem is to show (2.5), i.e. that E_{θ}^{μ} [f(X)]_{t} < ∞, where [f(X)]_{t} = ∫_{0}^{t} ds ∂_{x}f(X_{s})C_{θ}(X_{s})∂_{x}^{T} f(X_{s}) is the quadratic variation of f(X). Introducing the stopping times τ_{K} := inf {t : [f(X)]_{t} = K} , one finds that M_{K} and(M_{K}^{2} − [f(X)]_{τ_{K}∧t})_{t≥0} are martingales, where

M_{K,t} = f(X_{τ_{K}∧t}) − f(X_{0}) − ∫_{0}^{τ_{K}∧t} ds A_{θ}f(X_{s}).

Thus E_{θ}^{μ}M_{K,t}^{2} = E_{θ}^{μ} [f(X)]_{τ_{K}∧t} and for K ↑ ∞, by monotone convergence the expectation on the right increases to E_{θ}^{μ} [f(X)]_{t} while, since f, A_{θ}f are bounded, dominated convergence applied to E_{θ}^{μ}M_{K,t}^{2} yields a finite limit and (2.5) is proved.

Note that for d = 1, DC is dense in L_{0}^{2} (μ_{θ}) = 1_{θ}^{⊥} := {f ∈ L^{2} (μ_{θ}) : μ_{θ}(f) = 0} , a fact that is definitely not true for d > 1: thus, to determine μ_{θ} if d = 1 one need only consider (2.6) for f ∈ DC, but for d > 1 it is necessary to involve, say, all f ∈ DB_{θ}, which makes the determination of μ_{θ} more difficult. (A swift proof of the formula for μ_{θ}(x) for d = 1 (see (2.3) and the next line) is the following: if f ∈ DC, using partial integration on ∫ A_{θ}f dμ_{θ} = 0 gives ∫ (b_{θ}μ_{θ}f′ − \frac{1}{2} (σ_{θ}^{2}μ_{θ})′ f′) dx = 0 and since f′ satisfies ∫_{D} f′ dx = 0 and the collection of these derivatives is large enough, the density μ_{θ} must satisfy the differential equation b_{θ}μ_{θ} = \frac{1}{2} (σ_{θ}^{2}μ_{θ})′ from which the desired expression follows).

Notation. Write D_{0,θ} = D_{θ} ∩ 1_{θ}^{⊥}.

We shall need a final, more restrictive condition. Define

spec(A_{θ}) = {λ ∈ ℂ : A_{θ}f = λf for some complex-valued f ∈ D_{θ}} .

Assumption 3. There exists λ_{0} < 0 such that Re λ ≤ λ_{0} for all λ ∈ spec(A_{θ})\0.

If λ ∈ spec(A_{θ}), then certainly Re λ ≤ 0 with 0 ∈ spec(A_{θ}) always. With a spectral gap λ_{0} < 0, we have for all f ∈ 1_{θ}^{⊥} that ∥π_{t,θ}f∥ ≤ e^{λ_{0}t} ∥f∥ , where ∥·∥ is the norm on L^{2} (μ_{θ}) , and hence for any f ∈ L_{θ}^{2} (μ_{θ}) , π_{t,θ}f converges in L^{2} (μ_{θ}) at an exponential rate as t → ∞ to the function which is constant and equal to μ_{θ}(f). With Assumption 3 in force, it is therefore possible to define the potential operator U_{t,θ} for any t > 0 by

U_{t,θ}f = ∑_{n=0}^{∞} π_{nt,θ}f = ∑_{n=0}^{∞}(π_{t,θ})^{n}f  (2.7)

for f ∈ 1_{θ}^{⊥} with the series converging in L^{2} (μ_{θ}) to a limit in 1_{θ}^{⊥}.

Note also that with respect to P_{θ}^{μ}, for any t > 0 fixed, the strictly stationary sequence (X_{mt})_{m≥0} is mixing: since the sequence is a Markov chain, to show this it suffices to show that for all bounded and measurable f, all n_{0} ∈ ℕ and all sets A = ((X_{0}, X_{t}, …, X_{n_{0}t}) ∈ B) with B Borel,

lim_{n→∞} E_{θ}^{μ}1_{A}f(X_{(n_{0}+n)t}) = P_{θ}^{μ}(A) μ_{θ}(f).  (2.8)

But

E_{θ}^{μ}1_{A}f(X_{(n_{0}+n)t}) = E_{θ}^{μ} (1_{A} (π_{nt,θ}(f)) (X_{n_{0}t}))

and since π_{θ,nt}(f) → μ_{θ}(f) in L^{2} (μ_{θ}) it follows that

E_{θ}^{μ}1_{A} [(π_{nt,θ}(f)) (X_{n_{0}t}) − μ_{θ}(f)]^{2} ≤ ∥π_{nt,θ}(f) − μ_{θ}(f)∥_{μ_{θ}}^{2} → 0

with (2.8) now an easy consequence.

The mixing property ensures that the ergodic theorem holds, e.g.

\frac{1}{n} ∑_{m=1}^{n} φ (X_{(m−1)t}, X_{mt}) →_{n→∞} E_{θ}^{μ}φ (X_{0}, X_{t})  (2.9)

with convergence P_{θ}^{μ}−a.s. and in L^{1}(P_{θ}^{μ}) if φ ∈ L^{1}(Q_{t,θ}), and also in L^{2} (P_{θ}^{μ}) if φ ∈ L^{2}(Q_{t,θ}).

An important observation to be used later is

Proposition 2.1. If f ∈ D_{0,θ} and A_{θ}f ≡ 0, then f ≡ 0.

Proof. The process f(X) is a L^{2}−bounded P_{θ}^{μ}−martingale, hence lim_{t→∞} f(X_{t}) exists P_{θ}^{μ}−a.s. By ergodicity f visits any open subset of D infinitely often (cf. (2.9)) P_{θ}^{μ}−a.s, hence f must be constant, and since μ_{θ}(f) = 0, f ≡ 0 follows. □

Remark. If d = 1 the differential equation A_{θ}f ≡ 0 is solved by the collection of scale functions S_{θ} given by (2.2). Hence S_{θ} ∉ D_{θ}. More generally, if d is arbitrary and f ∈ L^{1} (μ_{θ}) ∩ C^{2}(D) with A_{θ}f ≡ 0 and (2.5) holds, then f is constant: f(X) is a L^{1}−bounded P_{θ}^{μ}−martingale and one can copy the argument from the proof of Proposition 2.1.

3. Classes of estimation functions

With X given by (2.1), suppose that finitely many observations X_{0}, X_{Δ_{1}}, …, X_{Δ_{n}} are available, where 0 < Δ_{1} < ⋯ < Δ_{n}. We wish to estimate θ, assuming that the distribution of X is determined by P_{θ_{0}}^{μ} for some true parameter value θ_{0}. The estimation is performed using a flow of estimating functions G = (g_{t,θ})_{t>0,θ∈Θ} with each g_{t,θ} : D^{2} → ℝ^{p×1} belonging to L^{2} (Q_{t,θ}) , i.e.

E_{θ}^{μ}g_{t,θ}^{T}g_{t,θ} (X_{0}, X_{t}) < ∞ (t > 0, θ ∈ Θ)  (3.1)

and satisfying the vital unbiasedness condition

E_{θ}^{μ}g_{t,θ}(X_{0}, X_{t}) = 0 (t > 0, θ ∈ Θ).  (3.2)

Using g, an estimator θ̂ of θ is obtained by solving the equation

∑_{m=1}^{n} g_{Δ_{m}−Δ_{m−1},θ} (X_{Δ_{m−1}}, X_{Δ_{m}}) = 0  (3.3)

in θ. (Notation: Δ_{0} = 0).

(3.3) is of course the type of estimating equation used in the literature on inference for discretely observed diffusions, with the idea of using g depending on neighbouring observations motivated by the Markovian nature of X, which in particular renders the estimating equation for the maximum likelihood estimator of the form (3.3), see (3.6). Below we discuss some of the different types of g’s that have been suggested.

We shall from now on assume that the observations are equidistant, Δ_{m} = mΔ for some Δ > 0. Then (3.3) becomes

∑_{m=1}^{n} g_{Δ,θ} (X_{(m−1)Δ}, X_{mΔ}) = 0,  (3.4)

an equation that, as is well known, under mild regularity conditions leads to estimators that are consistent and asymptotically Gaussian as n → ∞, (see Section 4). An important condition on g is of course that g can distinguish different parameter values, e.g.

E_{θ}^{μ}g_{Δ,θ′} (X_{0}, X_{Δ}) = 0  (3.5)

iff θ = θ′. Although this condition is not formally required below, we subsume it and will comment further on its importance in Section 4, see the paragraph below (4.7).

We write g_{Δ,θ} ∈ E if (3.1), (3.2) hold for t = Δ and g_{Δ,θ}(x, y) is one time continuously differentiable in θ. In Section 7 we shall also need other partial derivatives. In accordance with the notation from the previous section, the derivatives are written

∂_{y}g_{t,θ}(x, y) = (\frac{∂}{∂y_{i}} g_{t,θ}^{k}(x, y))_{ki} ∈ ℝ^{p×d},
∂_{yy}^{2}g_{t,θ}(x, y) = (\frac{∂^{2}}{∂y_{i}∂y_{j}} g_{t,θ}^{k}(x, y))_{k(ij)} ∈ ℝ^{p×d^{2}},

g^{·}_{t,θ} (x, y) = (\frac{∂}{∂θ_{l}} g_{t,θ}^{k}(x, y))_{kl} ∈ ℝ^{p×p},
∂_{t}g_{t,θ}(x, y) = (\frac{∂}{∂t}g_{t,θ}^{k}(x, y))_{k} ∈ ℝ^{p×1}.

We write G ⊂ E if g_{t,θ} ∈ E for all t > 0, θ ∈ Θ.

From (3.4) we have p equations to solve for p unknowns θ_{1}, …, θ_{p}. The equations may involve estimating functions g of different types and we now list some of the types studied in the literature. When we refer to these types later, it is always assumed that G ⊂ E or, if a given Δ is considered, that g_{Δ,θ} ∈ E.

MLE. The flow (s_{t,θ})_{t>0,θ∈Θ}used for obtaining the maximum likelihood estimator (conditionally on X_{0}),

s_{t,θ}(x, y) = \frac{ṗ_{t,θ}^{T}(x, y)}{p_{t,θ}(x, y)}.  (3.6)

M. The class of martingale estimating functions: for a given k and Δ > 0, g_{Δ,θ}^{k} ∈ M if (∑_{m=1}^{n} g_{Δ,θ}^{k} (X_{(m−1)Δ}, X_{mΔ}) , F_{nΔ})_{n≥1} is a P_{θ}^{μ}−martingale.

S. The class of simple estimating functions: for a given k and Δ > 0, g_{Δ,θ}^{k} ∈ S if it is of the form g_{Δ,θ}^{k}(x, y) = f_{θ}(x) or = h_{θ}(y).

T . The class of explicit, transition dependent estimating functions: for a given k and Δ > 0, g_{Δ,θ}^{k} ∈ T if it is of the form

g_{Δ,θ}^{k}(x, y) = A_{θ}f(x) h(y) − f(x)A_{θ}h(y).  (3.7)

In Section 6 we shall introduce

R. The class of reversible estimating functions.

Notation. If K is one of the classes of estimating functions, we write g_{Δ,θ} ∈ K if all g_{Δ,θ}^{k} ∈ K. Also we write T (f, h)(x, y) for the right hand side of (3.7).

As we shall see, the martingale estimating functions are particularly important, not only for applications but also for the theory to be developed presently. The classes S and T are important because they provide explicit estimating functions – but T (and also R) are mostly relevant only if d = 1, see Section 6.

Of course (subject to mild regularity conditions) each coordinate of the MLEestimating function (3.6) belongs to M. Bibby and Sørensen [3] studied functions in M of the form

g_{Δ,θ}^{k}(x, y) = h_{Δ,θ}(x) (f(y) − π_{Δ,θ}f(x)) ,  (3.8)

in particular with f(y) = y or = y^{2} in the case d = 1. Other special cases of (3.8) are obtained if f = f_{θ} is an eigenfunction for A_{θ} corresponding to an eigenvalue λ_{θ} < 0, in which case (3.8) becomes

g_{Δ,θ}^{k}(x, y) = h_{Δ,θ}(x) (f_{θ}(y) − e^{Δλ_{θ}}f(x)) ,

see Kessler and Sørensen [11].

The bilinearity inherent in (3.8) immediately gives rise to the following large class in M,

g_{Δ,θ}^{k}(x, y) = ∑_{q=1}^{r} h_{Δ,θ}^{q}(x) (f^{q}(y) − π_{Δ,θ}f^{q}(x))  (3.9)

for different pairs (h_{Δ,θ}^{q}, f_{θ}^{q})_{1≤q≤r} , see e.g. Sørensen [15], Section 3.

The defining property of M requires that

E_{θ}^{ν} (g_{Δ,θ}^{k} (X_{(m−1)Δ}, X_{mΔ}) |X_{(m−1)Δ} ) = 0 P_{θ}^{μ} − a.s.

for all m ∈ ℕ. Thus g_{Δ,θ}^{k} ∈ M if (and essentially only if)

g_{Δ,θ}^{k,∗} ≡ 0  (3.10)

where in general g_{Δ,θ}^{k,∗} ∈ 1_{θ}^{⊥} is given by

g_{Δ,θ}^{k,∗}(x) := ∫_{D} dy p_{Δ,θ}(x, y)g_{Δ,θ}^{k}(x, y),

see Kessler [8] and also [15]. Another useful characterization is that g_{Δ,θ}^{k} ∈ M iff g_{Δ,θ}^{k} ⊥ {φ ∈ L^{2} (Q_{Δ,θ}) : φ (x, y) = h(x) for some h} .

Both the characterization (3.10) of M and the definition of g_{Δ,θ}^{k,∗} will be used frequently in the remainder of the paper.

The class S was studied in particular by Kessler [9]. Among his examples are

g_{Δ,θ}(x, y) = \frac{μ̇_{θ}^{T} (x)}{μ_{θ}(x)},  (3.11)

the collection of estimating functions one would use if the X_{mΔ} were independent and identically distributed with the correct marginal distribution μ_{θ_{0}}; also g_{Δ,θ}^{k}(x, y) = f_{θ}(x) with f_{θ} an eigenfunction for A_{θ} with an eigenvalue < 0 (in which case (3.2) is automatic), and finally and most importantly, the class of simple, explicit estimating functions of the form

g_{Δ,θ}^{k}(x, y) = A_{θ}f(x)  (3.12)

(see also Baddeley [2] and Hansen and Scheinkman [7], C1, p. 774) which satisfy (3.2) because of (2.6).

Hansen and Scheinkman [7], C2, p. 775 also introduced the class T of estimating functions, which we shall discuss in Section 6 and which in the version presented here, apply only to reversible models, i.e. essentially only for d = 1: for d > 1 (3.2) will hold only in special models or for particular choices of the f, h appearing in (3.7).

In some of the cases discussed above, we stressed that the estimating functions were explicit, meaning essentially that they could be written in closed analytic form. Of course only in very few cases is MLE explicit and for (3.8) to be explicit one may have to use particular choices for f. Also, for the examples involving eigenfunctions, explicit expressions are only rarely available. Thus the main classes of explicit estimating functions are (3.12) and T . Here, while the collection of functions obtained from (3.12) when f varies, forms a linear space, the bilinearity in (3.7) permits the definition of a large class of explicit functions, viz. finite sums

g_{Δ,θ}^{k}(x, y) = ∑_{q=1}^{r} (A_{θ}f^{q}(x) h^{q}(y) − f^{q}(x)A_{θ}h^{q}(y)) .  (3.13)

A different possibility, not yet fully explored, for obtaining explicit estimating functions, is to look for real-valued f̃_{t,θ}(x) such that (f̃_{t,θ} (X_{t}) , F_{t})_{t≥0} is a continuous-time P_{θ}^{μ}−martingale, and then use

g_{Δ}^{k}(x, y) = h_{Δ,θ}(x) (f̃_{Δ,θ}(y) − f̃_{0,θ}(x)) ,

which is in M. Here, subject to differentiability and integrability requirements, f̃ must satisfy

∂_{t}f̃_{t,θ} + A_{θ}f̃_{t,θ} ≡ 0,

as is seen from Itô’s formula.

A last comment for now on estimating functions is that it is perfectly possible, but perhaps not very useful, to allow f in (3.8) and f and h in (3.7) to depend on both θ and Δ: the critical unbiasedness (3.2) still holds.

4. Asymptotics

We shall briefly review the asymptotic theory for the estimators obtained from the estimating equations (3.4) with g satisfying the unbiasedness condition (3.2) and the integrability condition (3.1). The asymptotics are done for equidistant observations (X_{mΔ})_{0≤m≤n} for Δ > 0 fixed, n → ∞ with the diffusion X satisfying Assumptions 1,2,3 from Section 2.

Recall the definition (2.7) of the potential operator U_{Δ,θ} acting on functions in D_{0,θ} with values in 1_{θ}^{⊥}. Of course

(I − π_{Δ,θ}) U_{Δ,θ}f = f (f ∈ D_{0,θ}).  (4.1)

Suppose now that G = (g_{t,θ}) is a flow of estimating functions in E. Since each g_{Δ,θ}^{k,∗} ∈ 1_{θ}^{⊥} (because of (3.2) and since g_{Δ,θ}^{k} ∈ L^{2}(Q_{Δ,θ}), see (3.1)), we find that

ğ_{Δ,θ}(x, y) := g_{Δ,θ}(x, y) + U_{Δ,θ}g_{Δ,θ}^{∗}(y) − U_{Δ,θ}g_{Δ,θ}^{∗}(x)  (4.2)

(with U acting separately on each component g^{k,∗}) defines a p−dimensional estimating function with all components in M: (3.10) holds for ğ because using (4.1) it is seen that

ğ_{Δ,θ}^{∗}(x) = ∫_{D} dy p_{Δ,θ}(x, y)ğ_{Δ,θ}(x, y)
= g_{Δ,θ}^{∗}(x) + π_{Δ,θ} (U_{Δ,θ}g_{Δ,θ}^{∗}) (x) − U_{Δ,θ}g_{Δ,θ}^{∗}(x)
= 0.

We shall call ğ_{Δ,θ} the martingale estimating function associated with g_{Δ,θ}. It will play a very important role in the remainder of this paper. The fact that any estimating function is associated with an element of M means that for theoretical purposes, it suffices to consider martingale estimating functions. (For practical purposes ğ is of course useless – with the transition operators known explicitly only in exceptional cases, there are of course even fewer (if any) examples where the potential operators can be expressed explicitly).

The potential operators appear naturally in the statement of the central limit theorem for ergodic Markov chains. In our setup we obtain as a direct consequence of the martingale central limit theorem and the ergodicity property (2.8) that

\frac{1}{\sqrt{n}} ∑_{m=1}^{n} g_{Δ,θ} (X_{(m−1)Δ}, X_{mΔ}) →^{d_{θ}} N_{p} (0, v_{Δ,θ}(ğ))  (4.3)

as n → ∞, where the limiting covariance matrix is

v_{Δ,θ}(ğ) = E_{θ}^{μ}ğ_{Δ,θ}ğ_{Δ,θ}^{T} (X_{0}, X_{Δ}) ,  (4.4)

cf. Lemma 7 in Dacunha-Castelle and Florens-Zmirou [4].

Notation. →^{d_{θ}} means convergence in distribution under P_{θ}^{μ}. N_{p}(ξ, Γ) denotes the p−dimensional Gaussian distribution with mean vector ξ, covariance matrix Γ.

A comment on the derivation of (4.3): a direct application of the martingale central limit theorem gives that (4.3) holds with g replaced by ğ. But

\frac{1}{\sqrt{n}} ∑_{m=1}^{n} (ğ_{Δ,θ} − g_{Δ,θ}) (X_{(m−1)Δ}, X_{mΔ}) = \frac{1}{\sqrt{n}} (U_{Δ,θ}g_{Δ,θ}^{∗}(X_{nΔ}) − U_{Δ,θ}g_{Δ,θ}^{∗}(X_{0}))

which converges to 0 in P_{θ}^{μ}−probability and (4.3) itself follows.

For g_{Δ,θ} a simple estimating function, ğ_{Δ,θ} was used by Florens-Zmirou [5] and also by Kessler [9]. Sørensen (the review [16] for instance), used (4.3) together with a Taylor expansion to obtain asymptotic normality for the estimator θ̂_{n} solving the estimating equation (3.4), viz.

\sqrt{n} (θ̂_{n} − θ) →^{d_{θ}} N (0, Λ_{Δ,θ}^{−1}(g) v_{Δ,θ}(ğ) (Λ_{Δ,θ}^{−1}(g))^{T} )  (4.5)

where

Λ_{Δ,θ}(g) := E_{θ}^{μ} g^{·}_{Δ,θ} (X_{0}, X_{Δ}) ,  (4.6)

i.e. the (k, l)’th element of the p × p−matrix Λ_{Δ,θ}(g) is E_{θ}^{μ}∂_{θ_{l}}g_{Δ,θ}^{k} (X_{0}, X_{Δ}) . The asymptotic variance matrix in (4.5) we denote by var_{Δ,θ} (g, θ̂) ,

var_{Δ,θ} (g, θ̂) = Λ_{Δ,θ}^{−1}(g) v_{Δ,θ}(ğ) (Λ_{Δ,θ}^{−1}(g))^{T} .  (4.7)

We shall not here discuss the precise conditions under which (4.5) holds, but refer the reader to the literature, e.g. [16]. Typically the assertion of the result yielding (4.5) is that with a P_{θ_{0}}^{μ}−probability tending to 1, (θ_{0} denoting the true parameter value), (3.4) has a consistent solution which is asymptotically Gaussian as stated in (4.5). For this it is important that g_{Δ,θ} distinguishes between parameter values, e.g. that (3.5) holds. Note that obviously Λ_{Δ,θ}(ğ) = Λ_{Δ,θ}(g).

Most of what follows relies on (4.5) being true. We shall make some formal assumptions so that the statement at least makes sense and otherwise refer to g_{Δ,θ} ∈ E as well behaved if it satisfies Assumption 4 below and if, for all θ_{0}, there is with P_{θ_{0}}^{μ}−probability tending to 1, a consistent solution to (3.4), which is asymptotically Gaussian according to (4.5) with θ = θ_{0}.

Assumption 4. Assume that g_{Δ,θ} ∈ E that v_{Δ,θ}(ğ) > 0, that ∂_{θ_{l}}g_{Δ,θ}^{k} ∈ L^{1} (Q_{Δ,θ}) for all k, l and that Λ_{Δ,θ}(g) is non-singular.

The assumption that v_{Δ,θ}(ğ) > 0 means essentially that the estimating equation enables us to estimate all the parameters θ_{k}. The assumption that Λ be non-singular, which is obviously necessary for (4.5) to make sense, is more critical and may fail in innocuous looking situations, as we shall now see. Here and below, by a reparametrization we mean a differentiable homeomorphism ι : Θ → Θ̃, Θ̃ an open subset of ℝ^{p}, with θ̃ = ι(θ) the parameter vector corresponding to θ after the reparametrization.

Example 4.1. Suppose that μ_{θ} does not depend on all the parameters, i.e. that, possibly after a reparametrization, μ_{θ} depends on (θ_{1}, …, θ_{p′}) only where p′ < p. Then θ cannot be estimated according to (4.5)–asymptotics using only simple estimating functions, more precisely Λ_{Δ,θ}(g) is singular for all well behaved g_{Δ,θ} ∈ S for which the differentiation under the integral sign below is valid: suppose g_{Δ,θ}(x, y) = f_{Δ,θ}(x) so that

Λ_{Δ,θ}(g) = ∫_{D} dx μ_{θ}(x)ḟ_{Δ,θ}(x).

Then for l > p′, using (3.2),

0 = ∂_{θ_{l}} (∫_{D} dx μ_{θ}(x)f_{Δ,θ}(x))
= ∫_{D} dx μ_{θ}(x)∂_{θ_{l}}f_{Δ,θ}(x),

hence the last p − p′ columns of Λ_{Δ,θ}(g) vanish.

Example 4.2. Suppose that, possibly after a reparametrization, the differential operator A_{θ} is linear in θ′ = (θ_{1}, …, θ_{p′}) for some p′ ≥ 1, in the sense that

b_{θ}(x) = b̃_{θ_{−}}(x)θ′, C_{θ}(x) = C̃_{θ_{−}}(x)θ′

for some b̃_{θ_{−}}(x) ∈ ℝ^{d×p′}, C̃_{θ_{−}}(x) ∈ ℝ^{d^{2}×p′}, writing θ_{−} = (θ_{p′+1}, …, θ_{p}). (For p′ = p, b̃, C̃ must not depend on θ). Then, subject to mild analytic conditions, Λ_{Δ,θ}(g) is singular for all well behaved g, which are either in S of the form (3.12),

g_{Δ,θ}^{k}(x, y) = A_{θ}h_{Δ,θ}^{k}(x)  (4.8)

for all k, or, for the reversible models where the members of T can be used, of the form

g_{Δ,θ}^{k}(x, y) = T (f_{Δ,θ}^{k}, h_{Δ,θ}^{k}) (x, y),  (4.9)

see (3.7) and the notation introduced just below (3.7).

To see that Λ_{Δ,θ}(g) is singular, consider (4.9) for a fixed k. Then by calculation, omitting subscripts from f_{Δ,θ}^{k}, h_{Δ,θ}^{k},

∑_{l=1}^{p′} θ_{l}∂_{θ_{l}}g_{Δ,θ}^{k}(x, y) = g_{Δ,θ}^{k}(x, y) + ∑_{l=1}^{p′} θ_{l} (T (∂_{θ_{l}}f^{k}, h^{k}) + T (f^{k}, ∂_{θ_{l}}h^{k})) (x, y),

and taking expectations with respect to Q_{Δ,θ} it follows that the linear combination with coefficients θ_{l} of the first p′ columns of Λ_{Δ,θ}(g) is equal to 0. (Note that for any θ = (0, … 0, θ_{p′+1}, … θ_{p}) , A_{θ} ≡ 0 which is not a diffusion generator, and hence θ′ ≠ 0 for all θ ∈ Θ).

Simple examples of models that have the linearity property described here include the one-dimensional Ornstein-Uhlenbeck models

\begin{matrix}dX_{t} = −θX_{t} dt + \sqrt{2θ} dB_{t} & (θ > 0) \\ dX_{t} = −θX_{t} dt + σ dB_{t} & (θ, σ > 0) \\ dX_{t} = (α − θX_{t}) dt + σ dB_{t} & (θ, σ > 0, α ∈ ℝ). \end{matrix}

Note that for the first case, μ_{θ} = N(0, 1) does not depend on θ, so that no simple estimating functions can be used. Note also that in the two last cases, if one parameter is assumed known it is perfectly possible to find a combination of g^{k} of the form (4.8) or (4.9) for each k, such that Λ is non-singular.

5. Optimality

Let Δ > 0 be fixed, and consider a given class (g_{Δ,θ})_{θ∈Θ} of well behaved estimation functions. To find the best element from the class one would try to minimize var_{Δ,θ} (g, θ̂) from (4.7), (which for p ≥ 2 means minimizing in the partial ordering of symmetric positive semidefinite matrices, cf. the notation introduced after Assumption 2 above). Of course there may not be a variance minimizing g (in particular not if p ≥ 2), and even if there is, it may be difficult to find. However, with a suitable linear structure imposed on the class of estimating functions, it is possible theoretically to describe the optimal g, but actually finding it may still be difficult, and even then the expression may be too complicated for practical purposes. In Section 7 we shall return to the optimality problem, proposing a solution that yields explicit results and which we believe can be useful in practice.

The proofs of the optimality inequalities rely on the following result, where, if M = (M_{ij}) is a matrix-valued random variable, EM is the matrix (EM_{ij}) of expectations.

Lemma 5.1. Suppose that Y, Z, S are matrix-valued random variables of dimensions a × b, a × b, b × b respectively with S symmetric and strictly positive definite with probability 1. Assuming that all entries in the matrices Y Z^{T} , Y SY^{T} , ZS^{−1}Z^{T} are integrable, it then holds that:

(i) if E (Y SY^{T} ) is non-singular, then

E (ZY^{T} ) (E (Y SY^{T} ))^{−1} E (Y Z^{T} ) ≤ E (ZS^{−1}Z^{T} );  (5.1)

(ii) if E (Y SY^{T} ) and E(Y Z^{T}) are non-singular, then

(E (ZY^{T} ))^{−1} (E (ZS^{−1}Z^{T} )) (E (Y Z^{T} ))^{−1} ≥ (E (Y SY^{T} ))^{−1} ; (5.2)

(iii) in (5.1), (5.2) there is equality if for some non-random, non-singular K ∈ ℝ^{a×a}, Y = KZS^{−1}, equivalently if Z = K^{−1}Y S.

Proof. (ii) follows directly from (i) and (iii) is verified immediately. To show (5.1), define

Σ_{11} = E (ZS^{−1}Z^{T} ) , Σ_{12} = E (ZY^{T} ) , Σ_{22} = E (Y SY^{T} ) ,

and consider the symmetric 2a × 2a matrix

Σ = ( \begin{matrix}Σ_{11} & Σ_{12} \\ Σ_{12}^{T} & Σ_{22} \end{matrix} ) .

If Σ ≥ 0 also Σ_{11} − Σ_{12}Σ_{22}^{−1}Σ_{12}^{T} ≥ 0, which is precisely (5.1). So we complete the proof by showing that

s(u, v) := ( u^{T} v^{T} ) Σ ( \begin{matrix}u \\ v \end{matrix} ) ≥ 0

for all u, v ∈ ℝ^{a×1}. But

s(u, v) = E (u^{T} ZS^{−1}Z^{T} u + 2u^{T} ZY^{T} v + v^{T} Y SY^{T} v) ,

and defining ũ = Z^{T} u, ṽ = SY^{T} v, the random variable under the expectation becomes

(ũ^{T} + ṽ^{T} ) S^{−1} (ũ + ṽ) ≥ 0.
□

In the scalar case a = b = 1, (5.1), (5.2) follow from the Cauchy-Scwartz inequality and equality holds iff Y = cZ/S for some constant c ≠ 0. If b = 1, S ≡ 1, (5.2) gives for Y, Z a−dimensional random vectors

(E (ZY^{T} ))^{−1} E(ZZ^{T}) (E (Y Z^{T} ))^{−1} ≥ (E (Y Y^{T} ))^{−1}  (5.3)

with equality if Z = K^{−1}Y for some non-random matrix K. Inequalities of this type were used by Godambe and Heyde [6].

For the first optimality result, that maximum-likelihood is best, we consider the estimation function

s_{Δ,θ} = \frac{ṗ_{Δ,θ}^{T}}{p_{Δ,θ}} ,

see (3.6).

Notation. To lighten the notation, if Δ > 0 is given and ψ(x, y) is a scalar, vector- or matrix-valued function on D × D, we also write ψ for the random variable ψ(X_{0}, X_{Δ}).

Assuming s_{Δ,θ} to be well behaved, the asymptotic variance for the MLE is of course

var_{Δ,θ} (s, θ̂) = (E_{θ}^{μ}s_{Δ,θ}s_{Δ,θ}^{T})^{−1}  (5.4)

as is seen from (4.7) with g = s, provided differentiation and integration can be interchanged as is done in

0 = ∂_{θ}E_{θ}^{x}s_{Δ,θ} = ∫_{D} dy (s_{Δ,θ}(x, y)ṗ_{Δ,θ}(x, y) + p_{Δ,θ}(x, y)ṡ_{Δ,θ}(x, y))

for all x, since then (see (4.6))

Λ_{Δ,θ}(s) = −E_{θ}^{μ}s_{Δ,θ}s_{Δ,θ}^{T}.

Proposition 5.2. If s_{Δ,θ} is well behaved with the asymptotic variance for the MLE given by (5.4), then

var_{Δ,θ} (g, θ̂) ≥ var_{Δ,θ} (s, θ̂)

for all Δ, θ and any well behaved g, provided E_{θ}^{μ}ğ_{Δ,θ}s_{Δ,θ}^{T} is non-singular and, for all x,

∂_{θ} ∫_{D} dy (p_{Δ,θ}ğ_{Δ,θ}) (x, y) = ∫_{D} dy ∂_{θ} (p_{Δ,θ}ğ_{Δ,θ}) (x, y).  (5.5)

Proof. Because E_{θ}^{x}ğ_{Δ,θ} = 0 (which is just (3.10) applied to ğ), the left hand side

of (5.5) is 0. Thus E_{θ}^{x} ğ^{·}_{Δ,θ}= −E_{θ}^{x}ğ_{Δ,θ}s_{Δ,θ}^{T}, and then also

E_{θ}^{μ} ğ^{·}_{Δ,θ}= −E_{θ}^{μ}ğ_{Δ,θ}s_{Δ,θ}^{T}.

Now apply (5.3) with Z = ğ_{Δ,θ}, Y = s_{Δ,θ}. □

The next result is a simple multivariate generalization of Kessler’s [9] projection result Lemma 3.3.

Let Δ > 0 be given and let for each θ, L_{θ} be a closed linear subspace of the Hilbert space L^{2}(Q_{Δ,θ}) such that ∫_{D^{2}} dxdy q_{Δ,θ}(x, y)κ(x, y) = 0 for all κ ∈ L_{θ}. Let L^{⌣}_{θ} denote the closure in L^{2}(Q_{Δ,θ}) of the space of functions κ̆ of the form κ̆(x, y) = κ(x, y) + U_{Δ,θ}κ^{∗}(y) − U_{Δ,θ}κ^{∗}(y) for some κ ∈ L_{θ}, cf. (4.2). Also define L_{θ,p} as the linear space of all p−variate functions g_{Δ,θ} = (g_{Δ,θ}^{k})_{1≤k≤p} with each

g_{Δ,θ}^{k} ∈ L_{θ} and finally define

γ_{Δ,θ} = proj_{L^{⌣}}_{θ} \frac{ṗ_{Δ,θ}^{T}}{p_{Δ,θ}}

where the projection is performed componentwise, within L^{2}(Q_{Δ,θ}).

Proposition 5.3. Suppose that γ_{Δ,θ} is well behaved. Then for any well behaved g_{Δ,θ} ∈ L_{p,θ},

var_{Δ,θ}(g, θ̂) ≥ var_{Δ,θ}(γ, θ̂) = (E_{θ}^{μ}γ_{Δ,θ}γ_{Δ,θ}^{T})^{−1} .  (5.6)

Proof. γ is characterized by the properties γ_{Δ,θ}^{k} ∈L^{⌣}_{θ} for each k and

E_{θ}^{μ}κ̆ (r_{Δ,θ}^{k} − γ_{Δ,θ}^{k}) = 0  (5.7)

for all k and all κ ∈ L_{θ}, where r_{Δ,θ} = ṗ_{Δ,θ}^{T}/p_{Δ,θ}. But if differentiation and integration can be interchanged so that

0 = ∂_{θ} ∫_{D} dxdy (q_{Δ,θ}ğ_{Δ,θ})(x, y) = ∫_{D} dxdy ∂_{θ}(q_{Δ,θ}ğ_{Δ,θ})(x, y),

using (5.7) with κ an arbitrary component of g_{Δ,θ}, we obtain

Λ_{Δ,θ}(g) = −E_{θ}^{μ}ğ_{Δ,θ}r_{Δ,θ}^{T} = −E_{θ}^{μ}ğ_{Δ,θ}γ_{Δ,θ}^{T}.  (5.8)

Now use (5.3) with Z = ğ_{Δ,θ}, Y = γ_{Δ,θ} to obtain the lower bound on the right of (5.6) for var_{Δ,θ}(g, θ̂). Since γ_{Δ,θ} ∈ M, it follows finally from (5.8) with g = ğ = γ that var_{Δ,θ}(γ, θ̂) has the form stated in (5.6). □

Remark. With p ≥ 2 it is only possible to formulate this result if all components g_{Δ,θ}^{k} are allowed to vary in the same space L_{θ}. But for estimation purposes it is also quite natural that it should be so: introducing a subspace L_{θ}^{k} for each k would imply that the p equations used for estimating θ had been numbered in some fashion, something which would be quite artificial since for instance an arbitrary permutation of the p equations does not change the estimator θ̂.

Example 5.4. Let d = 1. Kessler [9] studied the case where L_{θ} = S_{θ}, the space of simple unbiased estimating functions g_{Δ,θ}(x, y) = f_{Δ,θ}(x), and found the projection using eigenfunction expansions. We give here an expression valid in general, that does not involve eigenvalues or eigenfunctions.

We have

L^{⌣}_{θ}=S^{⌣}_{θ}= {h(x) + U_{Δ,θ}h(y) − U_{Δ,θ}h(x) : h ∈ 1_{θ}^{⊥}}

and write

ψ^{k}(x) + U_{Δ,θ}ψ^{k}(y) − U_{Δ,θ}ψ^{k}(x)

for the projection onto S^{⌣}_{θ} of r^{k} := ∂_{θ_{k}}p_{Δ,θ}/p_{Δ,θ}. Thus

E_{θ}^{μ}Z (h(X_{0}) + U_{Δ,θ}h(X_{Δ}) − U_{Δ,θ}h(X_{0})) = 0  (5.9)

for all h ∈ 1_{θ}^{⊥}, where

Z = r^{k} (X_{0}, X_{Δ}) − ψ^{k}(X_{0}) − U_{Δ,θ}ψ^{k}(X_{Δ}) + U_{Δ,θ}ψ^{k}(X_{0}).

The idea is now to rewrite the expectation in (5.9) as an inner product in L^{2} (μ_{θ}) between h and some function ψ̃ ∈ 1_{θ}^{⊥}. It then follows that ψ̃ ≡ 0, and from this equation ψ^{k} is identified. But using stationarity, the Markov property and time reversal (e.g. that q_{Δ,θ}(x, y) is symmetric in x, y and that U_{Δ,θ} is selfadjoint, see Section 6), writing π = π_{Δ,θ}, U = U_{Δ,θ}, one finds

E_{θ}^{μ}Zh(X_{0}) = E_{θ}^{μ}r^{k∗}(X_{0})h(X_{0}),
E_{θ}^{μ}ZUh(X_{Δ}) = E_{θ}^{μ} (r^{k∗} + \frac{∂_{θ_{k}}μ_{θ}}{μ_{θ}} − (π \frac{∂_{θ_{k}}μ_{θ}}{μ_{θ}} ) − (I + π)ψ^{k}) (X_{0})Uh(X_{0})
= E_{θ}^{μ} (Ur^{k∗} + \frac{∂_{θ_{k}}μ_{θ}}{μ_{θ}} − (2U − I) ψ^{k}) (X_{0})h(X_{0}),
E_{θ}^{μ}Z(Uh)(X_{0}) = E_{θ}^{μ}Ur^{k∗}(X_{0})h(X_{0}).

Since r^{k∗} ≡ 0,

ψ̃(x) = \frac{∂_{θ_{k}}μ_{θ}}{μ_{θ}} (x) − (2U − I) ψ^{k}(x)

so the equation ψ̃ ≡ 0 translates into

(I − π)\frac{∂_{θ_{k}}μ_{θ}}{μ_{θ}} = (I + π)ψ^{k}

or, equivalently,

ψ^{k} = U_{2Δ,θ} (I − π_{Δ,θ})^{2} \frac{∂_{θ_{k}}μ_{θ}}{μ_{θ}} .

We have shown that

proj_{S^{⌣}}_{θ} \frac{ṗ_{Δ,θ}^{T}}{p_{Δ,θ}} (x, y) = ψ(x) + U_{Δ,θ}ψ(y) − U_{Δ,θ}ψ(x),

where ψ is the unique solution with all ψ^{k} ∈ 1_{θ}^{⊥} to the equation

(I − π_{Δ,θ}) \frac{μ̇_{θ}^{T}}{μ_{θ}} = (I + π_{Δ,θ}) ψ.

Example 5.5. For p = 1, Bibby and Sørensen [3] studied optimality of estimating functions of the form (3.8). In our formulation this would correspond to taking, for a given f ∈ L^{2}(μ_{θ}) \underline{not} depending on θ,

L_{θ} = {h(x) (f(y) − π_{Δ,θ}f(x)) : h ∈ L^{2} (μ_{θ})} .

In this case L^{⌣}_{θ}= L_{θ} and the projection takes the form

γ_{Δ,θ}(x, y) = h_{Δ,θ}^{opt}(x) (f(y) − π_{Δ,θ}f(x))

and using (5.7) it is not difficult to find that the optimal h is given by

h_{Δ,θ}^{opt} = \frac{∂_{θ} (π_{Δ,θ}f)}{π_{Δ,θ}f^{2} − (π_{Δ,θ}f)^{2}} ,

cf. [3], provided

∂_{θ} ∫ dy κ(y)p_{Δ,θ}(x, y) = ∫ dy κ(y)ṗ_{Δ,θ}(x, y)

for κ ≡ f, κ ≡ 1.

For p ≥ 2 the natural approach is to fix p functions f^{k} ∈ L^{2} (μ_{θ}) not depending on θ, and then use

L_{θ} =L^{⌣}_{θ}= {∑_{k=1}^{p} h^{k}(x) (f^{k}(y) − π_{Δ,θ}f^{k}(x)) : h^{1}, …, h^{p} ∈ L^{2} (μ_{θ})} ,  (5.10)

cf. (3.9). The result is

h_{Δ,θ}^{opt} = [π_{Δ,θ} (ff^{T} ) − (π_{Δ,θ}f) (π_{Δ,θ}f)^{T} ]^{−1} ∂_{θ} (π_{Δ,θ}f)

with the k’th column of h_{Δ,θ}^{opt} ∈ ℝ^{p×p} giving the choices for the h^{1}, …, h^{p} in (5.10) when projecting ∂_{θ_{k}}p_{Δ,θ}/p_{Δ,θ} onto L_{θ}. For discussions of optimality for this class, see Kessler [8] (quoted in Sørensen [15], Theorem 3.1), and Pedersen [12].

6. Time reversal

Since the diffusion X has an invariant measure, it is possible to define X as a strictly stationary process X = (X_{t})_{−∞<t<∞} in doubly infinite time, each X_{t} having density μ_{θ}. But then also the reversed process \overleftarrow{X} = (\overleftarrow{X}_{t})_{−∞<t<∞} , where

\overleftarrow{X}_{t} = X_{−t}, is a diffusion with invariant density μ_{θ} and transition densities

\overleftarrow{p}_{t,θ}(x, y) = μ_{θ}(y)p_{t,θ}(y, x)\frac{1}{μ_{θ}(x)}.  (6.1)

If d = 1, for the diffusions we are considering, \overleftarrow{p}_{t,θ} = p_{t,θ}, i.e. the onedimensional diffusions are reversible. Put differently, for d = 1, the transition operators π_{t,θ} (acting on L^{2} (μ_{θ})) are selfadjoint, and the generator A_{θ} (acting on D_{θ}) is selfadjoint. (A quick proof of the reversibility property is the following: as noted p. 7 above, for d = 1, the space DC of functions with compact support is dense in 1_{θ}^{⊥}, and it therefore suffices to show that for any f, h ∈ DC,

∫_{D} dx μ_{θ}(x)A_{θ}f(x)h(x) = ∫_{D} dx μ_{θ}(x)f(x)A_{θ}h(x).

But A_{θ}f = k_{θ} (f′/S′_{θ})′ (see (2.2) for the definition of S′_{θ}), for some consatnt k_{θ} and by partial integration the expression on the left then becomes −k_{θ} ∫ dx (f′h′) /S′_{θ}, which is symmetric in f, h).

For d ≥ 1, it is known (see e.g. the overview in [7]) that \overleftarrow{X} satisfies the stochastic differential equation

d\overleftarrow{X}_{t} = \overleftarrow{b}_{θ}(\overleftarrow{X}_{t}) dt + \overleftarrow{σ}_{θ}(\overleftarrow{X}_{t}) d\overleftarrow{B}_{t}

where \overleftarrow{B} is a d−dimensional Brownian motion, and

\overleftarrow{b}_{θ}^{i}(x) = −b^{i}(x) + \frac{1}{μ_{θ}(x)} ∑_{j=1}^{d} ∂_{x_{j}} (μ_{θ}C_{θ}^{ij}) (x), (1 ≤ i ≤ d),  (6.2)

\overleftarrow{C}_{θ}(x) := (\overleftarrow{σ}_{θ}\overleftarrow{σ}_{θ}^{T} ) (x) = C_{θ}(x),

so that typically \overleftarrow{b}_{θ} ≠ b_{θ}, while always \overleftarrow{C}_{θ} = C_{θ}. Thus, only in special cases, is a multidimensional diffusion model reversible.

In the remainder of this section, unless explicitly stated otherwise, we assume that X is reversible for all θ (with d = 1 by far the most important case). Let G = (g_{t,θ})_{t≥0,θ∈Θ} be a flow of well behaved estimating functions, and define for each t, θ,

\overleftarrow{g}_{t,θ}(x, y) = g_{t,θ}(y, x).

Because of the reversibility, \overleftarrow{g} satisfies (3.1), (3.2). If \overleftarrow{g}_{Δ,θ} is also well behaved for a given Δ, the estimating function \overleftarrow{g}_{Δ,θ} may be used as an alternative to g_{Δ,θ}, and as we shall now see, by combining the two by simply averaging, one may obtain an estimating function which is better than either.

In general models (reversible or not), we shall call an estimating function g_{Δ,θ} such that E_{θ}^{μ}\overleftarrow{g}_{Δ,θ}(X_{0}, X_{Δ}) = 0 for all θ, reversible if (cf. (4.2))

g_{Δ,θ}(x, y) + U_{Δ,θ}g_{Δ,θ}^{∗}(y) − U_{Δ,θ}g_{Δ,θ}^{∗}(x) = g_{Δ,θ}(y, x) + U_{Δ,θ} ^{∗}g_{Δ,θ}(y) − U_{Δ,θ} ^{∗}g_{Δ,θ}(x)

(6.3)

for Q_{Δ,θ}−almost all (x, y). Here

∗g_{Δ,θ}(x) = ∫_{d} dy p_{Δ,θ}(x, y)g_{Δ,θ}(y, x),

so (6.3) merely states that the martingale estimating functions associated with g_{Δ,θ} and \overleftarrow{g}_{Δ,θ} are the same.

We denote the class of reversible estimating functions by R (cf. p. 11).

Again, from now on X is assumed to be reversible.

Proposition 6.1. If g, \overleftarrow{g} are both well behaved, then

var_{Δ,θ}(\overleftarrow{g}, θ̂) = var_{Δ,θ}(g, θ̂).  (6.4)

Furthermore, if also \overline{g} := \frac{1}{2} (\overleftarrow{g} + g) is well behaved, then

var_{Δ,θ}(\overline{g}, θ̂) ≤ var_{Δ,θ} (g, θ̂)  (6.5)

with equality if and only if g ∈ R.

Proof. (6.4) is intuitively obvious and corresponds to comparing the estimates of θ obtained using g as estimating function on the observations (X_{mΔ})_{0≤m≤n} and (\overleftarrow{X}_{mΔ})_{0≤m≤n} = (X_{−mΔ})_{0≤m≤n} respectively. Note however the asymmetry as n → ∞ with the most recent observations, X_{nΔ} and X_{−nΔ} respectively, being added after, respectively before, the earlier observations. We shall therefore give a formal proof of (6.4) which, as it turns out, is not an entirely trivial matter.

Clearly Λ_{Δ,θ}(\overleftarrow{g}) = Λ_{Δ,θ}(g) (see (4.6)) by reversibility, so we need only show that

E_{θ}^{μ} \overleftarrow{g}^{⌣}\overleftarrow{g}^{⌣}^{T} = E_{θ}^{μ}ğğ^{T} ,  (6.6)

where

\overleftarrow{g}^{⌣} (x, y) = g(y, x) + U ^{∗}g(y) − U ^{∗}g(x).

(To ease the notation, the subscripts _{Δ,θ} have been omitted from \overleftarrow{g}^{⌣}, g, U). But writing out

ğğ^{T} = (g + (Ug^{∗}) (X_{Δ}) − (Ug^{∗}) (X_{0})) (g^{T} + (Ug^{∗})^{T} (X_{Δ}) − (Ug^{∗})^{T} (X_{0})) ,

into 9 terms, taking expectations, conditioning on X_{0} or X_{Δ} as appropriate and using reversibility and stationarity, one arrives at

E_{θ}^{μ}ğğ^{T} = E_{θ}^{μ}gg^{T}
+E_{θ}^{μ}(^{∗}g (Ug^{∗})^{T} − g^{∗} (Ug^{∗})^{T} + (Ug^{∗}) ^{∗}g^{T} + (Ug^{∗}) (Ug^{∗})^{T}

− (πUg^{∗}) (Ug^{∗})^{T} − (Ug^{∗}) g^{∗T} − (Ug^{∗}) (πUg^{∗})^{T} + (Ug^{∗}) (Ug^{∗})^{T}),

writing π = π_{Δ,θ}, and where all terms in the last expectation are evaluated at X_{0}. Using πU = U − I this collapses to

E_{θ}^{μ}ğğ^{T} = E_{θ}^{μ}gg^{T} + E_{θ}^{μ} (^{∗}g (Ug^{∗})^{T} + (Ug^{∗}) ^{∗}g^{T} ) .

The same calculation applied to \overleftarrow{g} gives

E_{θ}^{μ} \overleftarrow{g}^{⌣}\overleftarrow{g}^{⌣}^{T} = E_{θ}^{μ}\overleftarrow{g}\overleftarrow{g}^{T} + E_{θ}^{μ} (^{∗}\overleftarrow{g} (U\overleftarrow{g}^{∗})^{T} + (U\overleftarrow{g}^{∗})^{∗} \overleftarrow{g}^{T} )

and (6.6) follows since E_{θ}^{μ}gg^{T} = E_{θ}^{μ}\overleftarrow{g}\overleftarrow{g}^{T} (reversibility), ^{∗}\overleftarrow{g} = g^{∗}, \overleftarrow{g}^{∗} = ^{∗}g, and the fact that U is selfadjoint.

Since Λ_{Δ,θ}(\overline{g}) = Λ_{Δ,θ}(g), to show (6.5) it suffices to show that E_{θ}^{μ} \overline{g}^{⌣} \overline{g}^{⌣}^{T} ≤ E_{θ}^{μ}ğğ^{T} with equality iff g ∈ R. Let u ∈ ℝ^{p}\0. Of course

E_{θ}^{μ} (u^{T} g)^{2} = E_{θ}^{μ} (u^{T} \overleftarrow{g})^{2} ,  (6.7)

hence

u^{T} (E_{θ}^{μ} \overline{g}^{⌣} \overline{g}^{⌣}^{T} ) u = E_{θ}^{μ} (u^{T} \overline{g}^{⌣})^{2} = \frac{1}{2}E_{θ}^{μ} (u^{T} ğ)^{2} + \frac{1}{2}E_{θ}^{μ} (u^{T} ğ) (u^{T} \overleftarrow{g}^{⌣}) ,

and here by Cauchy-Schwarz and (6.7),

|E_{θ}^{μ} (u^{T} ğ) (u^{T} \overleftarrow{g}^{⌣})| ≤ E_{θ}^{μ} (u^{T} ğ)^{2}  (6.8)

proving (6.5). Furthermore, equality holds in (6.8) iff u^{T} ğ = K(u)u^{T} \overleftarrow{g}^{⌣} Q_{Δ,θ}−a.s. for some constant K(u). But (6.7) forces K(u) = ±1 and by inspection it is

then seen that E_{θ}^{μ} (u^{T} \overline{g}^{⌣})^{2} = E_{θ}^{μ} (u^{T} ğ)^{2} iff K(u) = 1. Letting u run through

a countable dense subset of ℝ^{p}, it finally follows that equality in (6.5) holds iff

ğ =\overleftarrow{g}^{⌣} Q_{Δ,θ}−a.s. □

Proposition 6.1 shows that in reversible models, if g ∉ R, one can always improve an estimating function g by symmetrizing: using \overline{g} instead of g. Of course \overline{g} ∈ R.

We give two examples of estimating functions of the types dicussed earlier, where g ∈ R and symmetrizing therefore does not help.

Example 6.2. By Propositions 5.2, 6.1, for a reversible model MLE ∈ R. A direct proof is the following: since s = s_{Δ,θ} ∈ M, s^{∗} ≡ 0. Further, from s(x, y) = q̇_{Δ,θ}^{T}/q_{Δ,θ}(x, y) − μ̇_{θ}^{T} /μ_{θ}(x) and the symmetry of q_{Δ,θ} follows that

\overleftarrow{s}(x, y) = s(x, y) + \frac{μ̇_{θ}^{T}}{μ_{θ}} (x) − \frac{μ̇_{θ}^{T}}{μ_{θ}} (y),  (6.9)

and hence, since s^{∗} ≡ 0, that

\overleftarrow{s}^{∗} = ^{∗}s = (I − π_{Δ,θ}) \frac{μ̇_{θ}^{T}}{μ_{θ}} .

Thus, U_{Δ,θ} ^{∗}s = μ̇_{θ}^{T} /μ_{θ}^{T} , and using (6.9), \overleftarrow{s}^{⌣} (x, y) = s(x, y), i.e. s ∈ R, follows.

Example 6.3. If all g_{t,θ} are simple, also in non-reversible models (3.2) is automatic for \overleftarrow{g}_{t,θ}, and as we shall see, S ⊂ R. If e.g. g_{Δ,θ}(x, y) = f_{Δ,θ}(x), (each component of g has this form) we find (omitting subscripts _{Δ,θ}) that g^{∗} = f,

∗g = πf which quickly gives ğ(x, y) = Uf(y) − πUf(x) =\overleftarrow{g}^{⌣} (x, y). The intuitive content behind this fact that simple estimating functions are reversible is this: the estimating equations based on g and \overleftarrow{g} respectively are ∑_{0}^{n−1} f_{Δ,θ} (X_{mΔ}) = 0 and ∑_{1}^{n} f_{Δ,θ}(X_{mΔ}) = 0, and clearly they are asymptotically equivalent.

We saw that s_{Δ,θ} ∈ M, but other martingale estimating functions are typically not reversible: for g^{k} given by (3.8) to be reversible it is necessary and sufficient that either h is constant or that f and h are proportional. (The only term in ğ^{k}, which is not just a function of x or a function of y, is the product h(x)f(y). For g^{k} to be reversible this term must cancel against h(y)f(x), which is possible only if f or h is constant or if f and h are proportional. Since f constant results in g^{k} ≡ 0 this possibility can be ruled out, and it is then easily checked directly that with h constant or f ∝ h, g^{k} is indeed reversible).

For reversible models one can use the explicit, transition dependent estimating functions from the class T , introduced by Hansen and Scheinkman [7], see (3.7),

g_{Δ,θ}^{k}(x, y) = (A_{θ}f^{k}) (x)h^{k}(y) − f^{k}(x) (A_{θ}h^{k}) (y),  (6.10)

(with (3.12) the special case h^{k} ≡ 1) where f^{k}, h^{k} are allowed to depend on Δ, θ. That (3.2) holds is an easy consequence of reversibility and the fact that π_{Δ,θ} and A_{θ} are selfadjoint. Instead of (6.10) one could also use

g_{Δ,θ}^{k}(x, y) = (A_{θ}f^{k}) (x)h^{k}(y) − f^{k}(y) (A_{θ}h^{k}) (x).  (6.11)

Because of Proposition 6.1, if g^{k} is of the form (6.10) or (6.11) one should symmetrize. The two \overline{g}^{k}−functions obtained are identical, viz.

\frac{1}{2} ((A_{θ}f^{k}) (x)h^{k}(y) − f^{k}(x) (A_{θ}h^{k}) (y) + (A_{θ}f^{k}) (y)h^{k}(x) − f^{k}(y) (A_{θ}h^{k}) (x)) ,

which is therefore the preferred choice. Even better estimating functions are obtained by symmetrizing in (3.13).

In models that are not reversible, Hansen and Scheinkman [7] propose that (6.10) be replaced by

g_{Δ,θ}^{k}(x, y) = (\overleftarrow{A}_{θ}f^{k}) (x)h^{k}(y) − f^{k}(x) (A_{θ}h^{k}) (y),

with \overleftarrow{A}_{θ} the differential operator that defines the generator for \overleftarrow{X}. This gives an explicit, transiton dependent estimating function if μ_{θ} is known explicitly, cf. (6.2), but of course, for d ≥ 2 it may be difficult to determine μ_{θ}.

We conclude this section with a brief comment on the class U of useless estimating functions: For a reversible model of course

g_{Δ,θ}(x, y) = ς_{Δ,θ}(x, y) − ς_{Δ,θ}(y, x)

satisfies the unbiasedness condition (3.2) for any ς_{Δ,θ} ∈ L^{2} (Q_{Δ,θ}) . But (3.5) is not satisfied and for any g_{Δ,θ} ∈ U, Λ_{Δ,θ}(g) = 0 so there is no hope of asymptotics as in (4.5). It may however be noted that it is sometimes possible to obtain a variance reduction by considering the sum of a well behaved estimating function and a useless one. Also, by using functions from U, it may be shown that an arbitrarily large amount in efficiency may be gained by symmetrizing as proposed in Proposition 6.1: suppose that p = 1 and let g_{Δ,θ}^{∘} be well-behaved for a given Δ. Now destroy the good properties of g_{Δ,θ}^{∘} by adding a member from U thus,

g_{Δ,θ}(x, y) = g_{Δ,θ}^{∘}(x, y) + a (ς(x, y) − ς(y, x))

with |a| large. Then Δ_{Δ,θ}(g) does not depend on θ and it is an easy matter to arrange things so that E_{θ}^{μ}ğ_{Δ,θ}^{2} (X_{0}, X_{Δ}) = O(a^{2}) as |a| → ∞. But ḡ_{Δ,θ} = g_{Δ,θ}^{∘}, and so

\frac{var_{Δ,θ}(ḡ)}{var_{Δ,θ}(g)}

can be made arbitrarily small!

7. Small Δ−optimality

Proposition 5.3 showed how the best estimating function from a linear space may be found using projections in L^{2}(Q_{Δ,θ}). But as already noted, this rarely leads to explicit formulas, and the method is therefore difficult to apply in practice. Here we shall propose the concept of small Δ−optimality and discuss how it may be applied to give estimating functions with good properties.

For Δ large, the observations (X_{mΔ})_{0≤m≤n} are almost i.i.d. μ_{θ}, hence optimal inference as Δ → ∞ should be performed using the simple estimating function

g_{∞,θ}^{opt}(x, y) := \frac{μ̇_{θ}^{T}}{μ_{θ}} (x),

see (3.11), corresponding to finding the MLE if the X_{mΔ} were truly i.i.d. (Recall from Example 4.1 that only parameters appearing in μ_{θ} can be estimated using (3.11)). Therefore, if optimal estimating functions g_{0}^{opt} can be found for Δ → 0, by a suitable interpolation between g_{0,θ}^{opt} and g_{∞,θ}^{opt}, one may find flows (g_{t,θ}^{opt}) that perform well for all Δ. (And such a flow will give useful estimators even if Δ is large and μ_{θ} does not depend on all the parameters, only in that case estimators for some parameters will have large variances: the model does not allow for precise estimation of all the parameters).

With the optimality as Δ → ∞ in place, we shall focus on the problem of minimizing for a given flow, var_{Δ,θ}(g, θ̂) as Δ → 0. The conditions we arrive at (for multivariate, multiparameter diffusion models) are similar to the ones found by Kessler [10], (who studied models with p = 2 of the form dX_{t} = b_{θ_{1}} (X_{t}) dt + σ_{θ_{2}} (X_{t}) dB_{t}, θ = (θ_{1}, θ_{2}) ∈ Θ ⊂ ℝ^{2}, with n discrete observations Δ_{n} apart, and used moment estimating functions to obtain efficiency for Δ_{n} → 0 at a suitable rate), and Woerner [17] (who for d = 1 gave conditions for LAN in one-parameter models (p = 1) in the two cases where (i) the parameter enters in the drift b_{θ} only, (ii) σ_{θ} depends on θ and b_{θ} possibly also).

Our idea is to derive a power expansion in Δ of var_{Δ,θ}(g, θ̂), which, as we shall see, takes the form

var_{Δ,θ}(g, θ̂) = \frac{1}{Δ}v_{−1} + v_{0} + O(Δ),  (7.1)

and then call g ∈ G small Δ−optimal if the leading term(s) in this expansion, v_{−1} or v_{0} or a suitable combination of the two, is minimized. Depending on the structure of the model, we shall distinguish between three cases:

(i) Minimizing v_{−1}: σ_{θ} = σ does not depend on θ;

(ii) Minimizing v_{0} with v_{−1} ≡ 0: σ_{θ} depends on all parameters θ_{1}, …, θ_{p};

(iii) Minimizing a combination of v_{−1}, v_{0}: σ_{θ} depends on some, but not all the parameters.

As Δ → 0, we are approaching a limiting continuous time observation scheme, (X_{t})_{0≤t≤nΔ} . For that it is essentially true that if ℙ_{θ,t} is the distribution of (X_{s})_{0≤s≤t} under P_{θ}^{μ}, then ℙ_{θ′,t} ≪ ℙ_{θ,t} for all t, θ, θ′ in case (i), while in case (ii), ℙ_{θ′,t} ⊥ ℙ_{θ,t} if θ′ ≠ θ, and in case (iii) absolute continuity holds iff the parameters on which σ_{θ} depend are fixed. For n fixed, Δ → 0, we would in case (i) expect the information about θ to be approximately proportional to Δ, explaining the leading term Δ^{−1}v_{−1} in (7.1). In case (ii), due to the continuous time singularity between measures, we gain an order of magnitude in estimation precision: it is possible to obtain v_{0} ≡ 0.

In case (i) under local absolute continuity, the log-likelihood for observation on [0, t] given X_{0} has the form

∫_{0}^{t} b_{θ}^{T} (X_{s})C^{−1}(X_{s}) dX_{s} − \frac{1}{2} ∫_{0}^{t} b_{θ}^{T} (X_{s})C^{−1}(X_{s})b_{θ}(X_{s}) ds,

corresponding to the score function

s_{t,θ}^{c} = ∫_{0}^{t} ḃ_{θ}^{T} (X_{s})C^{−1}(X_{s}) dX_{s} − ∫_{0}^{t} ḃ_{θ}^{T} (X_{s})C^{−1}(X_{s})b_{θ}(X_{s}) ds
≈ ḃ_{θ}^{T} (X_{0})C^{−1}(X_{0}) (X_{t} − X_{0}) − tḃ_{θ}^{T} (X_{0})C^{−1}(X_{0})b_{θ}(X_{0}).

The latter approximation suggests the approximate estimating function (which does not in general satisfy (3.2)),

g̃_{t,θ}(x, y) = ḃ_{θ}^{T} (x)C^{−1}(x) (y − x) − tḃ_{θ}^{T} (x)C^{−1}(x)b_{θ}(x),
g̃_{0,θ}(x, y) = ḃ_{θ}^{T} (x)C^{−1}(x) (y − x) .

It should be noted that ∂_{y}g̃_{0,θ}(x, x) = ḃ_{θ}^{T} (x)C^{−1}(x), which is precisely the optimality criterion for martingale estimating functions presented in Theorem 7.5 (i) below!

To obtain (7.1) we shall simply use Itô-Taylor expansions of the random matrices ğ_{Δ,θ}ğ_{Δ,θ}^{T}, ġ_{Δ,θ} that appear in the expression for var_{Δ,θ}(g, θ̂), see (4.7), (4.6).

Suppose first that (s, x, y) → ϕ_{s}(x, y) is a continuous real-valued function of s ≥ 0 and (x, y) ∈ D^{2}, continuously differentiable in s, twice continuously differentiable in y. Let H_{θ} denote the differential operator given by

H_{θ}ϕ_{s}(x, y) = ∂_{s}ϕ_{s}(x, y) + A_{θ,y}ϕ_{s}(x, y),

where A_{θ,y}ϕ_{s}(x, y) denotes the function A_{θ}ϕ_{s}(x, ·) for s, x fixed evaluated at y. By Itô’s formula, under P_{θ}^{ν} for any ν,

\begin{aligned}ϕ_{t}(X_{0}, X_{t}) = ϕ_{0}(X_{0}, X_{0}) & + ∫_{0}^{t} ds H_{θ}ϕ_{s}(X_{0}, X_{s}) \\ & + ∫_{0}^{t} ∑_{i,j=1}^{d} ∂_{y_{i}}ϕ_{s}(X_{0}, X_{s})σ_{θ}^{ij}(X_{s}) dB_{s}^{j}, \end{aligned}  (7.2)

with ∂_{y_{i}}ϕ_{s}(x, z) denoting the function ∂_{y_{i}}ϕ_{s}(x, ·) for s, x fixed evaluated at z.

We shall denote by Φ_{θ} the class of functions ϕ satisfying the continuity and differentiability requirements above, together with

E_{θ}^{μ}ϕ_{s}^{2}(X_{0}, X_{s}) < ∞,
E_{θ}^{μ} (H_{θ}ϕ_{s}(X_{0}, X_{s}))^{2} < ∞,  (7.3)
E_{θ}^{μ}∂_{y}ϕ_{s}(X_{0}, X_{s})C_{θ}(X_{s})∂_{y}^{T} ϕ_{s}(X_{0}, X_{s}) < ∞,

for all s, (so if ϕ_{s}(x, y) = f(y) is a function of y only, this is simply the conditions for f ∈ D_{θ}, see p. 7). The operator H_{θ} acting on functions ϕ_{s}(x, z) in Φ_{θ} that do not depend on x, is the generator for the space-time process (t, X_{t})_{t≥0}.

If ϕ ∈ Φ_{θ}, the local martingale in (7.4) is a true P_{θ}^{μ}−martingale and

E_{θ}^{μ}ϕ_{t}(X_{0}, X_{t}) = E_{θ}^{μ}ϕ_{0}(X_{0}, X_{0}) + ∫_{0}^{t} E_{θ}^{μ}H_{θ}ϕ_{s} (X_{0}, X_{s}) ds.  (7.4)

Also, since P_{θ}^{μ} = ∫_{D} μ_{θ}(dx) P_{θ}^{x}, the three conditions in (7.3) are satisfied with E_{θ}^{μ} replaced by E_{θ}^{x}, at least for μ_{θ}−almost all x, and then (7.4) also holds with E_{θ}^{x} instead of E_{θ}^{μ}.

In some cases we shall need to expand the integrand on the right of (7.4). This will be done for ϕ ∈ Φ_{θ}, with the function (s, x, y) → H_{θ}ϕ_{s} (x, y) also in Φ_{θ}, which ensures that all local martingales are true martingales.

Consider now a given flow G = (g_{t,θ})_{t>0,θ∈Θ} of well behaved estimating functions. In the remainder of this section we shall assume that for all θ, all components g_{t,θ}^{k}(x, y) are one time continuously differentiable in t, two times continuously differentiable in y. We just write H_{θ}g_{t,θ} (x, y) ∈ ℝ^{p×1} for (H_{θ}g_{t}^{k} (x, y))_{k} . One more crucial assumption will now be made: with G given, a renormalization, replacing g_{t,θ} by K_{t,θ}g_{t,θ}, where K_{t,θ} ∈ ℝ^{p×p} is non-singular and does not depend on x, y, does not change the solutions of the estimating equation (3.4) for any t = Δ, and in particular, does not change var_{Δ,θ}(g, θ̂). We shall now assume that, possibly after renormalization with some K_{t,θ}, the flow (g_{t,θ}) extends continuously to include t = 0 with g_{t,θ}(x, y) for t ≥ 0 continuously differentiable in t and twice differentiable in y. In particular we then have a power expansion,

g_{t,θ}(x, y) = g_{0,θ}(x, y) + t∂_{s}g_{0,θ}(x, y) + o_{θ,x,y}(t)  (7.5)

and assume that g_{0,θ} is non-vanishing in the sense that P_{θ}^{μ} (g_{0,θ}^{k}(X_{0}, X_{t}) ≠ 0) > 0 for all t > 0, all θ, all components k.

Example 7.1. Suppose X is a one-dimensional Ornstein-Uhlenbeck process (so d = 1),

dX_{t} = −θX_{t} dt + σ dB_{t},  (7.6)

cf. Example 4.2. Then p_{t,θ,σ}(x, ·) ∼ N (e^{−θt}x, σ^{2} (1 − e^{−2θt}) /2θ) and we shall see how the MLE estimating function s_{t} = ṗ_{t}/p_{t} may be renormalized to give a non-trivial s_{0} in three different one-parameter models (p = 1), as well as in the full model (7.6) (p = 2, θ > 0, σ > 0).

(i) θ > 0, σ = 1. By computation,

lim_{t→0} s_{t,θ}(x, y) = −\frac{1}{2} (y^{2} − x^{2}) ,

so s_{0,θ}(x, y) = −\frac{1}{2} (y^{2} − x^{2}), not depending on θ.

(ii) θ = 1, σ > 0. With σ^{2} the parameter

lim_{t→0} ts_{t,σ^{2}}(x, y) = \frac{1}{2σ^{4}} (y − x)^{2} ,

so s_{0,σ^{2}}(x, y) = (y − x)^{2} /2σ^{4}.

(iii) θ > 0, σ = \sqrt{2θ}. Then

lim_{t→0} ts_{t,θ}(x, y) = \frac{1}{4θ^{2}} (y − x)^{2}

so s_{0,θ}(x, y) = (y − x)^{2} /4θ^{2}.

(iv) θ > 0, σ > 0. Then with (θ_{1}, θ_{2}) = (σ^{2}, θ) the labeling of the parameters,

lim_{t→0} ( \begin{matrix}t & 0 \\ 0 & 1 \end{matrix} ) s_{t,σ^{2},θ}(x, y) = ( \begin{matrix}\frac{1}{2σ^{4}} & (y − x) & 2 \\ −\frac{1}{2σ^{2}} (y^{2} − x^{2}) \end{matrix} )

with s_{0,σ^{2},θ}(x, y) given by the expression on the right.

Returning to the flow G satisfying (7.5), we also from now on assume that G ⊂ Φ, i.e. for every θ, each component g_{t,θ}^{k}(x, y) of g, viewed as a function of (t, x, y), belongs to Φ_{θ}.

The results about small Δ−optimality will be established first if all g_{t,θ} ∈ M, and then in the general case by using the associated martingale estimating functions (4.2). Naturally, studying the behaviour of g_{t,θ} for t → 0 implies that everything is expressed in terms of g_{0,θ}(x, y) and its partial derivatives evaluated along the diagonal y = x.

If g_{Δ,θ} ∈ M, for all x (with H_{θ} acting on g componentwise),

0 = E_{θ}^{x}g_{Δ,θ}(X_{0}, X_{Δ})
= g_{0,θ}(x, x) + E_{θ}^{x} ∫_{0}^{Δ} ds H_{θ}g_{s,θ}(x, X_{s})
= g_{0,θ}(x, x) + ΔH_{θ}g_{0,θ}(x, x) + o(Δ),  (7.7)

and thus

g_{0,θ}(x, x) = 0,  (7.8)
H_{θ}g_{s,θ}(x, x) = 0.  (7.9)

(In (7.7) the appearance of a remainder term o(Δ) is justified provided s → E_{θ}^{x}H_{θ}g_{s,θ} (x, X_{s}) is continuous. Conditions of this nature are deliberately ignored here and below!)

We begin by finding the expansions of Λ_{Δ,θ}(g) and E_{θ}^{μ} (g_{Δ,θ}g_{Δ,θ}^{T}) (X_{0}, X_{Δ}).

Notation. In the remainder of this section, if ψ is a function of x, y (or x only) and possibly t, θ,the symbol ψ, when appropriate, will also denote the random variable ψ(X_{0}, X_{0}) (ψ(X_{0}) respectively), conforming with the notation from p. 18 when taking Δ = 0.

Proposition 7.2. Suppose G ⊂ Φ with expansion (7.5) and all g_{Δ,θ} ∈ M. Suppose also that ġ_{Δ,θ} ∈ Φ_{θ}, g_{Δ,θ}g_{Δ,θ}^{T} ∈ Φ_{θ}. Then

E_{θ}^{μ}ġ_{Δ,θ}(X_{0}, X_{Δ}) = −ΔE_{θ}^{μ} [(∂_{y}g_{0,θ}) ḃ_{θ} + \frac{1}{2} (∂_{yy}^{2}g_{0,θ}) Ċ_{θ}] + o(Δ),  (7.10)
E_{θ}^{μ} (g_{Δ,θ}g_{Δ,θ}^{T}) (X_{0}, X_{Δ}) = ΔE_{θ}^{μ} (∂_{y}g_{0,θ}) C_{θ} (∂_{y}g_{0,θ})^{T} + o(Δ).  (7.11)

Proof. We have

E_{θ}^{μ}ġ_{Δ,θ}(X_{0}, X_{Δ}) = E_{θ}^{μ} [∂_{θ}g_{0,θ} + Δ (∂_{s}∂_{θ}g_{0,θ} + A_{θ,y} (∂_{θ}g_{0,θ}))] + o(Δ).

By (7.8) ∂_{θ}g_{0,θ} ≡ 0 while (7.9) implies

0 = ∂_{θ} (∂_{s}g_{0,θ} + A_{θ,y}g_{0,θ}) (x, x)
= (∂_{s}∂_{θ}g_{0,θ} + A_{θ,y} (∂_{θ}g_{0,θ})) (x, x) + ((∂_{y}g_{0,θ}) ḃ_{θ} + \frac{1}{2} (∂_{yy}^{2}g_{0,θ}) Ċ_{θ}) (x, x),

and (7.10) follows. Similarly, again using (7.8),

E_{θ}^{μ} (g_{Δ,θ}g_{Δ,θ}^{T}) (X_{0}, X_{Δ}) = ΔE_{θ}^{μ} (∂_{s} (g_{0,θ}g_{0,θ}^{T}) + A_{θ,y} (g_{0,θ}g_{0,θ}^{T})) + o(Δ).

But (7.8) also gives ∂_{s} (g_{0,θ}g_{0,θ}^{T}) (x, x) ≡ 0 and that

∂_{y_{i}y_{j}}^{2} (g_{0,θ}^{k}g_{0,θ}^{l}) (x, x) = (∂_{y_{i}}g_{0,θ}^{k}∂_{y_{j}}g_{0,θ}^{l} + ∂_{y_{j}}g_{0,θ}^{k}∂_{y_{i}}g_{0,θ}^{l}) (x, x),

whence

A_{θ,y} (g_{0,θ}g_{0,θ}^{T}) (x, x) = (∂_{y}g_{0,θ}) C_{θ} (∂_{y}g_{0,θ})^{T} (x, x)

and (7.11) follows. □

As a direct consequence we obtain

Corollary 7.3. If G is as in Proposition 7.2 and

Λ_{0,θ}(g) := E_{θ}^{μ} [(∂_{y}g_{0,θ}) ḃ_{θ} + \frac{1}{2} (∂_{yy}^{2}g_{0,θ}) Ċ_{θ}]  (7.12)

is non-singular, then

var_{Δ,θ}(g, θ̂) = \frac{1}{Δ}Λ_{0,θ}^{−1}(g) (E_{θ}^{μ} (∂_{y}g_{0,θ}) C_{θ} (∂_{y}g_{0,θ})^{T} ) (Λ_{0,θ}^{−1}(g))^{T} + o(\frac{1}{Δ}). (7.13)

The corollary will be used to obtain small Δ−optimality in case (i), p. 28 above. For case (ii), if ∂_{y}g_{0,θ}(x, x) vanishes, the main term in (7.13) is 0, and a further expansion in (7.11) is required.

Proposition 7.4. Suppose that g_{Δ,θ}g_{Δ,θ}^{T} ∈ Φ_{θ}, H_{θ} (g_{Δ,θ}g_{Δ,θ}^{T}) ∈ Φ_{θ} and that ∂_{y}g_{0,θ}(x, x) ≡ 0. Then

E_{θ}^{μ} (g_{Δ,θ}g_{Δ,θ}^{T}) (X_{0}, X_{Δ}) = \frac{1}{2}Δ^{2}E_{θ}^{μ} [(∂_{yy}^{2}g_{0,θ}) C_{θ}^{⊗2} (∂_{yy}^{2}g_{0,θ})^{T} ] + o(Δ^{2}). (7.14)

If it is only known that ∂_{y}g_{0,θ}^{k}(x, x) ≡ ∂_{y}g_{0,θ}^{l}(x, x) ≡ 0 for some k, l, the identity (7.14) still applies to the kl’th elements of the matrices involved.

Note. Recall that ∂_{yy}^{2}g_{0,θ} ∈ ℝ^{p×d^{2}} and that C_{θ}^{⊗2} ∈ ℝ^{d^{2}×d^{2}} with (ij), (i′j′)’th element C_{θ}^{ii′}C_{θ}^{jj′}.

Proof. If ρ_{t}(x, y) ∈ Φ_{θ} is a real-valued function of (t, x, y) with H_{θ}ρ ∈ Φ_{θ}, by Itô’s formula,

E_{θ}^{μ}ρ_{Δ}(X_{0}, X_{Δ}) = E_{θ}^{μ} [ρ_{0} + ΔH_{θ}ρ_{0} + \frac{1}{2}Δ^{2}H_{θ}^{2}ρ_{0}] (X_{0}, X_{0}) + o (Δ^{2}) . (7.15)

We are going to apply this expansion to ρ_{t}(x, y) = (g_{t,θ}^{k}g_{t,θ}^{l})(x, y) and first note that (omitting the subscript θ in the remainder of the proof) H acts on products as follows,

H (φ_{t}ψ_{t}) (x, y) = ((Hφ_{t}) ψ_{t} + φ_{t} (Hψ_{t})) (x, y) + ∑_{i,j=1}^{d} C^{ij}(y) (∂_{y_{i}}φ_{t}∂_{y_{j}}ψ) (x, y).

(7.16)

Now, with φ = g^{k}, ψ = g^{l}, because of (7.8) and the assumption ∂_{y}g_{0,θ}(x, x) ≡ 0, applying (7.15) to ρ_{t} = g_{t,θ}^{k}g_{t,θ}^{l} gives

E_{θ}^{μ} (g_{Δ}^{k}g_{Δ}^{l}) (X_{0}, X_{Δ}) = \frac{1}{2}Δ^{2}E_{θ}^{μ}H^{2} (g_{0}^{k}g_{0}^{l}) (X_{0}, X_{0}) + o(Δ^{2}).  (7.17)

But again using (7.16), (7.8), (7.9) and the assumption on ∂_{y}g_{0,θ},

H^{2} (g_{0}^{k}g_{0}^{l}) (x, x) = ∑_{i,j=1}^{d} H (C_{y}^{ij}∂_{y_{i}}g_{0}^{k}∂_{y_{j}}g_{0}^{l}) (x, x)  (7.18)

where the notation C_{y}^{ij} signifies that the function of (t, x, y) on which H is acting, involves the factor C^{ij}(y). By (7.16),

H (C_{y}^{ij}∂_{y_{i}}g_{t}^{k}∂_{y_{j}}g_{t}^{l}) = H(C_{y}^{ij}) ∂_{y_{i}}g_{t}^{k}∂_{y_{j}}g_{t}^{l}

+C_{y}^{ij} [H (∂_{y_{i}}g_{t}^{k}) ∂_{y_{j}}g_{t}^{l} + ∂_{y_{i}}g_{t}^{k} H (∂_{y_{j}}g_{t}^{l}) + ∑_{i′,j′} C_{y}^{i′j′}∂_{y_{i}y_{i′}}^{2}g_{t}^{k}∂_{y_{j}y_{j′}}^{2}g_{t}^{l}]

+ ∑_{i′,j′} C_{y}^{i′j′} (∂_{y_{i′}}C_{y}^{ij}) ∂_{y_{j′}} (∂_{y_{i}}g_{t}^{k}∂_{y_{j}}g_{t}^{l}) ,

and evaluating this for t = 0, y = x, using the assumption ∂_{y}g_{0}(x, x) = 0 all terms except those involving second derivatives ∂_{yy}^{2}g vanish, and we are left with

H (C_{y}^{ij}∂_{y_{i}}g_{0}^{k}∂_{y_{j}}g_{0}^{l}) (x, x) = ∑_{i,j,i′,j′} (∂_{y_{i}y_{i′}}^{2}g_{0}^{k}C_{y}^{ij}C_{y}^{i′j′}∂_{y_{j}y_{j′}}^{2}g_{0}^{l}) (x, x) ,

which inserted into (7.18) and (7.17) gives (7.14). □

We can now state and prove the main result on small Δ−optimality for martingale estimating functions. The three cases corresponds to the listing on page 28.

Theorem 7.5. Suppose that G = (g_{t,θ}) ⊂ Φ ∩ M is a well behaved flow of martingale estimating functions satisfying the expansion (7.5) with a non-vanishing g_{0,θ}, such that for every Δ > 0, θ ∈ Θ, ġ_{Δ,θ} ∈ Φ_{θ}, g_{Δ,θ}g_{Δ,θ}^{T} ∈ Φ_{θ}, and for (ii), (iii) also such that H_{θ}g_{Δ,θ}g_{Δ,θ}^{T} ∈ Φ_{θ}.

(i) If C = σσ^{T} does not depend on θ and if the p × p−matrices

E_{θ}^{μ} (∂_{y}g_{0,θ}) (X_{0}, X_{0})ḃ_{θ}(X_{0}),
E_{θ}^{μ}ḃ_{θ}^{T} (X_{0})C^{−1}(X_{0})ḃ_{θ}(X_{0})

are non-singular, then

var_{Δ,θ}(g, θ̂) = \frac{1}{Δ}v_{−1,θ}(g, θ̂) + o(\frac{1}{Δ}),

where

v_{−1,θ}(g, θ̂) ≥ (E_{θ}^{μ}ḃ_{θ}^{T} (X_{0})C^{−1}(X_{0})ḃ_{θ}(X_{0}))^{−1} .

Here equality holds, and g is small Δ−optimal, provided

∂_{y}g_{0,θ}(x, x) = K_{θ}ḃ_{θ}^{T} (x)C^{−1}(x)

for some constant, non-singular p × p−matrix K_{θ}.

(ii) If C_{θ} = σ_{θ}σ_{θ}^{T} depends on all parameters θ_{1}, …, θ_{p} and if the p × p−matrices

E_{θ}^{μ} (∂_{yy}^{2}g_{0,θ}) (X_{0}, X_{0})Ċ_{θ}(X_{0}),
E_{θ}^{μ}Ċ_{θ}^{T} (X_{0}) (C^{⊗2} (X_{0}))^{−1} Ċ_{θ}(X_{0})

are non-singular, then

var_{Δ,θ}(g, θ̂) = \frac{1}{Δ}v_{−1,θ}(g, θ̂) + v_{0,θ}(g, θ̂) + o(Δ),

with v_{−1,θ}(g, θ̂) = 0 if ∂_{y}g_{0}(x, x) = 0 for all x, and

v_{0,θ} (g, θ̂) ≥ 2 (E_{θ}^{μ}Ċ_{θ}^{T} (X_{0}) (C^{⊗2} (X_{0}))^{−1} Ċ_{θ}(X_{0}))^{−1} .

Here equality holds, and g is small Δ−optimal, if ∂_{y}g_{0,θ}(x, x) ≡ 0 and

∂_{yy}^{2}g_{0,θ}(x, x) = K_{θ}Ċ_{θ}^{T} (x) (C^{⊗2} (x))^{−1}

for some constant, non-singular p × p−matrix K_{θ}.

(iii) If for some p′, 1 ≤ p′ < p, C_{θ} = σ_{θ}σ_{θ}^{T} depends on θ_{1}, …, θ_{p′} but not on θ_{p′+1}, …, θ_{p}, and if the matrices

E_{θ}^{μ} [(∂_{y}g_{0,θ}) (X_{0}, X_{0}) ḃ_{θ}(X_{0}) + \frac{1}{2} (∂_{yy}^{2}g_{0,θ}) (X_{0}, X_{0})Ċ_{θ}(X_{0})] ,

E_{θ}^{μ}ḃ_{2,θ}^{T}(X_{0})C_{θ}^{−1}(X_{0})ḃ_{2,θ}(X_{0}),

E_{θ}^{μ} (Ċ_{1,θ}^{T}(X_{0}) (C_{θ}^{⊗2} (X_{0}))^{−1} Ċ_{1,θ}(X_{0}))

are non-singular, then

var_{Δ,θ}(g, θ̂) = \frac{1}{Δ}v_{−1,θ}(g, θ̂) + v_{0,θ}(g, θ̂) + o(Δ),

where

v_{−1,θ}(g, θ̂) ≥ ( \begin{matrix}0 & 0 \\ 0 & (E_{θ}^{μ}ḃ_{2,θ}^{T}(X_{0})C_{θ}^{−1}(X_{0})ḃ_{2,θ}(X_{0}))^{−1} \end{matrix} )  (7.19)

with equality if for some constant c_{θ} ≠ 0

∂_{y}g_{0,θ}(x, x) = c_{θ} ( \begin{matrix}0 \\ ḃ_{2,θ}^{T}(x)C_{θ}^{−1}(x) \end{matrix} ) .  (7.20)

In that case the upper left p′ × p′-block v_{11,0,θ} of v_{0,θ} satisfies

v_{11,0,θ} (g, θ̂) ≥ 2 (E_{θ}^{μ} (Ċ_{1,θ}^{T}(X_{0}) (C_{θ}^{⊗2} (X_{0}))^{−1} Ċ_{1,θ}(X_{0})))^{−1}  (7.21)

and here equality holds, and g is small Δ-optimal, if ∂_{y}g_{0,θ} satisfies (7.20) and

∂_{yy}^{2}g_{1,0,θ} = K̃_{θ}Ċ_{1,θ}^{T}(x) (C_{θ}^{⊗2} (x))^{−1}

for some constant, non-singular p′ × p′-matrix K̃_{θ}.

Notation. In (iii) block-matrix notation is used. Thus in (7.19) the interesting part on the right refers to the last p−p′ rows and columns of v_{−1,θ} with ḃ_{2,θ} ∈ ℝ^{d×(p−p′)} comprising the last p − p′ columns of ḃ_{θ}. Similarly, Ċ_{1,θ} ∈ ℝ^{d^{2}×p′} consists of the first p′ columns of Ċ_{θ} and g_{1,0,θ} contains the first p′ components of g_{0,θ}.

Proof.

(i) Since C does not depend on θ,

Λ_{0,θ}(g) = E_{θ}^{μ} (∂_{y}g_{0,θ}) ḃ_{θ},

cf. (7.12). Now just apply Corollary 7.3 and Lemma 5.1 with Z = ∂_{y}g_{0,θ}, Y = ḃ_{θ}^{T} and S = C^{−1}.

(ii) We have that if ∂_{y}g_{0,θ}(x, x) ≡ 0, then (see (7.12))

Λ_{0,θ}(g) = \frac{1}{2}E_{θ}^{μ} (∂_{yy}^{2}g_{0,θ}) Ċ_{θ}.

Now use Proposition 7.4 and Lemma 5.1 with Z = ∂_{yy}^{2}g_{0,θ}, Y = Ċ_{θ}^{T} and

S = (C_{θ}^{⊗2})^{−1}.

(iii) (7.19) follows from Corollary 7.3 using Lemma 7.6 below with Z = ∂_{y}g_{0,θ}, Y = ḃ_{θ}^{T} , U = Ċ_{θ}^{T} , V = \frac{1}{2}∂_{yy}^{2}g_{0,θ}, and S = C_{θ}^{−1}. Next, taking ∂_{y}g_{0,θ} as in (7.20), in block matrix notation Λ_{0,θ}(g) has the form

( \begin{matrix}0 & 0 \\ α_{1} & α_{2} \end{matrix} ) + ( \begin{matrix}β_{1} & 0 \\ β_{2} & 0 \end{matrix} )

and with an expansion

E_{θ}^{μ} (g_{Δ,θ}g_{Δ,θ}^{T}) (X_{0}, X_{Δ}) = Δγ + Δ^{2} ( \begin{matrix}δ_{11} & δ_{12} \\ δ_{21} & δ_{22} \end{matrix} ) + o(Δ^{2})

available, it is an easy matter to see that the upper left corner of v_{0,θ}(g, θ̂) equals β_{1}^{−1}δ_{11} (β_{1}^{T} )^{−1} , which must then be minimized. But β_{1} = \frac{1}{2}∂_{yy}^{2}g_{1,0,θ}Ċ_{θ}, while by Proposition 7.4, since ∂_{y}g_{1,0,θ} ≡ 0,

δ_{11} = \frac{1}{2}E_{θ}^{μ} [(∂_{yy}^{2}g_{1,0,θ}) C_{θ}^{⊗2} (∂_{yy}^{2}g_{1,0,θ})^{T} ] ,

and the proof is completed using Lemma 5.1 with Z = ∂_{yy}^{2}g_{1,0,θ}, Y = Ċ_{θ} and

S = (C_{θ}^{⊗2})^{−1} . □

Remarks. Part (iii) of Theorem 7.5 is formulated as a mixture of (i) and (ii). Note that we have not minimized v_{0,θ}(g), which appears hopeless, only the upper left block. Suppose now that g is small Δ−optimal and that

W = ( \begin{matrix}W_{1} \\ W_{2} \end{matrix} ) ∼ N_{p} (0, \frac{1}{Δ}v_{−1,θ}^{opt}(g) + v_{0,θ}^{opt}(g)) ,

with v_{−1,θ}^{opt}(g) the lower bound in (7.19) and the upper left block of v_{0,θ}^{opt}(g) the lower bound in (7.21), is a Gaussian random vector, which according to the theorem, for Δ small has a distribution close to the asymptotic distribution of \sqrt{n} (θ̂ − θ) when the esimating function g is used on observations Δ apart. Thus the variance of W has the form

\frac{1}{Δ} ( \begin{matrix}0 & 0 \\ 0 & E_{θ}^{μ}ḃ_{2}^{T} C_{θ}^{−1}ḃ_{2} \end{matrix} ) + ( \begin{matrix}2E_{θ}^{μ}Ċ_{1,θ}^{T} (C_{θ}^{⊗2})^{−1} Ċ_{1,θ} & δ_{12} \\ δ_{21} & δ_{22} \end{matrix} ) ,

where the δ-matrices are unknown and depend on unspecified characteristics of g. However, regardless of the values of the δ’s,

( \begin{matrix}W_{1} \\ \sqrt{Δ}W_{2} \end{matrix} ) →^{D_{θ}} ( W̃_{1} ; W̃_{2} )

as Δ → 0, where W̃_{1}, W̃_{2} are independent mean zero Gaussian with covariances 2E_{θ}^{μ}Ċ_{1,θ}^{T} (C_{θ}^{⊗2})^{−1} Ċ_{1,θ} and E_{θ}^{μ}ḃ_{2}^{T} C_{θ}^{−1}ḃ_{2}. Thus the result is sharp enough to yield

the joint distribution of (θ̂_{1}, …, θ̂_{p′}) and (θ̂_{p′+1}, …, θ̂_{p}) when the two vectors are properly scaled.

Theorem 7.5 (iii) should be compared to Kessler’s [10] Theorem 1 and may perhaps be viewed as a generalization to a multidimensional, multiparameter setting of his result.

It was assumed in part (iii) that C_{θ} depend on (θ_{1}, …, θ_{p′}) only. For applications it may be necessary to reparametrize before this is fulfilled. Of course also any small Δ−optimal g as described in (iii) may be relaced by K_{θ}g with K_{θ} a constant, non-singular p × p−matrix.

If d = 1, p = 1 the lower bounds in parts (i), (ii) are exactly the variances for the limiting Gaussian experiment found by Woerner [17] in her discussion of local asymptotic normality (LAN).

In the proof of part (iii), Theorem 7.5, the following result was used:

Lemma 7.6. Suppose that Y, Z, U, V, S are matrix-valued random variables of dimensions a × b, a × b, a × b′, a × b′, b × b respectively with S symmetric and strictly positive definite with probability 1. Suppose further that U = \binom{U_{1}}{0}, where U_{1} ∈ ℝ^{a′×b′} comprises the first a′ rows of U, and where 1 ≤ a′ < a. Assuming that all entries in the matrices Y Z^{T} , UV^{T} , ZS^{−1}Z^{T} , Y_{2}SY_{2}^{T} are integrable, writing Y_{2} for the last a − a′ rows of Y, if E (Y Z^{T} + UV^{T} ) , EU_{1}U_{1}^{T} and EY_{2}SY_{2}^{T} are non-singular, it holds that

(E (ZY^{T} + V U^{T} ))^{−1} EZS^{−1}Z^{T} (E (Y Z^{T} + UV^{T} ))^{−1} ≥ ( \begin{matrix}0 & 0 \\ 0 & (EY_{2}SY_{2}^{T} )^{−1} \end{matrix} )

(7.22)

with equality if EU_{1}V_{1}^{T} is non-singular and Z = κ\binom{0}{Y_{2}S} for some constant κ ≠ 0.

Proof. The left hand side M_{0} of (7.22) equals lim_{h→0} M_{h}, where M_{h} is given by the

expression for M_{0} when replacing EZS^{−1}Z^{T} by E (ZS^{−1}Z^{T} + hV V^{T} ), h > 0. But by Lemma 5.1,

M_{h} ≥ [E ( Y U ) ( \begin{matrix}S & 0 \\ 0 & \frac{1}{h}I \end{matrix} ) ( \begin{matrix}Y^{T} \\ U^{T} \end{matrix} )]^{−1} ,  (7.23)

(the assumptions EU_{1}U_{1}^{T} > 0 and EY_{2}SY_{2}^{T} > 0 ensuring that the inverse exists for h small enough), and by inspection the determinant of the matrix

N_{h} = (E (Y SY^{T} + \frac{1}{h}UU^{T} ))^{−1}

on the right of (7.23) is of order O(h^{−a′}), while the subdeterminant obtained by deleting the k’th row and l’th column, because the last a − a′ rows of U vanish, is of the same order only if k, l > a′. Thus we may write

N_{h} = ( \begin{matrix}0 & 0 \\ 0 & Q_{0} \end{matrix} ) + hR + o(h),

and it is then an easy matter to verify that Q_{0} = (EY_{2}SY_{2}^{T} )^{−1} . (7.22) now follows taking limits in (7.23).

Suppose now that Z = κ\binom{0}{Y_{2}S}. Then E (ZY^{T} + V U^{T} ) is non-singular because EU_{1}V_{1}^{T} is, and that equality holds in (7.22) amounts to showing

EZS^{−1}Z^{T} = (EZY^{T} ) ( \begin{matrix}0 & 0 \\ 0 & Q_{0} \end{matrix} ) (EY Z^{T} ) ,  (7.24)

for Z = \binom{0}{Y_{2}S}, where we have used that

( \begin{matrix}0 & 0 \\ 0 & Q_{0} \end{matrix} ) U = 0.

(7.24) is checked directly. □

Note. The assumption EU_{1}U_{1}^{T} > 0 may be replaced by EU_{1}S̄U_{1}^{T} > 0 for any random b′ × b′−matrix S̄ > 0 almost surely. This is actually used in Theorem 7.5 (iii).

Let now G ⊂ Φ be a well behaved flow of estimating functions, not necessarily in M. Small Δ−optimality is then discussed applying Theorem 7.5 to the flow of associated martingale estimating functions ğ_{t,θ}, cf. (4.2).

We assume that after a suitable renormalization G allows the expansion (7.5). To find the corresponding expansion of \binom{}{ğ_{t,θ}} we use the following result

Lemma 7.7. Consider a component g_{t,θ}^{k} of g_{t,θ}. Then as t → 0,

U_{t,θ}g_{t,θ}^{k∗}(x) = \frac{1}{t}a_{−1}^{k}(x) + a_{0}^{k}(x) + o(1),  (7.25)

where

(i) if g_{0,θ}^{k}(x, x) is not identically 0, a_{−1} ∈ D_{0,θ} is the unique solution to

A_{θ}a_{−1}^{k}(x) = −g_{0,θ}^{k}(x, x);  (7.26)

(ii) if g_{0,θ}^{k}(x, x) ≡ 0, then a_{−1}^{k} ≡ 0 and a_{0} ∈ D_{0,θ} is the unique solution to

A_{θ}a_{0}^{k}(x) = − (∂_{s}g_{0,θ}^{k} + A_{θ,y}g_{0,θ}^{k}) (x, x).  (7.27)

Proof. Applying I − π_{t,θ} to both sides of (7.25) gives

g_{t,θ}^{k∗} = \frac{1}{t} (I − π_{t,θ}) a_{−1}^{k} + (I − π_{t,θ}) a_{0}^{k} + o(1).

But, see (7.5),

g_{t,θ}^{k∗}(x) = g_{0}^{k}(x, x) + t (∂_{s}g_{0,θ}^{k} + A_{θ,y}g_{0,θ}^{k}) (x, x) + o(t)

and comparing this with the expansion

π_{t,θ}h = h + tA_{θ}h + o(t)

applied to h = a_{−1}^{k}, a_{0}^{k}, the result follows, recalling that the uniqueness of the solutions to the equations determining a_{−1}^{k}, a_{0}^{k} follows from Proposition 2.1. □

With this result in mind, we define the renormalized martingale flow \binom{}{ǧ_{t,θ}} associated with G as follows (with g_{0,θ}^{k}(x, x) ≠ 0 meaning that g_{0,θ}^{k}(x, x) is not identically 0),

g_{t,θ}^{k}(x, y) + U_{t,θ}g_{t,θ}^{k∗}(y) − U_{t,θ}g_{t,θ}^{k∗}(x) = \{ \begin{matrix}t^{−1}ǧ_{t,θ}^{k}(x, y) & if g_{0,θ}^{k}(x, x) ≠ 0 \\ ǧ_{t,θ}^{k}(x, y) & if g_{0,θ}^{k}(x, x) ≡ 0 \end{matrix}

so that

ǧ_{0,θ}^{k}(x, y) = \{ \begin{matrix}a_{−1}^{k}(y) − a_{−1}^{k}(x) & if g_{0,θ}^{k}(x, x) ≠ 0 \\ g_{0}^{k}(x, y) + a_{0}^{k}(y) − a_{0}^{k}(x) & if g_{0,θ}^{k}(x, x) ≡ 0, \end{matrix}  (7.28)

with a_{−1}^{k}, a_{0}^{k} determined by (7.26), (7.27) respectively. Applying Theorem 7.5 with ǧ instead of g now gives the following general result, where e.g. in part (ii) the condition g_{0,θ}(x, x) ≡ 0 is argued as follows: suppose that for some k, g_{0,θ}^{k}(x, x) is not identically 0; by Theorem 7.5 (ii), we then need ∂_{y}ğ_{0,θ}^{k}(x, x) ≡ 0 for all k, and by (7.28) this forces a_{−1}^{k} ≡ 0 and therefore also g_{0,θ}^{k} ≡ 0.

Theorem 7.8. Suppose that G = (g_{t,θ}) ⊂ Φ is a well behaved flow of estimatimg functions satisfying the expansion (7.5) with a non-vanishing g_{0,θ} and let for every k, a_{−1}^{k}, a_{0}^{k} denote the solutions to (7.26), (7.27) respectively. Further assume that

for every Δ > 0, θ ∈ Θ, ǧ^{·}_{Δ,θ}∈ Φ_{θ}, ǧ_{Δ,θ}ǧ_{Δ,θ}^{T} ∈ Φ_{θ}, and for (ii), (iii) also that H_{θ}ǧ_{Δ,θ}ǧ_{Δ,θ}^{T} ∈ Φ_{θ}.

(i) If C = σσ^{T} does not depend on θ and if the p × p−matrices

E_{θ}^{μ} (∂_{y}ǧ_{0,θ}) (X_{0}, X_{0})ḃ_{θ}(X_{0}),
E_{θ}^{μ}ḃ_{θ}^{T} (X_{0})C^{−1}(X_{0})ḃ_{θ}(X_{0})

are non-singular, then var_{Δ,θ}(g, θ̂) has the same expansion with the same lower bound as in Theorem 7.5 (i), and g is small Δ−optimal, provided

∂_{y}ǧ_{0,θ}(x, x) = K_{θ}ḃ_{θ}^{T} (x)C^{−1}(x)

for some constant, non-singular p × p−matrix K_{θ}. Here, for every k

∂_{y}ǧ_{0,θ}^{k}(x, x) = \{ \begin{matrix}∂_{x}a_{−1}^{k}(x) & if g_{0,θ}^{k}(x, x) ≠ 0 \\ ∂_{y}g_{0}^{k}(x, x) + ∂_{x}a_{0}^{k}(x) & if g_{0,θ}^{k}(x, x) ≡ 0. \end{matrix}  (7.29)

(ii) If C_{θ} = σ_{θ}σ_{θ}^{T} depends on all parameters θ_{1}, …, θ_{p} and if the p × p−matrices

E_{θ}^{μ} (∂_{yy}^{2}ǧ_{0,θ}) (X_{0}, X_{0})Ċ_{θ}(X_{0}),
E_{θ}^{μ}Ċ_{θ}^{T} (X_{0}) (C^{⊗2} (X_{0}))^{−1} Ċ_{θ}(X_{0})

are non-singular, then var_{Δ,θ}(g, θ̂) has the same expansion with the same lower bound as in Theorem 7.5 (ii), and g is small Δ−optimal, provided g_{0,θ}(x, x) ≡ 0, ∂_{y}g_{0,θ}(x, x) + ∂_{x}a_{0}(x) ≡ 0 and

∂_{yy}^{2}ǧ_{0,θ}(x, x) = K_{θ}Ċ_{θ}^{T} (x) (C^{⊗2} (x))^{−1}

for some constant, non-singular p × p−matrix K_{θ}. Here, for every k

∂_{yy}^{2}ǧ_{0,θ}^{k}(x, x) = ∂_{yy}^{2}g_{0}^{k}(x, x) + ∂_{xx}^{2}a_{0}^{k}(x).

(iii) If for some p′, 1 ≤ p′ < p, C_{θ} = σ_{θ}σ_{θ}^{T} depends on θ_{1}, …, θ_{p′} but not on θ_{p′+1}, …, θ_{p}, and if the matrices

E_{θ}^{μ} [(∂_{y}ǧ_{0,θ}) (X_{0}, X_{0})ḃ_{θ}(X_{0}) + \frac{1}{2} (∂_{yy}^{2}ǧ_{0,θ}) (X_{0}, X_{0})Ċ_{θ}(X_{0})] ,
E_{θ}^{μ}ḃ_{2,θ}^{T}(X_{0})C_{θ}^{−1}(X_{0})ḃ_{2,θ}(X_{0}),
E_{θ}^{μ} (Ċ_{1,θ}^{T}(X_{0}) (C_{θ}^{⊗2} (X_{0}))^{−1} Ċ_{1,θ}(X_{0}))

are non-singular, then var_{Δ,θ}(g, θ̂) has the same expansion with the same lower bounds as in Theorem 7.5 (iii), and g is small Δ−optimal, provided g_{1,0,θ}(x, x) ≡ 0 and

∂_{y}ǧ_{0,θ}(x, x) = c_{θ} ( \begin{matrix}0 \\ ḃ_{2,θ}^{T}(x)C_{θ}^{−1}(x) \end{matrix} ) ,
∂_{yy}^{2}ǧ_{1,0,θ}(x, x) = K̃_{θ}Ċ_{1,θ}^{T}(x) (C_{θ}^{⊗2} (x))^{−1}

for some constant c_{θ} ≠ 0 and some constant, non-singular K̃_{θ} ∈ ℝ^{p′×p′}, ǧ_{1,0,θ} comprising the first p′ components of ǧ_{0,θ} and ḃ_{2,θ} comprising the last p − p′ columns of ḃ_{θ}. Here, for k > p′

∂_{y}ǧ_{0,θ}^{k}(x, x) = \{ \begin{matrix}∂_{x}a_{−1}^{k}(x) & if g_{0,θ}^{k}(x, x) ≠ 0 \\ ∂_{y}g_{0}^{k}(x, x) + ∂_{x}a_{0}^{k}(x) & if g_{0,θ}^{k}(x, x) ≡ 0. \end{matrix}

and for k ≤ p′

∂_{yy}^{2}ǧ_{0,θ}^{k}(x, x) = ∂_{yy}^{2}g_{0}^{k}(x, x) + ∂_{xx}^{2}a_{0}^{k}(x).

As an illustration of Theorem 7.8 we shall discuss when simple estimating functions (p. 10) are small Δ-optimal.

Suppose that for some k, g_{t,θ}^{k}(x, y) = f_{t,θ}(x) has been properly normalized (see (7.5)) with g_{0,θ}^{k}(x, y) = f_{0,θ}(x) ≠ 0. Then g_{0,θ}^{k}(x, x) ≠ 0 and ǧ_{0,θ}^{k}(x, y) = a_{−1}^{k}(y) − a_{−1}^{k}(x), where A_{θ}a_{−1}^{k} = −f_{0,θ}. Thus Theorem 7.8 (ii) does not apply and we see that simple estimating functions cannot be small Δ−optimal if C_{θ} depends on all parameters. If however C = C_{θ} does not depend on θ, by Theorem 7.8 (i), f_{t,θ} satisfies the optimality criterion if

∂_{y}a_{−1}^{k} = k’th row of K_{θ}ḃ_{θ}^{T} C_{θ}^{−1}.  (7.30)

For d = 1, this is easy: just integrate the k’th element of K_{θ}ḃ_{θ}^{T} (x)C_{θ}^{−1}(x). But if d > 1, a miracle is required since (7.30) implies that

∂_{y_{j}} (K_{θ}ḃ_{θ}^{T} C_{θ}^{−1})_{ki} = ∂_{y_{i}y_{j}}^{2}a_{−1}^{k}

must be symmetric in i, j. Thus the message is that simple estimating functions can be small Δ-optimal only if d = 1 and C_{θ} does not depend on all the parameters – but perhaps the truly surprising fact is that they can be optimal at all, since for Δ small one might consider transition dependence essential for effective estimation.

If d = 1 and the simple estimating function is of the form (3.12), i.e. f_{t,θ}^{k}(x) = A_{θ}h^{k}(x) (1 ≤ k ≤ p) , not depending on t, we find from (7.26) that g_{0,θ}^{k}(x, x) = A_{θ}h^{k}(x) = −A_{θ}a_{−1}^{k}(x) so a_{−1}^{k} = −h^{k}, and thus small Δ−optimality is achieved using the flow

g_{t,θ}(x, y) = A_{θ}h(x), ∂_{x}h(x) = \frac{1}{σ^{2}(x)}ḃ_{θ}^{T} (x),  (7.31)

where h = (h^{k})_{k} and C(x) = σ^{2}(x) is scalar-valued.

The simple estimating function (7.31) was derived by Helle Sørensen [14] via a quite different approach.

Example 7.9. As illustration of how small Δ−optimality may be achieved, using different types of estimating functions, we consider the Ornstein-Uhlenbeck models from Example 7.1. Thus d = 1.

(i) We have p = 1 and can use (7.31). The result is that g_{t,θ}(x, y) = A_{θ}h(x) = −θxh′(x) + \frac{1}{2}h′′(x) is small Δ−optimal if h′(x) = −x, i.e. θx^{2} − \frac{1}{2} is small Δ−optimal. This is the simple estimating function found by Kessler [9], Section 5.1, which he shows is optimal in S for any Δ (cf. Proposition 5.7) with a very high efficiency.

(ii) Here we cannot use simple estimating functions to obtain small Δ−optimality, and instead consider martingale estimating functions of the form (3.9), viz.

g_{t,θ}(x, y) = h_{t}^{(1)}(x) (y − e^{−t}x) + h_{t}^{(2)}(x) (y^{2} − e^{−2t}x^{2} − \frac{σ^{2}}{2} (1 − e^{−2t})) .

Assuming that h_{t}^{(q)} → h_{0}^{(q)} for q = 1, 2 as t → 0, we find

g_{0,θ}(x, y) = h_{0}^{(1)}(x)(y − x) + h_{0}^{(2)}(x) (y^{2} − x^{2}) .

According to Theorem 7.5 (ii) we need ∂_{y}g_{0,θ}(x, x) ≡ 0, i.e.

h_{0}^{(1)}(x) + 2xh_{0}^{(2)}(x) = 0

for all x, and also that ∂_{yy}^{2}g_{0,θ}(x, x) is constant, e.g. 2h_{0}^{(2)} ≡ 1. Thus

g_{t,θ}(x, y) = −x (y − e^{−t}x) + \frac{1}{2} (y^{2} − e^{−2t}x^{2} − \frac{σ^{2}}{2} (1 − e^{−2t}))

is small Δ−optimal. Also small Δ−optimal is

g_{t,θ}(x, y) = −e^{−t}x (y − e^{−t}x) + \frac{1}{2} (y^{2} − e^{−2t}x^{2} − \frac{σ^{2}}{2} (1 − e^{−2t}))

= \frac{1}{2} (y − e^{−t}x)^{2} − \frac{σ^{2}}{4} (1 − e^{−2t})

which, as is easily checked, for all t = Δ yields the maximum-likelihood estimator! Of course the two estimating functions behave quite differently for large t = Δ. The example shows that there may be many g that are small Δ−optimal, but how to decide which one to use is for now an open problem.

(iii) Here also p = 1, and for studying small Δ−optimality, this model is equvalent to (ii) with 2θ = σ^{2}. For Δ large, inference in the two models differs radically, with no problems estimating σ^{2} in (ii), while since μ_{θ} in (iii) does not depend on θ, estimation of θ for large Δ is virtually impossible.

(iv) Now p = 2, p′ = 1 and g_{t,σ^{2},θ} is small Δ−optimal if e.g.

∂_{y}ǧ_{0,σ^{2},θ}(x, x) = κ_{σ^{2},θ}\binom{0}{−x/σ^{2}}, ∂_{yy}^{2}ǧ_{1,0,σ^{2},θ}(x, x) = 1.

One finds therefore that

g_{t,θ}(x, y) = ( \begin{matrix}\frac{1}{2} (y − e^{−θt}x)^{2} − \frac{σ^{2}}{4} (1 − e^{−2θt}) \\ θx^{2} − \frac{σ^{2}}{2} \end{matrix} )

is small Δ−optimal.

Our final result is obvious from Theorem 7.8 and Proposition 6.1. It is however quite interesting to verify directly that \overleftarrow{g}_{t,θ} satisfies the optimality criteria if g_{t,θ} does!

Proposition 7.10. In a reversible model, if (g_{t,θ}) is small Δ−optimal, so is the reversed flow (\overleftarrow{g}_{t,θ}) (provided it satisfies the relevant conditions listed in the opening paragraph of Theorem 7.8). □

Even if a flow G is small Δ−optimal, there is no guarantee that it will perform decently for Δ large. As mentioned earlier in this section, one may try to overcome this difficulty by combining small Δ−optimal flows with the optimal simple estimating function as Δ → ∞, e.g. through convex combinations of the form

e^{−λΔ}g_{Δ,θ} + (1 − e^{−λΔ}) \frac{μ̇_{θ}^{T}}{μ_{θ}}

for some λ > 0 (perhaps with a separate λ for each component). Here of course g must be chosen so that the convex combination is small Δ−optimal.

More generally, if for a given Δ > 0, g_{(q),θ} are well-behaved estimating functions, 1 ≤ q ≤ r, one may look for constant p × p−matrices A_{(q)} such that the asymptotic covariance for the estimator determined by the estimating function

∑_{q=1}^{r} A_{(q)}g_{(q),θ}

is minimized. The result (as is seen using e.g. Lemma 5.1) is that, in block matrix notation, the p × pr−matrix (A_{(1)} ⋯ A_{(r)}) should equal

K_{θ} ( Λ_{Δ,θ}^{T}(g_{(1),θ}) ⋯ Λ_{Δ,θ}^{T}(g_{(r),θ}) ) (E_{θ}^{μ} ( \begin{matrix}ğ_{(1),θ}ğ_{(1),θ}^{T} & ⋯ & ğ_{(1),θ}ğ_{(r),θ}^{T} \\ ⋮ & & ⋮ \\ ğ_{(r),θ}ğ_{(1),θ}^{T} & ⋯ & ğ_{(r),θ}ğ_{(r),θ}^{T} \end{matrix} ))^{−1}

for some constant, non-singular K_{θ} ∈ ℝ^{p×p}. This may appear useless for practical purposes, but through expansions for small Δ might perhaps be used to pinpoint better the best among several small Δ−optimal flows.

Acknowledgement. I would like to thank Michael Sørensen and Mathieu Kessler, not only for providing the inspiration, but also for many fruitful discussions.

References

[1] Aït-Sahalia, Ya. (1997, revised 1998). Maximum likelihood estimation of discretely sampled diffusions: a closed-form approach. Working paper 467, Graduate School of Business, University of Chicago.

[2] Baddeley, A.J. (1995). Time-invariance estimating equations. Research report 22, Department of Mathematics, University of Western Australia.

[3] Bibby, B.M. and Sørensen, M. (1995). Martingale estimating functions for discretely observed diffusion processes. Bernoulli 1, 17-39.

[4] Dacunha-Castelle, D. and Florens-Zmirou, D. (1986). Estimation of the coefficients of a diffusion from discrete observations. Stochastics 19, 263-284.

[5] Florens-Zmirou, D. (1987). Approximate discrete-time schemes for statistics of diffusion processes. Statistics 20, 547-557.

[6] Godambe, V.P. and Heyde, C.C. (1987). Quasi-likelihood and optimal estimation. Internat. Statist. Rev. 55, 231-244.

[7] Hansen, L.P. and Scheinkman, J.A. (1995). Back to the future: generating moment implications for continuous-time Markov processes. Econometrica 63, 767-804.

[8] Kessler, M. (1995). Martingale estimating functions for a Markov chain. Preprint, Laboratoire de Probabilités, Université de Paris VI.

[9] Kessler, M. (1996). Simple and explicit estimating functions for a discretely observed diffusion process. Research report 336, Department of Theoretical Statistics, University of Aarhus. (To appear in Scand. J. Statist).

[10] Kessler, M. (1997). Estimation of an ergodic diffusion from discrete observations. Scand. J. Statist. 24, 211-229.

[11] Kessler, M. and Sørensen, M. (1995). Estimating equations based on eigenfuctions for a discretely observed diffusion process. Research report 332, Department of Theoretical Statistics, University of Aarhus. (To appear in Bernoulli).

[12] Pedersen, A.R. (1994). Quasi-likelihood inference for discretely observed diffusion processes. Research report 295, Department of Theoretical Statistics, University of Aarhus.

[13] Pedersen, A.R. (1995). A new approach to maximum likelihood estimation for stocahstic differential equations based on discrete observations. Scand. J. Statist. 22, 55-71.

[14] Sørensen, H. (1998). Approximation of the score function for diffusion processes. Preprint 8, Department of Theoretical Statistics, University of Copenhagen.

[15] Sørensen, M. (1997). Estimating functions for discretely observed diffusions: a review. In Basawa, I.V., Godambe, V.P. and Taylor, R.L. (eds.): Selected Proceedings of the Symposium on Estimating Functions. IMS Lecture Notes - Monograph Series, Vol. 32, pp. 305-325.

[16] Sørensen, M. (1998). On asymptotics of estimating functions. Preprint 6, Department of Theoretical Statistics, University of Copenhagen.

[17] Woerner J.H.C. (1998). LAN for discretely observed diffusion processes in the ergodic case and applications to martingale estimating functions. Preprint Nr. 1, Albert-Ludwigs-Universität Freiburg, Mathematische Fakultät.

Martin Jacobsen

Department of Theoretical Statistics University of Copenhagen 5 Universitetsparken 2100 Copenhagen Ø Denmark

                       martin@math.ku.dk