On Oppenheim-type conjecture for systems of quadratic forms^{∗}

Alexander Gorodnik

Abstract

Let Q_{i}, i = 1, …, t, be real nondegenerate indefinite quadratic forms in d variables. We investigate under what conditions the closure of the set {(Q_{1}(x̄), …, Q_{t}(x̄)): x̄ ∈ ℤ^{d} − {¯0}} contains (0, …, 0). As a corollary, we deduce several results on the magnitude of the set Δ of g ∈ GL(d, ℝ) such that the closure of the set {(Q_{1}(gx̄), …, Q_{t}(gx̄)) : x̄ ∈ ℤ^{d} − {¯0}} contains (0, …, 0). Special cases are described when depending on the mutual position of the hypersurfaces {Q_{i} = 0}, i = 1, …, t, the set Δ has full Haar measure or measure zero and Hausdorff dimension d^{2} − \frac{d−2}{2}.

1 Introduction

It was conjectured by Oppenheim that if

Q(x̄) = ∑_{i,j} a_{ij}x_{i}x_{j}, x̄ = (x_{1}, …, x_{d}),

is a real nondegenerate indefinite quadratic form in d variables, d ≥ 5, which is not proportional to a rational form, then

∀ε > 0 ∃x̄ ∈ ℤ^{d} − {¯0}: |Q(x̄)| < ε.  (1)

This conjecture was proved by Margulis [Ma89]. In fact, he proved that if Q is a real nondegenerate indefinite quadratic form in d variables, d ≥ 3, which is not proportional to a rational form, then

∀ε > 0 ∃x̄ ∈ ℤ^{d} − {¯0}: 0 < |Q(x̄)| < ε.

See [Ma97] for an up-to-date survey. It is also known that (1) fails for some indefinite irrational forms in 2 variables. For example, (1) fails for Q = x_{1}^{2} − α^{2}x_{2}^{2} with a badly approximable number α (e.g. α = 1 + \sqrt{2}). Recall that a number α is called badly approximable if there exists c > 0 such that for all integers p and q > 0,

∗This article is a part of author’s PhD thesis at Ohio State University done under supervision of Prof. Bergelson.

|qα − p| ≥ \frac{c}{q}.

In this paper we study similar questions for systems (Q_{j}: j = 1, …, t) of quadratic forms. Namely, we would like to know when the following statement is true:

∀ε > 0 ∃x̄ ∈ ℤ^{d} − {¯0}: max_{j=1,...,t} |Q_{j}(x̄)| < ε.  (2)

We expect that an analog of the Oppenheim conjecture holds for systems of quadratic forms (see Conjecture 13 below). However, the proof of this conjecture seems to be beyond the reach of available methods.

One can give a convenient characterization of property (2) for systems quadratic forms in two variables: (2) holds for nondegenerate indefinite quadratic forms

Q_{j}(x_{1}, x_{2}) = (a_{j}x_{1} + b_{j}x_{2})(c_{j}x_{1} + d_{j}x_{2}), j = 1, …, t,

iff the set

⋂_{j=1}^{t} \{\frac{a_{j}}{b_{j}} , \frac{c_{j}}{d_{j}} \}

contains a number which is not badly approximable (we consider ∞ to be not badly approximable). In the case of one quadratic form, this was observed by Dani [Da00]. Theorem 1 below is an analog of this fact in dimension d > 2.

Validity of (2) depends on the common position of the hypersurfaces {Q_{j} = 0}, j = 1, …, t. For example, it is easy to see that if these hypersurfaces intersect only at the origin, then (2) fails. We investigate the case when the intersection of the hypersurfaces {Q_{j} = 0}, j = 1, …, t, is still relatively small: they intersect transversally in a strong sense. To formulate our result, we recall that a vector ᾱ = (α_{1}, …, α_{d}) ∈ ℝ^{d} is called well approximable of order r (r > 0) if there exist p̄_{n} ∈ ℤ^{d} and q_{n} > 0 such that

q_{n}^{r} · ∥q_{n}ᾱ − p̄_{n}∥ → 0 as n → ∞.  (3)

Theorem 1 Let Q_{j}, j = 1, …, t, be nondegenerate indefinite quadratic forms in d variables, d ≥ 3. Suppose that the intersection of the hypersurfaces {Q_{j} = 0}, j = 1, …, t, is not {¯0}, and at every point of intersection, which is different from ¯0, the space spanned by the normal vectors to the hypersurfaces has dimension d − 1. Then (2) holds iff the intersection of the hypersurfaces {Q_{j} = 0}, j = 1, …, t, contains a vector v̄ ∈ ℝ^{d} such that for some j = 1, …, t, v_{j} ≠ 0, and the vector (\frac{v_{i}}{v_{j}}: i ≠ j) ∈ ℝ^{d−1} is well approximable of order one.

We also study the case when zero hypersurfaces of the quadratic forms have a common tangent plane at a point of intersection. Choose and fix a line along which the hypersurfaces {Q_{j} = 0}, j = 1, …, t, have a common tangent plane. Let f̄_{1} be the direction vector of this line. We choose a basis {f̄_{i}: i = 1, …, d} such that the vector f̄_{2} is outside of the tangent plane, and f̄_{i}, i = 3, …, d, are in the tangent plane. Define a_{t} ∈ SL(d, ℝ) by

a_{t}f̄_{1} = e^{−t}f̄_{1}, a_{t}f̄_{2} = e^{t}f̄_{2}, a_{t}f̄_{i} = f̄_{i}, i = 3, …, d.  (4)

The following theorem establishes a connection between properties of the semiorbit of a_{t}, t > 0, in SL(d, ℝ)/SL(d, ℤ) and Diophantine condition (2). First results of this type were discovered by Dani [Da85, Da86].

Theorem 2 Let Q_{j}, j = 1, …, t, be nondegenerate indefinite quadratic forms in d variables, d ≥ 3. Suppose that the hypersurfaces {Q_{j} = 0}, j = 1, …, t, have a common tangent plane at a point of intersection, which is different from ¯0. If the semiorbit

{a_{t}SL(d, ℤ): t > 0} ⊆ SL(d, ℝ)/SL(d, ℤ),

where a_{t} is defined in (4), is not relatively compact, then (2) holds. Moreover, if for some α_{j} ∈ ℝ, j = 1, …, t, the quadratic form ∑_{j} α_{j}Q_{j} is definite of rank d − 1, then (2) implies that the semiorbit {a_{t}SL(d, ℤ): t > 0} is not relatively compact in SL(d, ℝ)/SL(d, ℤ).

Remark Since the choice of the vectors f̄_{i}, i = 2, …, d, is not unique, the transformation a_{t} is not uniquely defined. Nonetheless, one can check that the property that the positive semiorbit of a_{t} is bounded is independent of a chosen basis.

Theorems 1 and 2 allow us to investigate how often (2) holds in the space of systems of quadratic forms. For quadratic forms Q_{j}, j = 1, …, t in d variables, we consider a parametric family

(Q_{j}^{g}: j = 1, …, t), g ∈ GL(d, ℝ),

where Q_{j}^{g}(x̄) =^{def} Q(gx̄), and denote by Δ(Q_{1}, …, Q_{t}) the set of g ∈ GL(d, ℝ) such that (Q_{j}^{g}: j = 1, …, t) satisfies (2).

Corollary 3 Let Q_{j}, j = 1, …, t, be nondegenerate indefinite quadratic forms in d variables, d ≥ 3, and Δ = Δ(Q_{1}, …, Q_{t}).

(i) If the intersection of the hypersurfaces {Q_{j} = 0}, j = 1, …, t, is not {¯0}, and at every point of intersection, which is different from ¯0, the space spanned by the normal vectors to the hypersurfaces has dimension d − 1, then Δ has measure zero and Hausdorff dimension d^{2} − \frac{d−2}{2}.

(ii) If the hypersurfaces {Q_{j} = 0}, j = 1, …, t, have a common tangent plane at a point of intersection, which is different from ¯0, then Δ has full Haar measure in GL(d, ℝ) (i.e. its complement has measure zero). Moreover, if for some α_{j} ∈ ℝ, j = 1, …, t, the quadratic form ∑_{j} α_{j}Q_{j} is definite of rank d − 1, then the complement of Δ in GL(d, ℝ) has Hausdorff dimension d^{2}.

Recall that the radical Rad(Q) of a quadratic form Q, which is defined on a vector space S, is the subspace of vectors in S that are orthogonal to S with respect to Q.

We prove the following result that complements Corollary 3(ii):

Theorem 4 Let Q_{j}, j = 1, …, t, be nondegenerate indefinite quadratic forms in d variables, d ≥ 3, and Δ = Δ(Q_{1}, …, Q_{t}). Suppose that for some β_{j} ∈ ℝ, j = 2, …, d,

V =^{def} ⋂_{j=2,...,d} Rad(Q_{j} − β_{j}Q_{1})

has dimension at least 2, and V contains a vector v̄ ≠ ¯0 such that Q_{1}(v̄) = 0.

(a) Unless dim V = 2 and the quadratic form Q_{1}|_{V} is nondegenerate, the complement of Δ in GL(d, ℝ) is contained in a countable union of proper submanifolds of dimension at most d^{2} − d + 1.

(b) If dim V = 2, the quadratic form Q_{1}|_{V} is nondegenerate, and for some α_{j} ∈ ℝ, j = 1, …, t, the quadratic form ∑_{j} α_{j}Q_{j} is definite of rank d − 2, then the complement of Δ in GL(d, ℝ) has Hausdorff dimension d^{2}.

Note that the conditions of Theorem 4 imply that the hypersurfaces {Q_{j} = 0}, j = 1, …, t, have a common tangent plane at v̄. In particular, it follows from Corollary 3(ii) that the complement of Δ has measure 0.

The simplest example for Theorem 4(a) is provided by the system of quadratic forms

Q_{j}(x̄) = x_{1}^{2} + x_{2}x_{3} + a_{j}x_{3}^{2}, j = 1, …, t.

It follows from the result of Dani and Margulis [DM90] that the complement of Δ(Q_{1}, Q_{2}) is a countable union of proper submanifolds of GL(3, ℝ).

It is easy to refine Corollary 3 and Theorem 4 as follows. Denote by Δ′ the set of g ∈ GL(d, ℝ) such that

∀ε > 0 ∃x̄ ∈ ℤ^{d} − {¯0}: 0 < max_{j=1,...,t} |Q_{j}(gx̄)| < ε.  (5)

Corollary 5 Statements of Corollary 3 and Theorem 4 are valid for Δ′.

The next section is devoted to the proofs of stated results. Some open problems are discussed in Section 3.

Acknowledgments: Thanks are due to my advisor V. Bergelson for many stimulating and enlightening discussions. I am also grateful to the referee for several pertinent suggestions, which have significantly improved the quality of this paper.

2 Proofs

Proofs of Theorems 1 and 2 involve only elementary linear algebra. Theorem 1 follows from Lemmas 7 and 8 below. To prove Theorem 2, we reduce it to a question about a system of linear forms. The proof of Corollary 3(i) uses Lemmas 7 and 8 and the result of Jarník [Ja31]. Corollary 3(ii) and Theorem 4 are deduced from known properties of certain flows on the space of unimodular lattices.

If it is not stated otherwise, ∥ · ∥ denotes the Euclidean norm on ℝ^{d}.

The following three lemmas are used in the proof of Theorem 1.

Lemma 6 Let Q_{j}, j = 1, …, t, be nondegenerate indefinite quadratic forms in d variables, d ≥ 3, whose zero hypersurfaces intersect along a line ℓ, and the span of normal vectors to the hypersurfaces {Q_{j} = 0}, j = 1, …, t, at a point of ℓ has dimension d − 1. Denote by π the orthogonal projection on the line ℓ. For m, M, ε > 0, put

R_{m,M} = {x̄ ∈ ℝ^{d}: ∥π(x̄)∥ ≥ M, ∥x̄ − π(x̄)∥ ≤ m∥π(x̄)∥},
T (ε) = {x̄ ∈ R_{m,M}: ∥π(x̄)∥ · ∥x̄ − π(x̄)∥ ≤ ε},
S(ε) = {x̄ ∈ R_{m,M}: |Q_{j}(x̄)| ≤ ε, j = 1, …, t}.

Then there exist m, M, c_{1}, c_{2} > 0 such that for every ε ∈ (0, 1),

T (c_{1}ε) ⊆ S(ε) ⊆ T (c_{2}ε).

Proof It is enough to prove the lemma for a system of quadratic forms Q_{j}, j = 1, …, d − 1, such that normal vectors to {Q_{j} = 0}, j = 1, …, d − 1, at a point of ℓ are linearly independent.

For a vector x̄ = (x_{1}, …, x_{d}) ∈ ℝ^{d}, put x = (x_{1}, …, x_{d−1}).

Using a change of variables, we can transform ℓ to the line {x_{i} = 0: i = 1, …, d − 1}. Thus, it is enough to consider the case when the line ℓ is as above. Then

Q_{i}(x̄) = q_{i}(x) + x_{d}L_{i}(x), i = 1, …, d − 1.  (6)

Note that L_{i} is independent of x_{d} because Q_{i}(0, …, 0, 1) = 0. The tangent plane to {Q_{i} = 0} at (0, …, 0, 1) is {L_{i} = 0}.

Let

T (a, ε) = T (ε) ∩ {x_{d} = a} and S(a, ε) = S(ε) ∩ {x_{d} = a}.

For |a| ≥ M, we have

T (a, ε) = {(x, a): |a| · ∥x∥ ≤ ε, ∥x∥ ≤ m|a|},  (7)
S(a, ε) = {(x, a): |q_{i}(x) + aL_{i}(x)| < ε, i = 1, …, d − 1; ∥x∥ ≤ m|a|} .  (8)

First, we claim that for fixed sufficiently small m > 0,

diam(S(a, ε)) → 0  (9)

uniformly on ε ∈ (0, 1) as |a| → ∞. Let L = (l_{ij}) be a linear map where l_{ij} are the coefficients of the linear forms L_{i}. Note that L is nondegenerate because the normal vectors to {Q_{j} = 0}, j = 1, …, d − 1, at a point of ℓ are linearly independent. Using (6) and (8), we have

|aL_{i}(x)| ≤ ε + |q_{i}(x)| ≤ 1 + β_{i}∥x∥^{2} ≤ 1 + β_{i}m|a| · ∥x∥.  (10)

for x ∈ S(a, ε) and i = 1, …, d − 1.

Suppose that there exists a sequence (u^{(n)}, a_{n}) ∈ S(a_{n}, ε) with |a_{n}| ≥ 1, and ∥u^{(n)}∥ → ∞. We may assume that \frac{u^{(n)}}{∥u^{(n)}∥} → v_{0} for some v_{0} on the unit sphere S^{d−2}. From (10) we get

|L_{i}(u^{(n)})| ≤ \frac{1}{|a_{n}|} + β_{i}m∥u^{(n)}∥ ≤ 1 + β_{i}m∥u^{(n)}∥, i = 1, …, d − 1.

It follows that

|L_{i}(v_{0})| ≤ β_{i}m, i = 1, …, d − 1.  (11)

Take

m = \frac{1}{2} min \{max \{β_{i}^{−1}|L_{i}(x)|: i = 1, …, d − 1\}: x ∈ S^{d−2}\} .  (12)

Note that m > 0 because L_{i}, i = 1, …, d − 1, are linearly independent. Then the equations (11) and (12) contradict each other. This shows that diam(S(a, ε)) is bounded for ε ∈ (0, 1) and a ≥ 1. From the first part of (10), we see that |a| · ∥L(x)∥ for x ∈ S(a, ε) is bounded too. Since L is nondegenerate, for some c > 0, diam(S(a, ε)) < \frac{c}{|a|} when ε ∈ (0, 1) and a ≥ 1. This proves the claim (9).

Consider a transformation u: ℝ^{d−1} → ℝ^{d−1} defined by

u_{i} = q_{i}(x) + aL_{i}(x), i = 1, …, d − 1.

By (8),

S(a, ε) = {(x, a): |u_{i}(x)| < ε, i = 1, …, d − 1; ∥x∥ ≤ m|a|} .  (13)

The Jacobian of the transformation u is J = (\frac{∂q_{i}}{∂x_{j}} ) + aL. Note that det J(0) = det L ≠ 0. Therefore by (9), u is a diffeomorphism on S(a, ε) for sufficiently large |a|. Let g = dx_{1}^{2} +⋯+dx_{d−1}^{2} be the standard Riemann metric, u^{∗}g = du_{1}^{2} +⋯+du_{d−1}^{2} be the pull-back of g under the transformation u, and L^{∗}g be the pull-back of g under L. Clearly, for some r_{1}, r_{2} > 0,

r_{1}g < L^{∗}g < r_{2}g.

Since a^{−1}J → L as |a| → ∞ uniformly on compact sets of ℝ^{d−1}, \frac{u^{∗}g}{a^{2}} → L^{∗}g uniformly on compact sets. Therefore,

r_{1}g ≤ \frac{u^{∗}g}{a^{2}} ≤ r_{2}g  (14)

on S(a, ε) for |a| sufficiently large (say |a| ≥ M). Note that ∥x∥ is the distance to the origin with respect to the metric g, and ∥u(x)∥ is the distance to the origin with respect to u^{∗}g. Therefore, it follows from (14) that for x ∈ S(a, ε) and |a| ≥ M,

r_{1}^{1/2}|a| · ∥x∥ ≤ ∥u(x)∥ ≤ r_{2}^{1/2}|a| · ∥x∥.

By (7) and (13), for |a| ≥ M,

T (a, r_{2}^{−1/2}ε) ⊆ S(a, ε) ⊆ T (a, r_{1}^{−1/2}\sqrt{2}ε) .

The lemma is proved. □

Lemma 7 For a line ℓ in ℝ^{d}, denote by π_{ℓ} the orthogonal projection on the line ℓ. Assume that zero hypersurfaces of a system of nondegenerate indefinite quadratic forms Q_{j}, j = 1, …, t, in d variables intersect at a point, which is different from ¯0, and at every nonzero point of intersection, the span of the normal vectors to these hypersurfaces has dimension d − 1. Then (2) holds iff there exist a line ℓ in the intersection of the hypersurfaces of {Q_{j} = 0}, j = 1, …, t, and a sequence x̄^{(n)} ∈ ℤ^{d} − {¯0} such that

∥π_{ℓ}(x̄^{(n)})∥ → ∞ and ∥π_{ℓ}(x̄^{(n)})∥ · ∥x̄^{(n)} − π_{ℓ}(x̄^{(n)})∥ → 0.  (15)

Proof Let m, M, c_{1}, c_{2} > 0 be as in Lemma 6.

Suppose that Q_{j}, j = 1, …, t, satisfy (2). Then there exists a sequence x̄^{(n)} ∈ ℤ^{d} − {¯0} such that

ε_{n} =^{def} max{|Q_{j}(x̄^{(n)})|: j = 1, …, t} → 0.  (16)

Passing if needed to a subsequence, we may assume that

ȳ^{(n)} =^{def} \frac{x̄^{(n)}}{∥x̄^{(n)}∥} → ȳ_{0}

for some ȳ_{0} on the unit sphere. Then Q_{j}(ȳ_{0}) = 0, j = 1, …, t. Let ℓ be the line through ȳ_{0} and the origin.

If ∥x̄^{(n)}∥ ↛ ∞, we can take {x̄^{(n)}} to be bounded. Then for each j, Q_{j}(x̄^{(n)}) admits only finitely many values for n ≥ 1. Therefore, by (16), Q_{j}(x̄^{(n)}) = 0, j = 1, …, t, for sufficiently large n. We can replace x̄^{(n)} by nx̄^{(n)}. Thus, we may assume that ∥x̄^{(n)}∥ → ∞.

Since ȳ^{(n)} → ȳ_{0}, the sequence x̄^{(n)} is inside of the cone

∥x̄ − π_{ℓ}(x̄)∥ < m∥π_{ℓ}(x̄)∥

for sufficiently large n. It follows that if ∥π_{ℓ}(x̄^{(n)})∥ ↛ ∞, then ∥x̄^{(n)}∥ ↛ ∞. Therefore, π_{ℓ}(x̄^{(n)}) → ∞. In particular, x̄^{(n)} ∈ R_{m,M} for sufficiently large n. By Lemma 6,

∥π_{ℓ}(x̄^{(n)})∥ · ∥x̄^{(n)} − π_{ℓ}(x̄^{(n)})∥ ≤ c_{2}ε_{n} → 0.

Conversely, suppose that (15) is satisfied for some line ℓ in the intersection of the hypersurfaces {Q_{j} = 0}, j = 1, …, t. Then for sufficiently large n, x̄^{(n)} ∈ R_{m,M}. Applying Lemma 6, we get

|Q_{j}(x̄^{(n)})| ≤ c_{1}^{−1}∥π_{ℓ}(x̄^{(n)})∥ · ∥x̄^{(n)} − π_{ℓ}(x̄^{(n)})∥ → 0, j = 1, …, t.

This proves the lemma. □

Lemma 8 Let v̄ = (v_{1}, …, v_{d}) ∈ ℝ^{d} with v_{j} ≠ 0 for some j = 1, …, d, and π is the orthogonal projection on the direction v̄. Then the vector (\frac{v_{i}}{v_{j}}: i ≠ j) ∈ ℝ^{d−1} is well

approximable of order 1 iff for some sequence x̄^{(n)} ∈ ℤ^{d} − {¯0},

∥π(x̄^{(n)})∥ → ∞ and ∥π(x̄^{(n)})∥ · ∥x̄^{(n)} − π(x̄^{(n)})∥ → 0.  (17)

Proof Without loss of generality, j = 1. Put α_{i} = \frac{v_{i+1}}{v_{1}} , i = 1, …, d − 1.

Let L_{i}(x̄) = α_{i}x_{1} − x_{i+1}, i = 1, …, d − 1. Note that the planes defined by L_{i} = 0, i = 1, …, d − 1, intersect along the line in direction v̄.

Suppose that (α_{1}, …, α_{d−1}) is well approximable of order 1. There exists a sequence x̄^{(n)} ∈ ℤ^{d} such that

|x_{1}^{(n)}| · |L_{i} (x̄^{(n)})| → 0, i = 1, …, d − 1,  (18)

with x_{1}^{(n)} ≠ 0 for all n. It follows that

L_{i}(x̄^{(n)}) → 0, i = 1, …, d − 1.  (19)

Suppose that |x_{1}^{(n)}| is bounded. A linear map A: ℝ^{d} → ℝ^{d}, which is defined by

(Ax̄)_{i} = L_{i}(x̄), i = 1, …, d − 1, and (Ax̄)_{d} = x_{1},  (20)

is nondegenerate. Hence, by (19), x̄^{(n)} is bounded, and L_{i}(x̄^{n}) = 0, i = 1, …, d − 1, for sufficiently large n. In this case, (17) holds for the sequence nx̄^{(n)}.

If |x̄_{1}^{(n)}| is not bounded, we may assume that |x̄_{1}^{(n)}| → ∞. By (19), for sufficiently large n,

∥Ax̄^{(n)}∥ = (∑_{i=1}^{d−1} L_{i}(x̄^{(n)})^{2} + (x_{1}^{(n)})^{2})^{1/2} ≤ \sqrt{d}|x_{1}^{(n)}|.  (21)

Since A is nondegenerate, for some c_{1} > 0 and every x̄ ∈ ℝ^{d},

∥π(x̄)∥ ≤ c_{1}∥Ax̄∥.  (22)

Let P be the plane through the origin orthogonal to v̄. A map C: P → ℝ^{d−1} defined by

(Cx̄)_{i} = L_{i}(x̄), i = 1, …, d − 1,

is invertible. Therefore, for some c_{2} > 0 and every x̄ ∈ P,

∥x̄ − π(x̄)∥ ≤ c_{2} max{|L_{i}(x̄)|: i = 1, …, d − 1}.  (23)

In fact, the last inequality holds for every x̄ ∈ ℝ^{d} because it is independent of translations by vectors parallel to v̄. Now (17) follows from (18), (21), (22), (23).

Conversely, suppose that (17) holds for some sequence x̄^{(n)} ∈ ℤ^{d} − {¯0}. Note that for x̄ ∈ ℝ^{d}, |L_{i}(x̄)| is up to a constant the distance from x̄ to the plane {L_{i} = 0}, and ∥x̄ − π(x̄)∥ is the distance from x̄ to the line through v̄. Therefore, for some r_{i} > 0 and every x̄ ∈ ℝ^{d},

|L_{i}(x̄)| ≤ r_{i}∥x̄ − π(x̄)∥, i = 1, …, d − 1.  (24)

In particular, it follows that

L_{i}(x̄^{(n)}) → 0, i = 1, …, d − 1.  (25)

Clearly, x_{1}^{(n)} ≠ 0 for sufficiently large n.

Consider a linear map B defined by

(Bx̄)_{i} = L_{i}(x̄), i = 1, …, d − 1, and (Bx̄)_{d} = π(x̄).

Since B is nondegenerate, for some r_{3} > 0 and every x̄ ∈ ℝ^{d},

|x_{1}| ≤ r_{3}∥Bx̄∥.  (26)

By (17) and (25), for sufficiently large n,

∥Bx̄^{(n)}∥ = (∑_{i=1}^{d−1} L_{i}(x̄^{(n)})^{2} + ∥π(x̄^{(n)})∥^{2})^{1/2} ≤ \sqrt{d}∥π(x̄^{(n)})∥.  (27)

Finally, (18) follows from (17), (24), (26), and (27). Thus, the vector (α_{1}, …, α_{d−1}) is well approximable of order 1. □

Combining Lemmas 7 and 8, we deduce Theorem 1.

The proof of Corollary 3(i) uses Lemma 7, Lemma 8, and the result of Jarník on Hausdorff dimension of the set of well approximable vectors [Ja31]. It is known that the intersection of the set of well approximable vectors of order r in ℝ^{d} with every nonempty open set has Hausdorff dimension \frac{d+1}{r+1}.

Proof of Corollary 3(i) Let Q be the projective variety in the complex projective space ℙ^{d−1}(ℂ) defined by Q_{j} = 0, j = 1, …, t. Since at every point of Q(ℝ) the rank of Jacobian \frac{∂(Q_{1},...,Q_{t})}{∂(x_{1},...,x_{d})} is equal to d − 1, it follows that every irreducible component of Q that has nonempty intersection with Q(ℝ) has dimension zero. Therefore, Q(ℝ) consists of finitely many points. This means that the zero hypersurfaces {Q_{j} = 0}, j = 1, …, t, in ℝ^{d} intersect along only finitely many lines that pass through the origin. Let v̄_{s}, s = 1, …, N, be the direction vectors of these lines. Then zero hypersurfaces of Q_{j}^{g}, j = 1, …, t, intersect along vectors g^{−1}v̄_{s}, s = 1, …, N. For some g_{s} ∈ GL(d, ℝ), v̄_{s} = g_{s}ē_{1}, where e_{1} = (1, 0, …, 0). We have

g^{−1}v̄_{s} = ((g^{−1}g_{s})_{11}, …, (g^{−1}g_{s})_{d1}) , s = 1, …, N.

By Lemma 7,

Δ = ⋃_{s=1}^{N} Δ_{s},

where Δ_{s} is the set of g ∈ GL(d, ℝ) such that (15) holds for the line ℓ through the origin in direction g^{−1}v̄_{s}. Clearly, it is enough to prove the theorem for each of the sets Δ_{s} separately.

Fix s = 1, …, N. Let

G_{s,i} = {g ∈ GL(d, ℝ): (g^{−1}g_{s})_{i1} ≠ 0},

and Δ_{s,i} = Δ_{s} ∩ G_{s,i} for i = 1, …, d.

By Lemma 8, for g ∈ G_{s,i}, g belongs to Δ_{s,i} iff the vector

(\frac{(g^{−1}g_{s})_{j1}}{(g^{−1}g_{s})_{i1}} : j ≠ i)

is well approximable of order 1. Consider maps F_{s,i}: G_{s,i} → M(d, ℝ), i = 1, …, d, defined by

F_{s,i}(g)_{j1} = \frac{(g^{−1}g_{s})_{j1}}{(g^{−1}g_{s})_{i1}} for j ≠ i,
F_{s,i}(g)_{j_{1}j_{2}} = (g^{−1}g_{s})_{j_{1}j_{2}} for (j_{1}, j_{2}) ∉ {(j, 1), j ≠ i}.

Note that F_{s,i} is a diffeomorphism on G_{s,i}, and F_{s,i}(G_{s,i}) is a complement of a finite union of hypersurfaces in M(d, ℝ). Let W_{i} be the set of g ∈ M(d, ℝ) such that the vector (g_{j1}: j ≠ i) is well-approximable of order 1. We have that for g ∈ G_{s,i}, g belongs to Δ_{s,i} iff F_{s,i}(g) belongs to W_{i}, i.e. Δ_{s,i} = F_{s,i}^{−1} (W_{i} ∩ F_{s,i}(G_{s,i})). Therefore,

Δ_{s} = ⋃_{i=1}^{d} Δ_{s,i} = ⋃_{i=1}^{d} F_{s,i}^{−1} (W_{i} ∩ F_{s,i}(G_{s,i})) .  (28)

The intersection of the set of well approximable vectors of order 1 in ℝ^{d−1} with every nonempty open subset has Hausdorff dimension \frac{d}{2}. It follows that the intersection of the set W_{i}, i = 1, …, d, with every nonempty open subset of M(d, ℝ) has Hausdorff dimension d(d − 1) + 1 + \frac{d}{2} = d^{2} − \frac{d−2}{2} (see [BD, Section 3.5.5]). Since F_{s,i}(G_{s,i}) is open in M(d, ℝ), the set W_{i} ∩ F_{s,i}(G_{s,i}) has the same property. Therefore, by (28), the intersection of Δ_{s} with every nonempty open subset of M(d, ℝ) has Hausdorff dimension d^{2} − \frac{d−2}{2}. □

The following two lemmas are crucial for the proofs of Theorem 2 and Corollary 3(ii). The proofs of the lemmas are based on ideas of Dani [Da85, Da86].

Lemma 9 Let L_{i}, i = 1, …, d, be linearly independent linear forms in d variables, d ≥ 2. Define a_{t} ∈ SL(d, ℝ) by

L_{1}(a_{t}x̄) = e^{−t}L_{1}(x̄),  (29)
L_{2}(a_{t}x̄) = e^{t}L_{2}(x̄),  (30)
L_{i}(a_{t}x̄) = L_{i}(x̄), i = 3, …, d.

Then the set {a_{t}SL(d, ℤ): t > 0} is not bounded in SL(d, ℝ)/SL(d, ℤ) iff there exists a sequence x̄^{(n)} ∈ ℤ^{d} − {¯0} such that

L_{1}(x̄^{(n)})L_{2}(x̄^{(n)}) → 0 and L_{i}(x̄^{(n)}) → 0, i = 2, …, d.  (31)

Proof Let ∥ · ∥ be the norm on ℝ^{d} defined by

∥x̄∥ = max{|L_{i}(x̄)|: i = 1, …, d}, x̄ ∈ ℝ^{d}.

By Mahler Compactness Criterion, the orbit {a_{t}SL(d, ℤ): t > 0}, is unbounded iff for some t_{n} > 0 and x̄^{(n)} ∈ ℤ^{d} − {¯0}, a_{t_{n}}x̄^{(n)} → ¯0, i.e.

∥a_{t_{n}}x̄^{(n)}∥ = max{e^{−t_{n}}|L_{1}(x̄^{(n)})|, e^{t_{n}}|L_{2}(x̄^{(n)})|,  (32)
|L_{i}(gx̄^{(n)})|: i = 3, …, d} → 0.

It is clear that (32) implies (31). Conversely, suppose that (31) holds.

First, we consider the case when L_{2}(x̄^{(n)}) = 0 for infinitely many n. Taking a subsequence, we may assume that L_{2}(x̄^{(n)}) = 0 for all n. Then (32) holds for every sequence t_{n} → ∞ such that

e^{−t_{n}}|L_{1}(x̄^{(n)})| → 0.

Now we may assume that L_{2}(x̄^{(n)}) ≠ 0. If L_{1}(x̄^{(n)}) is bounded, then since the linear forms L_{i}, i = 1, …, d, are linearly independent, x̄^{(n)} is bounded. This implies that for sufficiently large n, L_{2}(x̄^{(n)}) = 0. Thus, we may assume that |L_{1}(x̄^{(n)})| → ∞. Then (32) holds for the sequence t_{n} such that e^{t_{n}} = |L_{1}(x̄^{(n)})|^{1/2} · |L_{2}(x̄^{(n)})|^{−1/2}. Note that t_{n} > 0 for sufficiently large n. This proves the lemma. □

Lemma 10 Let L_{i}, i = 1, …, d, be linearly independent linear forms in d variables, d ≥ 2. Then the set Φ of g ∈ GL(d, ℝ) such that

L_{1}(gx̄^{(n)})L_{2}(gx̄^{(n)}) → 0 and L_{i}(gx̄^{(n)}) → 0, i = 2, …, d.  (33)

for some sequence x̄^{(n)} ∈ ℤ^{d} − {¯0} has full Haar measure, and its complement has Hausdorff dimension d^{2}.

Proof Let L_{i}^{0}(x̄) = x_{i}, i = 1, …, d. For some g_{0} ∈ GL(d, ℝ), L_{i}(gx̄) = L_{i}^{0}(g_{0}gx̄), i = 1, …, d. Since multiplication by g_{0} is a diffeomorphism of GL(d, ℝ), we may assume without loss of generality that g_{0} = 1 and L_{i} = L_{i}^{0}, i = 1, …, d.

For every matrix u ∈ GL(d, ℝ),

u = diag(λ, 1, …, 1) · v

with λ ∈ ℝ^{×} and v ∈ SL(d, ℝ). Moreover, the map

GL(d, ℝ) → ℝ^{×} × SL(d, ℝ): u ↦ (λ, v)

is a diffeomorphism. Since

L_{1}(ux̄) = λL_{1}(vx̄), L_{i}(ux) = L_{i}(vx̄), i = 2, …, d,

(33) holds for g = u iff it holds for g = v. Therefore, if we show that the set Φ ∩ SL(d, ℝ) has full measure in SL(d, ℝ), this will imply that the set Φ has full measure in GL(d, ℝ) too. Also by the argument as in [BD, Section 3.5.5], if the set SL(d, ℝ) − (Φ ∩ SL(d, ℝ)) has Hausdorff dimension d^{2} − 1, then the set GL(d, ℝ) − Φ has Hausdorff dimension d^{2}. Hence, the proof reduces to a question about SL(d, ℝ).

By Lemma 9, (33) holds for some sequence x̄^{(n)} ∈ ℤ^{d} − {¯0} iff the set

{g^{−1}a_{t}gSL(d, ℤ): t > 0}

is unbounded in SL(d, ℝ)/SL(d, ℤ). This is equivalent to the set {a_{t}gSL(d, ℤ): t > 0} being unbounded. It is known that the set of g ∈ SL(d, ℝ) such that the orbit {a_{t}gSL(d, ℤ): t > 0} is bounded has Haar measure zero (Moore Ergodicity Theorem [Zi, Theorem 2.2.6]), and its intersection with every nonempty open subset has Hausdorff dimension d^{2} − 1 (Kleinbock, Margulis [KM96]). Therefore, the set of g ∈ SL(d, ℝ) such that (33) holds for some sequence x̄^{(n)} ∈ ℤ^{d} −{¯0} has full measure, and its complement in SL(d, ℝ) has Hausdorff dimension d^{2} − 1. This implies the lemma. □

Proof of Theorem 2 With respect to the basis {f̄_{i}: i = 1, …, d},

Q_{j}(ȳ) = q_{j}(y_{2}, …, y_{d}) + y_{1}l_{j}(y_{2}, …, y_{d}), j = 1, …, t,  (34)

where q_{j} is a quadratic form, and l_{j} is a linear form. Note that l_{j} is independent of y_{1} because Q_{j}(f̄_{1}) = 0. The tangent plane to {Q_{j} = 0} at f̄_{1} is {l_{j} = 0}. Thus, l_{j} = c_{j}l, j = 1, …, t, for some c_{j} ∈ ℝ − {0} and a linear form l. Since f̄_{1} and f̄_{i}, i = 3, …, d, are in the tangent plane, l(ȳ) = y_{2}. Let L_{i}(x̄), i = 1, …, d, denote the coordinates of vector x̄ with respect to the basis {f̄_{i}: i = 1, …, d}. Then

Q_{j}(x̄) = q_{j}(L_{2}(x̄), …, L_{d}(x̄)) + c_{j}L_{1}(x̄)L_{2}(x̄), j = 1, …, t,

and (29) holds. It follows that (2) holds for (Q_{j}: j = 1, …, t) provided that for some sequence x̄^{(n)} ∈ ℤ^{d} − {¯0},

L_{1}(x̄^{(n)})L_{2}(x̄^{(n)}) → 0 and L_{i}(x̄^{(n)}) → 0, i = 2, …, d.  (35)

Thus, the first statement of the theorem follows from Lemma 9.

Suppose that Q = ∑_{j} α_{j}Q_{j} is definite of rank d − 1. Since Q is definite, it follows from (34) that Q(ȳ) = Q(y_{2}, …, y_{d}). Therefore, (2) implies (35) for some sequence x̄^{(n)} ∈ ℤ^{d} − {¯0}, and the second statement of the theorem follows from Lemma 9. □

Proof of Corollary 3(ii) It was shown in the proof of Theorem 2 that (2) holds for (Q_{j}^{g}: j = 1, …, t) provided that for some sequence x̄^{(n)} ∈ ℤ^{d} − {¯0},

L_{1}(gx̄^{(n)})L_{2}(gx̄^{(n)}) → 0 and L_{i}(gx̄^{(n)}) → 0, i = 2, …, d,  (36)

and when a linear combination of Q_{j}, j = 1, …, d, is definite of rank d − 1, (Q_{j}^{g}: j = 1, …, t) satisfies (2) iff (36) holds. Thus, Corollary 3(ii) follows from Lemma 9. □

To prove Theorem 4(a), we use the following lemma:

Lemma 11 Let Q_{j}, j = 1, …, t, be a system of quadratic forms that satisfies the conditions of Theorem 4(a). Then there exists a nontrivial one-parameter unipotent subgroup of SL(d, ℝ) which leaves all Q_{j}, j = 1, …, t, invariant.

Proof Let Q_{1}|_{V} be nondegenerate. Then dim V ≥ 3. Write

ℝ^{d} = V ⊕ W

where W is the orthogonal complement of V with respect to Q_{1}. Then the group

H =^{def} SO(Q_{1}|_{V}) ⊕ id_{W}

leaves Q_{1} invariant. Since V is in the radical of Q_{j} − β_{j}Q_{1}, the forms Q_{j}, j = 2, …, t are invariant under H too. The group H is a noncompact semisimple Lie group. Hence, it contains a nontrivial one-parameter unipotent subgroup. This proves the lemma when Q_{1}|_{V} is nondegenerate.

Now suppose that Q_{1}|_{V} is degenerate, and Q_{1}|_{V} ≠ 0. Choose a basis {f̄_{i}: i = 1, …, d} of ℝ^{d} such that f̄_{1} is in the radical of Q_{1}|_{V}, f̄_{2} is in V, Q_{1}(f̄_{2}) ≠ 0, and f̄_{i}, i = 3, …, t, are orthogonal to f̄_{2} with respect to Q_{1}. Since f̄_{2} is in the radical of Q_{j} − β_{j}Q_{1}, j = 2, …, t, f̄_{2} is orthogonal to f̄_{i}, i = 3, …, t, with respect to Q_{j}, j = 2, …, t. In the basis {f̄_{i}: i = 1, …, d},

Q_{j}(ȳ) = β_{j}(y_{2}^{2} + y_{1}L(y_{3}, …, y_{d})) + q_{j}(y_{3}, …, y_{d}), j = 1, …, t

for some linear form L and quadratic forms q_{j}, j = 1, …, t. (Here we put β_{1} = 1.) Note that L is the same all j because f̄_{1} is in the radical of Q_{j} − β_{j}Q_{1}, j = 2, …, t. Define a linear transformation u_{t} ∈ SL(d, ℝ) by

u_{t} : y_{1} ↦ y_{1} − 2ty_{2} − t^{2}L(y_{3}, …, y_{d}),
u_{t} : y_{2} ↦ y_{2} + tL(y_{3}, …, y_{d}),
u_{t} : y_{i} ↦ y_{i}, i ≥ 3.

Then {u_{t}: t ∈ ℝ} is a nontrivial one-parameter unipotent group that stabilizes Q_{j}, j = 1, …, t.

It remains to consider the case when Q_{1}|_{V} = 0. Choose a basis {f̄_{i}: i = 1, …, d}, such that f̄_{1} and f̄_{2} is in V. With respect to this basis,

Q_{j}(ȳ) = β_{j}(y_{1}L_{1}(y_{3}, …, y_{d}) + y_{2}L_{2}(y_{3}, …, y_{d})) + q_{j}(y_{3}, …, y_{d})

for some linear forms L_{1}, L_{2}, and quadratic forms q_{j}, j = 1, …, t. (Here we put β_{1} = 1.) Note that L_{1} and L_{2} are independent of j because f̄_{1} and f̄_{2} are in the radical of Q_{j} − β_{j}Q_{1}, j = 2, …, t. The linear forms L_{1} and L_{2} are not zero because the quadratic form Q_{1} is nondegenerate. Define a linear transformation v_{t} ∈ SL(d, ℝ) by

v_{t} : y_{1} ↦ y_{1} + tL_{2}(y_{3}, …, y_{d}),
v_{t} : y_{2} ↦ y_{2} − tL_{1}(y_{3}, …, y_{d}),
v_{t} : y_{i} ↦ y_{i}, i ≥ 3.

Then {v_{t}: t ∈ ℝ} is a nontrivial one-parameter unipotent group that stabilizes Q_{j}, j = 1, …, t. We have proved the lemma. □

The following Lemma is used in the proof of Theorem 4(b). Its proof is essentially the same as the proof of Lemma 9 and is omitted.

Lemma 12 Let L_{i}, i = 1, …, d, be linearly independent linear forms in d variables, d ≥ 2, and a_{t} is defined as in (29). Then the set {a_{t}SL(d, ℤ): t ∈ ℝ} is not bounded in SL(d, ℝ)/SL(d, ℤ) iff there exists a sequence x̄^{(n)} ∈ ℤ^{d} − {¯0} such that

L_{1}(x̄^{(n)})L_{2}(x̄^{(n)}) → 0 and L_{i}(x̄^{(n)}) → 0, i = 3, …, d.  (37)

Proof of Theorem 4 Let G = SL(d, ℝ) and Γ = SL(d, ℤ).

To prove (a), we show first that the complement of Δ ∩ G in G is contained in a countable union of submanifolds of dimension at most d^{2} − d. Denote by U ⊂ G a one-parameter unipotent subgroup that stabilizes all Q_{j}, j = 1, …, t. Such a subgroup exists by Lemma 11. By Ratner’s topological rigidity [Ra91], the set Ω of g ∈ G such that UgΓ is not dense in G/Γ is a countable union of sets of the form F Γ where F is a connected Lie subgroup of G. It is known that the maximal dimension of a proper connected subgroup of G is d^{2} −d [OV, Sec. 3.3]. By Mahler compactness criterion, for every g ∈ G − Ω, there exist u_{n} ∈ U and x̄^{(n)} ∈ ℤ^{d} − {¯0} such that u_{n}gx̄^{(n)} → ¯0 as n → ∞. Then

Q_{j}(gx̄^{(n)}) = Q_{j}(u_{n}gx̄^{(n)}) → 0, j = 1, …, t.

Therefore, G − Δ ∩ G is contained in Ω.

Let GL^{±}(d, ℝ) be the group of matrices with positive/negative discriminant. It is clear that

Δ ∩ GL^{+}(d, ℝ) = ⋃_{λ>0} λ · (Δ ∩ G)  (38)

Therefore, the complement of Δ∩GL^{+}(d, ℝ) in GL^{+}(d, ℝ) is contained in a countable union of submanifolds of dimension at most d^{2} − d + 1. Similarly,

Δ ∩ GL^{−}(d, ℝ) = ⋃_{λ>0} λg_{0} · (Δ̃ ∩ G)  (39)

for a fixed g_{0} ∈ GL^{−}(d, ℝ) and Δ̃ =^{def} Δ(Q_{1}^{g_{0}}, …, Q_{t}^{g_{0}}). It follows that the complement of Δ ∩ GL^{−}(d, ℝ) in GL^{−}(d, ℝ) is contained in a countable union of submanifolds of dimension at most d^{2} − d + 1. This proves (a).

Now we prove (b). Take a basis {f̄_{i}: i = 1, …, d} such that V = ⟨f̄_{1}, f̄_{2}⟩, and f̄_{i}, i = 3, …, d is orthogonal to V with respect to Q_{1}. It follows from the definition of V that f̄_{i}, i = 3, …, d are orthogonal to V with respect to Q_{j}, j = 2, …, t, too. In addition, we can change f̄_{1} and f̄_{2} such that with respect to the basis {f̄_{i}: i = 1, …, d},

Q_{j}(ȳ) = β_{j}y_{1}y_{2} + q_{j}(y_{3}, …, y_{d}), j = 1, …, t  (40)

for some quadratic forms q_{j}, j = 1, …, t. (Here we put β_{1} = 1.) Let Q =^{def} ∑_{j} α_{j}Q_{j} be a definite form of rank d − 2. It follows from (40) that Q(ȳ) = Q(y_{3}, …, y_{d}). Denote by L_{i}(x̄), i = 1, …, d, the coordinates of a vector x̄ with respect to the basis {f̄_{i}: i = 1, …, d}. Then (2) holds for (Q_{j}^{g}: j = 1, …, t) with g ∈ G iff for some sequence x̄^{(n)} ∈ ℤ^{d} − {¯0},

L_{1}(gx̄^{(n)})L_{2}(gx̄^{(n)}) → 0 and L_{i}(gx̄^{(n)}) → 0, i = 3, …, d.

By Lemma 12, this is equivalent to the set {g^{−1}a_{t}gΓ: t ∈ ℝ} being unbounded in G/Γ. Therefore, by the result of Kleinbock and Margulis [KM96], the set G−(Δ∩G) has Hausdorff dimension d^{2} − 1.

To show that the complement of Δ in GL(d, ℝ) has Hausdorff dimension d^{2}, one may use (38), (39), and [BD, Section 3.5.5]. This proves (b). □

Proof of Corollary 5 Under the conditions in (i), the intersection of hypersurfaces Q_{j} = 0, j = 1, …, t, consists of finitely many lines that pass through the origin. Denote by v̄_{s}, s = 1, …, N, the direction vectors of these lines. Note that for g ∈ GL(d, ℝ), hypersurfaces Q_{j}^{g} = 0, j = 1, …, t, intersect along lines in directions g^{−1}v̄_{s}, s = 1, …, N. For s = 1, …, N and x̄ ∈ ℝ^{d} − {¯0}, denote by R_{s}(x̄) the set of g ∈ GL(d, ℝ) such that g^{−1}v̄_{s} is collinear with x̄. The set R_{s}(x̄) is a submanifold of dimension d^{2} − (d − 1). Since

Δ − Δ′ ⊆ ⋃_{1≤s≤N; x̄∈ℤ^{d}−{¯0}}R_{s}(x̄),

the set Δ − Δ′ has Hausdorff dimension at most d^{2} − (d − 1) < d^{2} − \frac{d−2}{2}. This shows that the set Δ′ has Hausdorff dimension d^{2} − \frac{d−2}{2}.

To prove (ii), we define

K_{j}(x̄) = {g ∈ M(d, ℝ): Q_{j}(gx̄) = 0}, j = 1, …, t, x̄ ∈ ℝ^{d}.

This is an algebraic subset of M(d, ℝ), and for x̄ ≠ 0, K_{j}(x̄) ≠ M(d, ℝ), which implies that it has measure 0. Note that

Δ − Δ′ ⊆ ⋃_{1≤j≤t; x̄∈ℤ^{d}−{¯0}}K_{j}(x̄),  (41)

where the last set has measure 0 and Hausdorff dimension < d^{2}. Therefore, when Δ has complement in GL(d, ℝ) of measure zero, so does Δ′. The statement about Hausdorff dimension follows from (41) too. This proves (ii).

We use notations from the proof of Theorem 4. If we show that G − Δ′ ∩ G is contained in Ω, then it is possible to finish the proof as in Theorem 4(a). There exists x̄^{0} ∈ ℝ^{d} such that Q_{j}(x̄^{0}) ≠ 0 for every j = 1, …, t. Therefore, for every ε > 0, there exists x̄_{ε} ∈ ℝ^{d} such that

0 < |Q_{j}(x̄_{ε})| < ε, j = 1, …, t.

Write x̄_{ε} = g_{ε}ē_{1} for some g_{ε} ∈ G and e_{1} = (1, 0, …, 0). For g ∉ Ω, \overline{UgΓ} = G. Thus, there exist u ∈ U and γ ∈ Γ such that

0 < |Q_{j}(ugγē_{1})| = |Q_{j}(gγē_{1})| < ε, j = 1, …, t.

Hence, g ∈ Δ′. This show that G − Δ′ ∩ G ⊆ Ω and proves Theorem 4(a).

In the case (b), the intersection of the hypersurfaces {Q_{j} = 0}, j = 1, …, t, consists of two lines. Thus, by the same argument as in the proof of Corollary 5(i), the set Δ − Δ′ has Hausdorff dimension at most d^{2} − (d − 1). This implies part (b) of Theorem 4. □

3 Open Problems

Theorems 1 and 2 provide information about property (2) only when the dimension of the space spanned by normal vectors to the hypersurfaces {Q_{j} = 0}, j = 1, …, t, is either 1 or d − 1. Clearly, this leaves a gap for d > 3. It is of interest to investigate property (2) assuming other conditions on the intersection of the hypersurfaces {Q_{j} = 0}, j = 1, …, t. At least, one should try to give a complete answer to the question about the magnitude of the set Δ(Q_{1}, …, Q_{t}), which was partially studied in Corollary 3.

Corollary 3 illustrates that in some cases, property (2) has complicated Diophantine nature. This is analogous to the situation with Oppenheim conjecture in dimension 2. In higher dimensions, the following conjecture seems plausible:

Conjecture 13 Let d ∈ ℕ be sufficiently large. Let (Q_{1}, Q_{2}) be a pair of real nondegenerate quadratic forms in d variables such that every linear combination αQ_{1} +βQ_{2} with α^{2}+β^{2} ≠ 0 is indefinite, has rank ≥ 3, and does not have all rational coefficients. Then (2) holds.

By a theorem of P. Finsler, for d ≥ 3 every linear combinations αQ_{1} + βQ_{2} with α^{2} + β^{2} ≠ 0 is indefinite iff the intersection of the hypersurfaces {Q_{1} = 0} and {Q_{2} = 0} is not equal to {¯0}. (This theorem was proved independently by many authors (see [Uh79]).) Therefore, under the conditions of Conjecture 13, the set

{x̄: |Q_{i}(x̄)| < ε, i = 1, 2}

is not compact.

A question analogous to this conjecture was studied by Dani and Margulis [DM90]. They showed that if Q is a nondegenerate indefinite quadratic form, and L is a linear form in 3 variables such that the plane {L = 0} is tangent to the hypersurface {Q = 0}, and every linear combination αQ + βL^{2} with α^{2} + β^{2} ≠ 0 does not have all rational coefficients, then {(Q(x̄), L(x̄)): x̄ ∈ ℝ^{d}} is dense in ℝ^{2}. Similar result holds for a pair consisting of a linear form and a quadratic form in d variables, d ≥ 4 [Go].

Some partial results towards Conjecture 13 were obtained in [Co73] by R. J. Cook. He studied pairs of diagonal quadratic forms in d variables, d ≥ 9, with algebraic coefficients.

Let us illustrate the conjecture by several examples:

1. Let ℝ^{d} = V_{1} ⊕ V_{2} be a direct sum of vector spaces V_{1} and V_{2} of dimension at least 3. Let Q_{1} and Q_{2} be indefinite quadratic forms such that Q_{i}|_{V_{i}} is nondegenerate, Q_{i}|_{V_{j}} = 0 for i ≠ j, and V_{1} ⊥ V_{2} with respect to Q_{1} and Q_{2}. According to Conjecture 13, (2) should hold for (Q_{1}, Q_{2}) provided that every nonzero linear combination of Q_{1} and Q_{2} does not have all rational coefficients. This seems to be one of the easiest special cases of the conjecture. It should be possible to attack this case using the original approach of Margulis [Ma89] and Ratner’s topological rigidity [Ra91].

2. Another promising case of Conjecture 13 is when a pair (Q_{1}, Q_{2}) of quadratic forms satisfies the conditions of Theorem 4(a). By Lemma 11, both Q_{1} and Q_{2} are invariant under a nontrivial unipotent subgroup. Hence, one can use Ratner’s topological rigidity.

3. In general, the group of linear transformations that leaves both forms invariant may be finite. If this is the case, the method of Margulis, which is based on dynamics on homogeneous spaces of Lie groups, does not work. This problem can appear in every dimension even when the hypersurfaces {Q_{1} = 0} and {Q_{2} = 0} have a common tangent plane. For example, when

Q_{1}(x̄) = ∑_{i=1}^{d} x_{d−i+1}x_{i}, Q_{2}(x̄) = ∑_{i=1}^{d−1} x_{d−i}x_{i} + αQ_{1}(x̄),

up to a linear change of variables, the group that stabilizes both Q_{1} and Q_{2} is finite.

References

[BD] V. I. Bernik, M. M. Dodson, Metric Diophantine approximation on manifolds, Cambridge Tracts in Mathematics 137, Cambridge University Press, Cambridge, 1999.

[Co73] R. J. Cook, Simultaneous quadratic inequalities, Acta Arith. 25, 337–346, 1973.

[Da85] S. G. Dani, Divergent trajectories of flows on homogeneous spaces and Diophantine approximation, J. Reine Angew. Math. 359, 55–89, 1985.

[Da86] S. G. Dani, Bounded orbits of flows on homogeneous spaces, Comment. Math. Helv. 61, 636-660, 1986.

[Da00] S. G. Dani, On Values of linear and quadratic forms at integer points, Number theory, pp. 107–119, Trends Math., Birkhäuser, Basel, 2000.

[DM90] S. G. Dani, G. A. Margulis, Orbit closures of generic unipotent flows on homogeneous spaces of SL(3, R), Math. Ann. 286, 101–128, 1990.

[Go] A. Gorodnik, Oppenheim conjecture for a pair consisting of a linear form and a quadratic form, to appear in Transactions of AMS.

[Ja31] V. Jarník, Über die simultanen diophantischen Approximationen, Math. Z. 33, 503–543, 1931.

[KM96] D. Kleinbock, G. Margulis, Bounded orbits of nonquasiunipotent flows on homogeneous spaces, Sinaĭ’s Moscow Seminar on Dynamical Systems, 141–172, Amer. Math. Soc. Transl. Ser. 2, 171, Amer. Math. Soc., Providence, RI, 1996.

[Ma89] G. Margulis, Discrete subgroups and ergodic theory, Number theory, trace formulas and discrete groups (Oslo, 1987), 377–398, Academic Press, Boston, MA, 1989.

[Ma97] G. Margulis, Oppenheim conjecture, Fields Medalists’ lectures, pp. 272–327, World Sci. Publishing, River Edge, NJ, 1997.

[OV] A. Onishchik, E. Vinberg, Lie groups and Lie algebras, in Encyclopaedia Math. Sci., Vol. 41, Springer-Verlag, New York – Berlin, 1994.

[Ra91] M. Ratner, Raghunathan topological conjecture and distributions of unipotent flows, Duke Math. J. 63, 235–280, 1991.

[Uh79] F. Uhlig, A recurring theorem about pairs of quadratic forms and extensions: a survey, Linear Algebra Appl. 25, 219–237, 1979.

[Zi] R. Zimmer, Ergodic theory and semisimple groups, Birkhäuser, Basel, 1984.

Department of Mathematics, University of Michigan, Ann Arbor, MI 48109

E-mail: gorodnik@umich.edu