IS THE LUNA STRATIFICATION INTRINSIC?

J. KUTTLER AND Z. REICHSTEIN

Abstract. Let G → GL(V) be a representation of a reductive linear algebraic group G on a finite-dimensional vector space V, defined over an algebraically closed field of characteristic zero. The categorical quotient V // G carries a natural stratification, due to D. Luna. This paper addresses the following questions:

(i) Is the Luna stratification of X intrinsic? That is, does every automorphism of V // G map each stratum to another stratum?

(ii) Are the individual Luna strata in X intrinsic? That is, does every automorphism of V // G maps each stratum to itself?

In general, the Luna stratification is not intrinsic. Nevertheless, we give positive answers to questions (i) and (ii) for interesting families of representations.

1. Introduction

Throughout this paper k will be an algebraically closed field of characteristic zero, G → GL(V) will be a representation of a reductive linear algebraic group G on a finite-dimensional vector space V defined over k, and π: V → X = V // G will denote the categorical quotient map for the G-action on V. For the definition and a discussion of the properties of the categorical quotient in this setting, see, e.g., [MFK], [PV] or [K].

There is a natural stratification on X, due to D. Luna; we shall refer to it as the Luna stratification. Recall that for every p ∈ X the fiber π^{−1}(p) has a unique closed orbit. Choose a point v_{p} in this orbit. Then the stabilizer subgroup Stab(v_{p}), is reductive, and its conjugacy class in G is independent of the choice of v_{p}. This subgroup determines the stratum of p. More precisely, the Luna stratum associated to the conjugacy class (H) of a reductive subgroup H ⊆ G is defined as

X^{(H)} = {p ∈ X | Stab(v_{p}) ∈ (H)} .

There are only finitely many Luna strata and each stratum is a locally closed nonsingular subvariety of X. The strata are naturally partially ordered as follows; S ⪯ T if S is contained in \overline{T}. This partial ordering has a unique maximal element X^{(H)}, called the principal stratum. The subgroup H associated to the principal stratum is called the principal stabilizer (it is defined up to conjugacy). Moreover, if we set

V^{⟨H⟩} = {v ∈ V | G · v is closed and Stab(v) = H}

2000 Mathematics Subject Classification. 14R20, 14L30, 14B05.

Key words and phrases. Categorical quotient, Luna stratification, matrix invariant, representation type.

J. Kuttler was a PIMS Postdoctoral Fellow at UBC.

Z. Reichstein was partially supported by an NSERC research grant.

then π restricts to a principal N_{G}(H)/H-bundle V^{⟨H⟩} → X^{(H)}. For proofs of these assertions see [PV, Section 6.9] or [S, Section I.5].

The Luna stratification provides a systematic approach to the problem of describing the G-orbits in V; it also plays an important role in the study of the geometry and (if k = ℂ) the topology of the categorical quotient X = V // G. In this paper we shall address the following questions.

Question 1.1. (i) Is the Luna stratification of X intrinsic? In other words, is it true that for every automorphism σ: X → X and every reductive subgroup H ⊂ G there is a reductive subgroup H′ ⊂ G such that σ(X^{(H)}) = X^{(H′)}?

(ii) Are the Luna strata in X intrinsic? Here we say that X^{(H)} is intrinsic if σ(X^{(H)}) = X^{(H)} for every automorphism σ: X → X.

In general, the Luna stratification is not intrinsic. Indeed, there are many examples, where V // G is an affine space (cf. e.g., [PV, Section 8]) and the automorphism group of an affine space is highly transitive, so that points in the same stratum can be taken by an automorphism to points in different strata. Moreover, even in those cases where the Luna stratification is intrinsic, the individual strata may not be. The purpose of this paper is to show that one can nevertheless give positive answers to Question 1.1 in many interesting situations.

The first natural case to consider is the one where G is a finite group. Recall that a non-trivial g ∈ GL(V) is called a pseudo-reflection if g has finite order and fixes (pointwise) a hyperplane in V. If G → GL(V) is a representation and G is generated by elements that act as pseudo-reflections on V then by a theorem of Chevalley and Shephard-Todd, V // G is an affine space. As we remarked above, in this case the Luna stratification cannot be intrinsic. To avoid this situation, assume that G contains no pseudo-reflections. In particular, this condition is automatically satisfied for representations G → SL(V) ⊂ GL(V). In this case the proof of [Pr, Theorem 2] can be modified to show that every automorphism of X = V // G lifts to an automorphism of V. From this one easily deduces that if G → GL(V) is a finite-dimensional representation of a finite group G then the Luna stratification on V // G is always intrinsic, and moreover, every stratum is intrinsic under mild additional assumptions on the representation (but not always). For details we refer the reader to [KR, Section 2]. (See also Proposition 3.6 for related results on finite group actions.)

The main focus of this paper will be on representations V of (possibly infinite) reductive groups G. The following theorem gives a positive answer to Question 1.1(i) for three families of such representations.

Theorem 1.2. Let G → GL(W) be a finite-dimensional linear representation of a reductive algebraic group G. Then the Luna stratification in W^{r} // G is intrinsic if

(a) r ≥ 2 dim(W), or

(b) G preserves a nondegenerate quadratic form on W and r ≥ dim(W) + 1, or (c) W = 𝔤 is the adjoint representation of G and r ≥ 3.

Note that in the setting of reductive groups there is no direct analogue of the Chevalley-Shephard-Todd theorem and we do not know under what circumstances an automorphism σ: V // G → V // G can be lifted to V; see [KR, Remark 2.8]. Our proof of Theorem 1.2 relies on a different (indirect) approach based on studying the singularities of the Luna strata; see Section 3. Along the way we will show that if V = W^{r} is as in Theorem 1.2 and S is a Luna stratum in V // G then \overline{S} is singular at every point x ∈ \overline{S} \ S.

Theorem 1.2 concerns representations V of G of a particular form, namely V is assumed to be the rth power of another representation W for sufficiently large r. This is clearly stronger than assuming that G contains no pseudo-reflections. To motivate this condition we remark that a general (and somewhat vague) principle in invariant theory says that replacing a G-variety Z by a power Z^{r} often “improves” the properties of the underlying action, assuming r is sufficiently large. For two unrelated recent results along these lines see [Lo, Corollary, p. 1606] and [Po_{2}]. Theorem 1.2 may be viewed as yet another manifestation of this principle.

Our second main result gives a positive answer to Question 1.1(ii) in the following situation. Consider the natural GL_{n} action on the space V = M_{n}^{r} of r-tuples of n × n-matrices by simultaneous conjugation. The variety X = M_{n}^{r} // GL_{n} has been extensively studied in the context of both invariant and PI theories; an overview of this research area can be found in [F], [Pro] or [DF]. In [Re_{1}, Re_{2}] the second author constructed a large family of automorphisms of X = V // PGL_{n} (for r ≥ n + 1). Every automorphism in that family preserves the Luna strata, so it is natural to conjecture that the same should be true for every automorphism of X. The following result proves this conjecture for any r ≥ 3.

Theorem 1.3. Suppose r ≥ 3. Then every Luna stratum in X = M_{n}^{r} // GL_{n} is intrinsic.

Note that Theorem 1.3 fails if r = 1 or (n, r) = (2, 2); see Remark 8.3.

The fact that the principal Lune stratum X^{({e})} in X = M_{n}^{r} // GL_{n} is intrinsic is an immediate consequence of a theorem of Le Bruyn and Procesi [lBP, Theorem II.3.4], which says that X^{({e})} is precisely the smooth locus of X. This result motivated Theorems 1.2 and 1.3 and served as a starting point of their proofs.

2. Actions of reductive groups

In this preliminary section we collect several well-known definition and results about actions of reductive groups on affine varieties.

2.1. The Luna Slice Theorem. Let G → GL(V) be a linear representation of a reductive group G and v ∈ V be a point with a closed G-orbit. Then by Matsushima’s theorem, the stabilizer H = Stab(v) is a reductive subgroup of G. Consequently, the H-subrepresentation T_{v}(G · v) of the natural representation of H on the tangent space T_{v}(V) has an H-invariant complement. We shall refer to this H-representation as the slice representation and denote it by Slice(v, V). The Luna Slice Theorem asserts that the horizontal maps in the natural diagram

\xymatrix{Slice(v, V) ∗_{H} G \ar[d] \ar[r] & V \ar[d]^{π} \\ Slice(v, V) // H \ar[r] & V // G}

are étale over v and π(v), respectively. In addition the above diagram becomes cartesian after a base change over a neighborhood of π(v). For an arbitrary smooth affine G-variety the same is true if Slice(v, V) is replaced by a suitable H-invariant étale neighborhood of the origin in an H-invariant normal space to T_{v}(Gv). For details, see [Lu] or [PV, Section 6].

2.2. Stability.

Definition 2.1. Let G be a reductive group and V be an affine G-variety. A point v ∈ V is called

stable if its orbit G · v is closed in V and

properly stable if v is stable and Stab_{G}(v) is finite. We shall say that the representation V is

stable if a point v ∈ V in general position is stable,

properly stable if a point v ∈ V in general position is properly stable, generically free if a point v ∈ V in general position has trivial stabilizer.

As an immediate corollary to Luna’s Slice Theorem we obtain the following:

Lemma 2.2. Let G be a reductive group, G → GL(V) be a linear representation, and v ∈ V, H = Stab(v) ⊂ G be as above. Then

(a) the G-representation on V is stable if and only if the H-representation on Slice(v, V) is stable,

(b) the G representation on V is generically free if and only if the if the Hrepresentation on Slice(v, V) is generically free. □

Note that “generically free” is not the same thing as “having trivial principal stabilizer”. The reason is that when we talk about the principal stabilizer, we are only interested in Stab(v), where v is a stable point. For example, the natural action of the multiplicative group 𝔾_{m} on V = 𝔸^{1} is generically free, but the principal stabilizer is all of 𝔾_{m}, because the only stable point in 𝔸^{1} is the origin. The precise relationship between these notions is spelled out in the following lemma.

Lemma 2.3. Let V be a linear representation of a reductive group G and π: V → V // G be the categorical quotient map.

(a) If π(v) ∈ (V // G)^{({e})} then v is a properly stable point in V.

(b) The following conditions are equivalent:

• V has trivial principal stabilizer,

• V is generically free and properly stable,

• V is generically free and stable.

Proof. (a) Let x = π(v) ∈ (V // G)^{({e})}. We claim that π^{−1}(x) is a single Gorbit in V. Indeed, let C = G · v_{0} be the unique closed orbit in π^{−1}(x). Then C is contained in the closure of every orbit in π^{−1}(x). On the other hand, by our assumption Stab(v_{0}) = {e}; hence, dim(C) = dim(G), and C cannot be contained in the closure of any other G-orbit. This shows that C = π^{−1}(x). Thus every point in π^{−1}(x) is stable (and hence, properly stable). This proves part (a). Part (b) is an immediate consequence of part (a). □

2.3. The Hilbert-Mumford criterion. Consider a linear 𝔾_{m}-representation on a vector space V. Any such representation can be diagonalized. That is, there is a basis e_{1}, …, e_{n} of V, so that t ∈ 𝔾_{m} acts on V by t · e_{i} ↦ t^{d_{i}}e_{i}.

In the sequel we shall use the following variant of the Hilbert-Mumford criterion; see [MFK, Section 2.1].

Theorem 2.4. Consider a linear representation of a reductive group G on a vector space V. Then

(a) v ∈ V is properly stable for the action of G if and only if it is properly stable for the action of every 1-dimensional subtorus 𝔾_{m} ↪ G.

(b) In the above notations, v = c_{1}e_{1} + ⋯ + c_{n}e_{n} ∈ V is properly stable for the action of 𝔾_{m} if and only if there exist i, j ∈ {1, …, n} such that d_{i} < 0, d_{j} > 0 and c_{i}, c_{j} ≠ 0. □

We give several simple applications of this theorem below.

Corollary 2.5. Suppose G is a reductive group, G → GL(V) is a linear representation, π_{G}: V → V // G is the categorical quotient map, v ∈ V and, π_{G}(v) ∈ (V // G)^{({e})}.

(a) If H ⊂ G be a reductive subgroup and π_{H}: V → V // G is the categorical quotient map for the induced H-action on V then π_{H}(v) ∈ (V // H)^{({e})}.

(b) If f: V′ → V is a G-equivariant linear map, π′: V′ → V′ // G is the categorical quotient and f(v′) = v then π′(v′) ∈ (V′ // G)^{({e})}.

Proof. Recall that, by definition, π_{G}(v) ∈ V // G^{({e})} if and only if into (i) Stab_{G}(v) = {e} and (ii) v is properly stable for the G-action on V.

(a) We need to show that (i) and (ii) remain valid if G is replaced by H. In case of (i) this is obvious, and in case of (ii), this follows from the Hilbert-Mumford criterion, since every 1-parameter subgroup of H is also a 1-parameter subgroup of G.

(b) Again, we need to check that Stab_{G}(v′) = {e} and v′ is properly stable for the G-action on V′. The former is obvious, and the latter follows from the Hilbert-Mumford criterion. □

2.4. Reductive groups whose connected component is central. Let H be a reductive group whose connected component H^{0} is central. In particular, H^{0} is abelian and hence, a torus. Note that this class of groups includes both tori and all finite groups (in the latter case H^{0} = {1}).

Lemma 2.6. Let H be a reductive group whose connected component H^{0} is central and ρ: H → GL(W) be a linear representation. Then

(a) Stab(w) = Ker(ρ) for w ∈ W in general position.

(b) Suppose H has trivial principal stabilizer in W^{s} for some s ≥ 1. Then H has trivial principal stabilizer in W.

Proof. (a) By [Ri_{1}, Theorem A] the H-action on W has a stabilizer in general position. That is, there is a subgroup S ⊂ H and an open subset U ⊂ W such that Stab(u) is conjugate to S for any u ∈ U. Clearly, Ker(ρ) ⊂ S; we only need to prove the opposite inequality.

Since H^{0} is central in H, S has only finitely many conjugates; denote them by S = S_{1}, …, S_{m}. Then U is contained in the union of finitely many linear subspaces

U ⊂ W^{S_{1}} ∪ ⋯ ∪ W^{S_{m}} .

Since U is irreducible, we see that U is contained in one of them, say, U ⊂ W^{S}. Then V = W^{S}, i.e., S ⊂ Ker(ρ), as claimed.

(b) Let ρ^{s} be the (diagonal) representation of H on W^{s}. Clearly Ker(ρ) = Ker(ρ^{s}) = {e}. Part (a) now tells us that since ρ^{s} is generically free, so is ρ.

By Lemma 2.3 it now suffices to check that the H-action ρ on W is properly stable. Let 𝔾_{m} ↪ H be a 1-dimensional subtorus. Diagonalize it in the basis e_{1}, …, e_{n} of W, so that it acts via

t: e_{i} → t^{d_{i}}e_{i} .

Note that if we diagonalize the 𝔾_{m}-action on W^{s} then the same exponents d_{1}, …, d_{n} will appear but each will be repeated s times. Thus by Theorem 2.4,

W^{s} is properly stable ⟺ d_{i} > 0 and d_{j} < 0 for some i, j ∈ {1, …, n}
⇕

W is properly stable,

and the lemma follows. □

Remark 2.7. Note that both parts of Lemma 2.6 fail if we only assume that H^{0} is a torus (but do not assume that it is central in H). For example, both parts fail for the natural action of the orthogonal group G = O_{2}(k) = 𝔾_{m} ⋊ ℤ/2ℤ on W = k^{2}; cf. [RV, Example 2.5].

2.5. Multiple representations.

Lemma 2.8. Let G be a reductive group and G → GL(W) be a linear representation. Suppose that for some r ≥ 1 an r-tuple w = (w_{1}, …, w_{r}) is chosen so that G · w is closed in W^{r} and that w_{d+1}, …, w_{r} are linear combinations of w_{1}, …, w_{d} for some 1 ≤ d ≤ r. Set H = Stab(w) and v = (w_{1}, …, w_{d}) ∈ W^{d}. Then

(a) Stab_{G}(v) = H.

(b) W^{r} has a G-subrepresentation W_{0} such that w ∈ W_{0} and the natural projection p: W^{r} → W^{d} onto the first d components restricts to an isomorphism between W_{0} and W^{d}.

(c) v has a closed orbit in W^{d}.

(d) Slice(w, W^{r}) ≃ Slice(v, W^{d}) ⊕ W^{r−d}, where ≃ denotes equivalence of Hrepresentations.

Proof. (a) is obvious. To prove (b), suppose w_{j} = ∑_{i=1}^{d} α_{ij}w_{j} for j = d + 1, …, r. Then

W_{0} = {(z_{1}, …, z_{r}) | z_{j} = ∑_{i=1}^{d} α_{ij}z_{j} for j = d + 1, …, r }.

has the desired properties. (c) follows from (b), since G · v is the image of G · w under p.

(d) Since H is reductive, W_{0} has an H-invariant complement W_{1} in W^{r}, so that W^{r} = W_{0} ⊕ W_{1}. Since W_{0} ≃ W^{d}, we conclude that W_{1} ≃ W^{r−d} (as an H-representation). The desired conclusion now follows from the fact that p is an isomorphism between W_{0} and W^{d}. □

Corollary 2.9. Let G be a reductive group and G → GL(W) be a linear representation of dimension n. Then

(a) The following are equivalent: (i) W^{r} is stable for some r ≥ n, (ii) W^{n} is stable, and (iii) W^{s} is stable for every s ≥ n.

(b) The following are equivalent: (i) W^{r} is generically free for some r ≥ n, (ii) W^{n} is generically free, and (iii) W^{s} is generically free for every s ≥ n.

(c) The following are equivalent: (i) W^{r} has trivial principal stabilizer for some r ≥ n, (ii) W^{n} has trivial principal stabilizer, and (iii) W^{s} has trivial principal stabilizer for every s ≥ n.

Proof. Suppose r ≥ n . Then for w = (w_{1}, …, w_{r}) ∈ W^{r} in general position, w_{1}, …, w_{n} span W. Keeping this in mind, we see that

(a) the implication (i) ⇒ (ii) follows from Lemma 2.8(c) (with d = n) and the implication (ii) ⇒ (iii) follows from Lemma 2.8(b) (again, with d = n). (iii) ⇒ (i) is obvious.

(b) follows from Lemma 2.8(a). (c) follows from (a) and (b); see Lemma 2.3(b).

□

3. Proof of Theorem 1.2: the overall strategy

Let G be a reductive algebraic group acting on a smooth affine variety Y and π: Y → X = Y // G be the categorical quotient map for this action. We will say that H is a stabilizer subgroup for Y (or simply a stabilizer subgroup, if the reference to Y is clear from the context), if H = Stab(y) for some stable point y ∈ Y. Clearly H is a stabilizer subgroup if and only if the Luna stratum X^{(H)} associated to its conjugacy class is non-empty.

We will say that a Luna stratum S is singular along its boundary if the singular locus of its closure \overline{S} is precisely \overline{S} \ S. The following lemma is the starting point for our proof of Theorem 1.2.

Lemma 3.1. Suppose every Luna stratum in X = Y // G is singular along its boundary. Then the Luna stratification in X is intrinsic.

Proof. Let σ be an automorphism of X and S be a Luna stratum in X. We will show that σ takes S to another Luna stratum S′ by descending induction with respect to the natural partial order on the (finite) set of Luna strata in X.

By our assumption the principal Luna stratum X^{(e)} is precisely the smooth locus of X_{0}; thus σ(X_{0}) = X_{0}. Now suppose σ(S) is another Luna stratum S′. Then σ takes the smooth locus of \overline{S} \ S to the smooth locus of \overline{S′} \ S′. The irreducible components of the smooth locus of \overline{S} \S are, by our assumption, precisely the Luna strata T which immediately precede S in the partial order. The automorphism σ takes these strata to the irreducible components of the smooth locus of \overline{S′} \ S′, which are the Luna strata T′ immediately preceding S′. Thus σ takes every Luna stratum T immediately preceding S, into another Luna stratum T′. The induction step is now complete. □

Definition 3.2. We shall call a family Λ of finite-dimensional linear representations G → GL(V) of reductive (but not necessarily connected) algebraic groups acceptable if it satisfies the following two conditions.

(i) If G → GL(V) is in Λ then for every stabilizer subgroup H in G, the induced representation N_{G}(H) → GL(V^{H}) is again in Λ.

(ii) For every representation G → GL(V) in Λ, the principal stratum in V // G is singular along its boundary.

Proposition 3.3. Suppose a linear representation G → GL(V) belongs to an acceptable family. Then every Luna stratum in X = V // G is singular along its boundary. In particular, the Luna stratification in X is intrinsic.

Proof. The second assertion follows from the first by Lemma 3.1. Hence, we only need to show that every Luna stratum in X is singular along its boundary. Let π: V → V // G = X be the categorical quotient map, H be a stabilizer subgroup for the G-action on V and S = X^{(H)} be a Luna stratum. Choose p ∈ \overline{S} \ S, say, p ∈ X^{(K)}, where K is a (reductive) stabilizer subgroup and H ⊊ K. Our goal is to show that \overline{S} is singular at p. We will argue by contradiction. Assume, to the contrary, that \overline{S} is smooth at p.

Let N = N_{G}(H) be the normalizer of H in G and write the surjective map π_{| V^{H}}: V^{H} → \overline{S} as a composition

\xymatrix{V^{H} \ar[d]^{π_{N}} \ar[dd]^{π} \\ V^{H} // N \ar[d]^{n} \\ \overline{S},}

where π_{N} is the categorical quotient map for the N-action on V^{H}. Here n is the normalization map for \overline{S}; cf. e.g., [PV, Theorem 6.16].

Let v ∈ V^{H} be an N-stable point with stabilizer K such that π(v) = p and let q = π_{N}(v). (Note that by Luna’s criterion, v is N-stable if and only if it is G-stable; see [PV, Theorem 6.17].) Recall that we are assuming that the Grepresentation on V belongs to an acceptable family Λ. Consequently, the Nrepresentation on V^{H} also belongs to Λ, and thus the smooth locus of V^{H} // N is precisely the principal stratum for the N-action on V^{H}. In other words, if q does not lie in the principal stratum in V^{H} // N then q is a singular point of V^{H} // N. Since n is the normalization map and n(q) = p, this implies that p is a singular point of \overline{S}, a contradiction.

We may thus assume that q lies in the principal stratum U of V^{H} // N. Since we are assuming that p is a smooth point of \overline{S}, the normalization map n is an isomorphism between Zariski open neighborhoods of p and q.

We claim that the action of N/H on V^{H} properly stable and generically free. By our assumption on H, V^{H} contains a G-stable point v with stabilizer H. By Luna’s criterion (see [PV, Theorem 6.17]) v is stable (and hence, properly stable) for the N/H-action on V^{H}. Since N/H-properly stable points form an open subset of V^{H}, we conclude that the N/H-action on V^{H} is properly stable. Moreover, v has a G-invariant Zariski open neighborhood V_{0} ⊂ V such that Stab_{G}(v_{0}) is conjugate to a subgroup of H for any v_{0} ∈ V_{0}; see, e.g., [PV, Theorem 6.3]. Intersecting V_{0} with V^{H}, we see that Stab_{G}(v_{0}) = H for v_{0} ∈ V^{H} in general position. This proves the claim.

The claim implies that π_{N}^{−1}(U) → U is a principal N/H-bundle and consequently, the differential dπ_{v} maps T_{v}(V^{H}) surjectively onto T_{p}(\overline{S}). We will now show that this is impossible. Indeed, since the quotient map π: V → X is Kequivariant (where K acts trivially on X) and v is fixed by K, the differential dπ_{v}: T_{v}(V) → T_{p}(X) is a K-equivariant linear map (where K acts trivially on T_{p}(X)). Consequently, dπ_{v} sends every non-trivial irreducible K-subrepresentation of T_{v}(V) to 0. Since V is smooth, we have (T_{v}(V))^{K} = T_{v}(V^{K}) and therefore dπ_{v} maps T_{v}(V^{K}) onto T_{p}(\overline{S}). On the other hand, since π(V^{K}) = \overline{X^{(K)}}, we conclude that

(1)  T_{p}(\overline{X^{(K)}}) ⊃ T_{p}(\overline{S}) .

Now recall that we are assuming that p is a smooth point of \overline{S}. Moreover, since p ∈ X^{(K)}, it is also a smooth point of \overline{X^{(K)}}. Thus (1) implies dim X^{(K)} ≥ dim S, contradicting the fact that X^{(K)} lies in \overline{S} \ S. □

We now record a corollary of the above argument for future reference.

Corollary 3.4. Suppose G is a reductive group and G → GL(V) is a linear representation with principal isotropy H. Let N = N_{G}(H). Then the natural map

\xymatrix{V^{H} // N \ar[d]^{n} \\ V // G}

is an isomorphism, which identifies the principal Luna stratum in V^{H} // N with the principal Luna stratum in V // G.

Proof. The fact that n is an isomorphism is proved in [LR, Corollary 4.4].

To show that n identifies the principal strata in V^{H} // N and V // G, let π: V → V // G and π_{N}: V^{H} → V // N be the categorical quotient maps. Choose p ∈ V^{H} // N and set q = n(p). Let v ∈ V^{H} be a point in the (unique) closed N-orbit in π_{N}^{−1}(p). By Luna’s criterion, the G-orbit of v is also closed; see [PV, Theorem 6.17]. Our goal is to show that

Stab_{G}(v) = H ⟺ Stab_{N}(v) = H .

The ⇒ direction is obvious, so suppose Stab_{N}(v) = H (i.e., p lies in the principal stratum in V^{H} // N), and let Stab_{G}(v) = K. We want to show that K = H. Assume the contrary: H ⊊ K. Since p lies in the principal stratum in V^{H} // N, dπ_{N} maps T_{v}(V^{H}) surjectively to T_{p}(V^{H} // N). Since n is an isomorphism and π = n ∘ π_{N} (on V^{H}), we see that dπ_{v} maps T_{v}(V^{H}) surjectively onto T_{v}(V // G). On the other hand, in the proof of Proposition 3.3 we showed that this is impossible if H ⊊ K. □

Remark 3.5. Our proof of Theorem 1.2 will be based on showing that each of the families of representations in parts (a), (b) and (c) is acceptable, i.e., satisfies conditions (i) and (ii) of Definition 3.2; the desired conclusion will then follow from Lemma 3.1 and Proposition 3.3. In particular, our argument will also show that for the representations G → GL(V) considered in Theorem 1.2, each Luna stratum in V // G is singular along its boundary.

To illustrate this strategy, we will apply it to the following simple example. We will say that a linear representation G → GL(V) has the codimension 2 property if dim(V^{A}) − dim(V^{B}) ≠ 1 for every pair of subgroups A ◁ B ≤ G (here A is normal in B).

Proposition 3.6. Let Λ be the family of representations ϕ: G → GL(V), where G is finite and ϕ has the codimension 2 property. Then Λ is acceptable. In particular, if ϕ ∈ Λ then

(a) every Luna stratum in V // G is singular along its boundary, and

(b) the Luna stratification in V // G is intrinsic.

Proof. The family Λ clearly satisfies condition (i) of Definition 3.2. To check condition (ii), choose a representation G → GL(V) in Λ. Let X = V // G be the categorical quotient and π: V → X be the quotient map. Choose v ∈ V and set and H = Stab_{G}(v). Our goal is to show that if H ≠ {1} then X is singular at π(v). Assume the contrary. Then by the Luna Slice Theorem, T_{v}(V) // H is smooth at the origin. By the Chevalley-Shephard-Todd theorem, this is only possible if H is generated by elements which act on T_{v}(V) as pseudo-reflections. On the other hand, it is easy to see that if h ∈ H acts on T_{v}(V) as a pseudo-reflection then h also acts on V as a pseudo-reflection. Setting A = {e} and B = ⟨h⟩, we see that the codimension 2 property fails: V^{B} has codimension 1 in V^{A} = V. This contradiction shows that V // G is singular at π(v). Thus Λ is an acceptable family. Assertions (a) and (b) now follow from Proposition 3.3. □

Note that Proposition 3.6(b) is a special case of [KR, Theorem 1.1].

Example 3.7. Every symplectic representation of a finite group has the codimension 2 property.

Indeed, since every symplectic representation is even-dimensional, the assertion of Example 3.7 is an immediate consequence of the following elementary lemma. This lemma will be used again in the sequel.

Lemma 3.8. Let G → GL(W) be a linear representation, leaving invariant a nondegenerate bilinear form b on W. Then for any reductive subgroup H of G the restriction of b to W^{H} is again non-degenerate.

Proof. Let a ∈ W^{H} and suppose that the linear form l_{a}: w ↦ b(a, w) is identically zero on W^{H}. Since H is reductive, W = W^{H} ⊕ W_{1} ⊕ ⋯ ⊕ W_{r}, where each W_{i} is a non-trivial irreducible H-subrepresentation of W. By Schur’s lemma, l_{a} is identically zero on every W_{i}. Hence, l_{a} is identically zero on all of W. Since b is non-degenerate on W, this is only possible if a = 0. □

4. Non-coregular representations

The main difficulty in implementing the strategy outlined in Remark 3.5 is in checking condition (ii) of Definition 3.2. That is, given a linear representation G → GL(V) of a reductive group G on a vector space V and a stable point v ∈ V, we want to show that V // G is singular at π(v). The Luna Slice Theorem reduces this problem to checking that Slice(v, V) // H is singular at π_{H}(0), where H = Stab(v), and π_{H}: Slice(v, V) → Slice(v, V) // H is the categorical quotient map. In other words, we are reduced to a problem of the same type, where v = 0 and G = H.

Linear representation G → GL(V), with the property that V // G is smooth at π(0) are called coregular. Here G is assumed to be reductive and π is the categorical quotient map V → V // G. It is easy to see (cf., e.g., [PV, Proposition 4.11]) that V // G is smooth at π(0) if and only if V // G is smooth everywhere if and only if V // G is isomorphic to an affine space 𝔸^{d} for some d ≥ 1. Coregular representations have been extensively studied; for a survey of this topic and further references, see [PV, Section 8]. Thus in order to implement the strategy for proving Theorem 1.2 outlined in Remark 3.5 we need a large family of representations that are known not to be coregular. The purpose of this section is to prove Proposition 4.1 below, which exhibits such a family.

Proposition 4.1. Let G be a reductive group and V_{1}, V_{2} be linear representations of G, such that V_{1} has trivial principal stabilizer and V_{2} is not fixed pointed (see Definition 4.2 below). Then V_{1} × V_{2} is not coregular. That is, (V_{1} × V_{2}) // G is singular.

We begin with preliminary results about fixed pointed representations, which will be used in the proof and in subsequent applications of Proposition 4.1.

4.1. Fixed pointed representations.

Definition 4.2. Following Bass and Haboush [BH], we will say that a linear representation G → GL(V) of a reductive group G is fixed pointed if the natural G-equivariant projection π: V → V^{G} is the categorical quotient for the G-action on V. (Note that the projection π is sometimes called the Reynolds operator; cf., e.g., [PV, Section 3.4].)

Lemma 4.3. Let G be a reductive group and ρ: G → GL(V) be a linear representation. Then the following conditions are equivalent:

(a) ρ is fixed pointed.

(b) The null cone N C(V) is (scheme-theoretically) a vector space.

(c) The null cone N C(V) is (scheme-theoretically) smooth. If (a), (b) and (c) hold then V = NC(V) ⊕ V^{G}.

Proof. (a) ⇒ (b). Let π: V → V // G be the categorical quotient map. If ρ is fixed pointed then π is a linear projection, so clearly N C(V) = π^{−1}(0) is a vector space and V = NC(V) ⊕ V^{G}.

(b) ⇒ (c) is obvious, since a vector space is smooth.

(c) ⇒ (a). Assume that N C(V) is scheme-theoretically smooth. Since G is reductive, we may write V = V^{G} ⊕ W for some G-invariant subspace W. Then clearly W^{G} = (0), N C(W) = N C(V) and V // G = V^{G} × W // G, where the quotient map π_{V} sends (v, w) ∈ V = V^{G} ⊕ W to (v, π_{W} (w)) ∈ V^{G} × W // G.

Thus, after replacing V by W, we may assume without loss of generality that V^{G} = (0). Our goal is then to show that V is fixed pointed, which, in this case, means that V // G is a single point, or equivalently,

N C(V) = V .

Indeed, assume the contrary. Then N C(V) is cut out by the equations f = 0, as f ranges over the homogeneous elements of k[V]^{G}.

Note that no nonzero homogeneous element f ∈ k[V]^{G} can be of degree 1. Indeed, if there were a non-zero G-invariant linear function f: V → k then V would contain a copy of the trivial representation, contradicting V^{G} = (0). We thus conclude that N C(V) is a subscheme of V cut out by a (possibly empty) collection of homogeneous polynomials of degree ≥ 2. In particular, the tangent space to N C(V) at 0 coincides with all of T_{0}(V). Since we are assuming that N C(V) is (scheme-theoretically) smooth, this is only possible if N C(V) = V. □

In the sequel we will primarily be interested in representations that are not fixed pointed. Two examples are given below.

Example 4.4. No nontrivial stable representation G → GL(V) can be fixed pointed. Indeed, in a fixed-pointed representation, the only stable points are those in V^{G}.

Example 4.5. A non-trivial orthogonal representation H → O_{ϕ}(L) of a reductive group H on a vector space L (preserving a non-degenerate quadratic form ϕ) is not fixed pointed.

Proof. Assume the contrary. Then

(2)  L = N C(L) ⊕ L^{H}

where N C(L) is the null-cone of L. Lemma 3.8, tells us that ϕ restricts to a non-degenerate quadratic form on L^{H}. Hence,

L = (L^{H})^{⊥} ⊕ L^{H} .

Now observe that L^{H} has a unique H-invariant complement in L (namely, the direct sum of all non-trivial H-subrepresentations in L). We thus conclude that N C(L) = (L^{H})^{⊥}. Since ϕ is non-degenerate on L and L^{H}, it is also non-degenerate on (L^{H})^{⊥}. Note that since the H-action on L is non-trivial, L^{H} ≠ L and thus N C(L) = (L^{H})^{⊥} ≠ (0). In particular, the H-invariant regular function L → k given by x → ϕ(x, x) is not constant on (L^{H})^{⊥}. On the other hand, every Hinvariant regular function on N C(L) has to be constant. This contradiction shows that L is not fixed pointed. □

4.2. Proof of Proposition 4.1. Set V = V_{1} × V_{2}. Assume the contrary: V // G is smooth, i.e.. is isomorphic to an affine space 𝔸^{d}. Let π: V → V // G, π_{1} : V_{1} → V_{1} // G, and π_{2} : V_{2} → V_{2} // G be the categorical quotient maps. We will denote the projection V → V_{i} by p_{i} and the induced morphism V // G → V_{i} // G by \overline{p_{i}}.

Let T = 𝔾_{m} × 𝔾_{m} be a two-dimensional torus acting on V = V_{1} × V_{2} by

(s, t): (v_{1}, v_{2}) → (sv_{2}, tv_{2}) .

This action commutes with the G-action on V and hence, descends to V // G. Clearly the T -fixed point π(0, 0) lies in the closure of every other T -orbit in V // G ≃ 𝔸^{d}. Thus by [BH, Corollary 10.6], the T -action on V // G is isomorphic to a linear action. That is, we may assume that V // G is a vector space with a linear action of T. This identifies V_{1} // G and V_{2} // G with the T -invariant linear subspaces (V // G)^{{1}×𝔾_{m}} and (V // G)^{𝔾_{m}×{1}} of V // G respectively. In particular, V_{i} // G is smooth for i = 1, 2. Moreover,

\overline{p_{1}}(\overline{v}) = lim_{t→0}(1, t) · \overline{v}

and

\overline{p_{2}}(\overline{v}) = lim_{s→0}(s, 1) · \overline{v}

are T -equivariant linear projections V // G → V_{1} // G and V // G → V_{2} // G respectively and

\overline{p} = (\overline{p_{1}}, \overline{p_{2}}): V // G → (V_{1} // G) × (V_{2} // G)

is a smooth map.

Consider the commutative diagram

[figure: V; π; V_{1} × V_{2}; π_{1}×π_{2}; V // G; \overline{p}; V_{1} // G × V_{2} // G]

Choose a stable point v_{1} ∈ V_{1} such that π_{1}(v_{1}) lies in the principal stratum of V_{1} // G (V_{1} has a dense open subset consisting of such points; cf. Lemma 2.3) and let x = (π_{1}(v_{1}), π_{2}(0)) ∈ V_{1} // G × V_{2} // G. The (scheme-theoretic) preimage of x under the map π_{1} × π_{2} is clearly G · v_{1} × N C(V_{2}), where N C(V_{2}) is the null cone in V_{2}. Since we are assuming that the G-action on V_{2} is not fixed pointed, N C(V_{2}) is singular; cf. Lemma 4.3. Thus

(π_{1} × π_{2})^{−1}(x) is singular.

On the other hand, as we saw above, the map \overline{p} is smooth. The map π is smooth over the principal Luna stratum in V // G. By Corollary 2.5(b), every point in p^{−1}(x) lies in the principal stratum of V // G. Hence, the composition map π_{1} × π_{2} = \overline{p}π: V → V_{1} // G × V_{2} // G is smooth over some Zariski open neighborhood of x. Consequently,

(π_{1} × π_{2})^{−1}(x) is smooth.

This contradiction shows that V // G cannot be smooth, thus completing the proof of Proposition 4.1. □

Corollary 4.6. Suppose V_{1}, V_{2} and V_{3} are three linear representations of a reductive group G, where V_{1} has trivial principal stabilizer, V_{2} is not fixed pointed, and V_{3} is arbitrary. Then V_{1} ⊕ V_{2} ⊕ V_{3} is not coregular.

Proof. The G-representation V′_{1} = V_{1} ⊕V_{3} has trivial principal stabilizer; see Corollary 2.5(a). Now apply Proposition 4.1 to V′_{1} ⊕ V_{2}. □

5. Proof of Theorem 1.2(a) and (b)

We will follow the strategy outlined in Remark 3.5 by exhibiting acceptable families Λ_{a} and Λ_{b} which include the representations in parts (a) and (b) of Theorem 1.2, respectively.

Elements of Λ_{a} are representations of the form V = W^{r}, where r ≥ 2 dim(W) and G → GL(W) is a representation of a reductive group G.

Elements of Λ_{b} are representations of the form V = W^{r}, where r ≥ dim(W) + 1 and G → O(W) is an orthogonal representation of a reductive group G. That is, G preserves some non-degenerate quadratic form on W.

In view of Proposition 3.3 it suffices to show that Λ_{a} and Λ_{b} are acceptable families. We begin by checking condition (i) of Definition 3.2.

(a) Suppose V = W^{r} is in Λ_{a}. Then V^{H} = (W^{H})^{r} is again in Λ_{a}, because r ≥ 2 dim(W) ≥ 2 dim(W^{H}).

(b) Suppose V = W^{r}, where W is an orthogonal representation of G and r ≥ dim(W) + 1. Once again, V^{H} = (W^{H})^{r}, where r ≥ dim(W) + 1 ≥ dim(W^{H}) + 1.

Moreover, in view of Lemma 3.8 the N_{G}(H)-representation on W^{H} is orthogonal. Thus V^{H} belongs to Λ_{b}, as claimed.

It remains to show that Λ_{a} and Λ_{b} satisfy condition (ii) of Definition 3.2. That is, given a representation V = W^{r} in Λ_{a} or Λ_{b}, we want to show that X = V // G is singular at every point x away from the principal stratum. We begin with two reductions.

First we claim that V may be assumed, without loss of generality, to have trivial principal stabilizer. Indeed, suppose the principal stabilizer in V is H ⊂ G. Set N = N_{G}(H) and \overline{N} = N/H. Then by Corollary 3.4

[figure: V^{H} // \overline{N}; V^{H} // N; n; X]

is an isomorphism which takes the principal stratum in V^{H} // N to the principal stratum in X. Thus it suffices to prove that V^{H} // N is singular away from its principal stratum. As we just showed, the representation N → GL(V^{H}) lies in Λ_{a} in part (a) and in Λ_{b} in part (b). Hence, so does \overline{N} → GL(V^{H}). Since the latter representation has trivial principal stabilizer, this proves the claim.

From now on, we will assume that the principal stabilizer subgroup of G in V = W^{r} (but not necessarily in W) is {e}. Suppose x is represented by an element w = (w_{1}, …, w_{r}) ∈ W^{r} whose G-orbit is closed. Let H = Stab_{G}(w). Note that since x does not lie in the principal stratum in V // G, H ≠ {e}. After permuting the components of W^{r} if necessary, we may assume that w_{n+1}, …, w_{r} are linear combinations of w_{1}, …, w_{n}.

Recall that by the Luna Slice Theorem (W^{r} // G, x) is étale isomorphic to

(Slice(w, W^{r}) // H, π(0)) ;

cf. e.g., [PV, Section 6]. Thus it suffices to prove that Slice(w, W^{r}) // H is singular, i.e., that the H-representation on Slice(w, W^{r}) is not coregular.

Let v = (w_{1}, …, w_{n}) ∈ W^{n}. By Lemma 2.8 Stab(v) = H, G · v is closed in W^{n}, and

Slice(w, W^{r}) ≃ Slice(v, W^{n}) ⊕ W^{r−n} .

Recall that we are assuming that the G-action (and hence, the H-action) on W^{r} has trivial principal stabilizer; cf. Corollary 2.5(a). Since r ≥ n (both in part (a) and in part (b)), Corollary 2.9(c) tells us that the H-action on W^{n} also has trivial principal stabilizer. By Lemma 2.2 this implies that the H-action on Slice(v, W^{n}) has trivial principal stabilizer as well.

We will now consider the families Λ_{a} and Λ_{b} separately.

(a) Suppose V is in Λ_{a}. Recall that we are assuming r ≥ 2n, i.e. r − n ≥ n. Thus by Corollary 2.9(c) W^{r−n} also has trivial principal stabilizer. Consequently, it is not fixed pointed; cf. Example 4.4. By Proposition 4.1 the H-representation

Slice(w, W^{r}) ≃ \underbrace{Slice(v, W^{n})}_{trivial principal stabilizer} ⊕ \underbrace{W^{r−n}}_{not fixed pointed}

is not coregular.

(b) Since G preserves the non-degenerate quadratic form q ⊕⋯⊕q (n−r times) on W^{r−n} and r ≥ n + 1, Example 4.5 tells us that the H-representation on W^{r−n} is not fixed pointed. Proposition 4.1 now tells us that the H-representation

Slice(w, W^{r}) ≃ \underbrace{Slice(v, W^{n})}_{trivial principal stabilizer} ⊕ \underbrace{W^{r−n}}_{not fixed pointed}

is not coregular. □

6. Proof of Theorem 1.2(c)

Once again, we will follow the strategy outlined in Remark 3.5, by defining a suitable family Λ_{c} of representations of reductive groups and then checking that Λ_{c} is acceptable. In the context of Theorem 1.2(c) the natural candidate for Λ_{c} is the family of representations of the form W^{r}, where W = Lie(G) is the adjoint representation for some reductive group G. Unfortunately, this family is not acceptable, because it does not satisfy condition (i) of Definition 3.2. To make the argument go through, we need to consider a slightly larger family which we now proceed to define.

Definition 6.1. Let G be a reductive group. We will say that a linear representation ρ: G → GL(W) is almost adjoint if Ker(ρ) contains a normal closed subgroup K of G such that W is isomorphic to the Lie algebra of G/K and ρ can be written as a composition

(3)  ρ: G → G/K →^{Ad_{G/K}} GL(W) ,

where G → G/K is the natural quotient map and Ad_{G/K} is the adjoint representation. Note that the groups G, K or G/K are assumed to be reductive but not necessarily connected.

We are now ready to define Λ_{c}. Fix an integer r ≥ 3 and let Λ_{c} be the family of representations of the form W^{r}, where G → GL(W) is almost adjoint. Following the strategy of Remark 3.5, in order to prove Theorem 1.2(c), it suffices to check that Λ_{c} is an acceptable family.

We begin by checking condition (i) of Definition 3.2. Suppose ρ: G → GL(W) is an almost adjoint representation, with K◁ G as in (3) and H ⊂ G is a stabilizer subgroup. Since (W^{r})^{H} = (W^{H})^{r}, it suffices to show that the natural representation of the normalizer N = N_{G}(H) on W^{H} is again almost adjoint.

Note that H contains K; since N/K = N_{G/K}(H/K), we may, after replacing G by G/K, assume without loss of generality that K = {e}, i.e., W = Lie(G) is the Lie algebra of G and ρ is the adjoint representation.

In this situation W^{H} = Lie(G)^{H} is a reductive Lie algebra. In fact, it is the Lie algebra of Z = Z_{G}(H), the centralizer of H in G. Note that both N and Z are reductive; cf. [LR, Lemma 1.1]. We claim that the natural representation

ρ: N → GL(Lie(Z))

is almost adjoint. Indeed, let (H, H) be the commutator subgroup of H. Since H acts trivially on Lie(Z), N^{0} = Z^{0}H^{0} (cf. e.g., [LR, p. 488]), and (H, H) ∩ Z is finite, we see that Lie(Z) is also the Lie algebra of \overline{N} = N/(H, H) and ρ descends to the adjoint action of \overline{N} on its Lie algebra. This proves the claim. Condition (i) is now established.

To check condition (ii) of Definition 3.2, we will need the following lemma.

Lemma 6.2. Let G be a reductive (but not necessarily connected) group, with Lie algebra W and let G → GL(W) be the adjoint representation. Then

(a) the G-action on W^{s} is stable for any s ≥ 1,

(b) the action of ρ(G) = G/Z_{G}(G^{0}) on W^{s} has trivial principal stabilizer for any s ≥ 2.

Parts (a) and (b) may be viewed as a variants of [Ri_{2}, Theorem 6.1] and [Ri_{2}, Theorem 4.1], respectively.

Proof. (a) Since R = Rad(G) acts trivially on W, the G-action on W^{s} descends to an action of its semisimple quotient G/R. Thus, by a theorem of Popov [Po_{1}] (cf. also [PV, p. 236]) it suffices to show that Stab_{G/R}(w_{1}, …, w_{s}) is reductive for (w_{1}, …, w_{s}) ∈ W^{s} in general position. Clearly Stab_{G^{0}/R}(w_{1}, …, w_{s}) is reductive if and only if Stab_{G^{0}}(w_{1}, …, w_{s}) is reductive. On the other hand, Stab_{G^{0}}(w_{1}, …, w_{s}) is contained in Stab_{G^{0}}(w_{1}), which is a maximal torus of G^{0}, assuming w_{1} ∈ W is in general position. Since any algebraic subgroup of a torus is reductive, this completes the proof of part (a).

(b) By part (a) the G-action on W^{s} is stable; hence, we only need to check that the ρ(G)-action on W^{s} is generically free. We may assume without loss of generality that s = 2.

Choose w_{1}, w_{2} ∈ W in general position and suppose g ∈ Stab_{G}(w_{1}, w_{2}). We want to show that g ∈ Z_{G}(G^{0}) or equivalently that Ad_{G}(g) = id on W.

Recall that a Lie subalgebra W′ of W is called algebraic if W_{0} is the Lie algebra of a closed subgroup G′ of G. In particular, the subalgebra W^{⟨g⟩} of Ad_{G}(g) in W is algebraic, because it is the Lie algebra of Z_{G^{0}}(⟨g⟩). On the other hand, by [Ri_{2}, Lemma 3.3(b)], the only algebraic Lie subalgebra containing w_{1}, w_{2} ∈ W is W itself (provided that w_{1}, w_{2} ∈ W are in general position). Thus W^{⟨g⟩} = W. Equivalently, Ad_{G}(g) = id on W, and part (b) follows. □

We are now ready to prove that the family of representations Λ_{c} defined at the beginning of this subsection, satisfies condition (ii) of Definition 3.2. More precisely, we want to show that if ρ: G → GL(W) is an almost adjoint representation and r ≥ 3 then W^{r} // G is singular away from its principal stratum.

Suppose K ◁ G is as in Definition 6.1. After replacing G by G/K (this doesn’t change the quotient W^{r} // G or the Luna strata in it), we may assume that W = Lie(G) and ρ is the adjoint representation. It now suffices to prove the following lemma.

Lemma 6.3. Let G be a reductive (but not necessarily connected) group, W be the Lie algebra of G and G → GL(W) be the adjoint representation. Then W^{r} // G is singular away from its principal stratum for any r ≥ 3.

Proof. Let π be the quotient map W^{r} → W^{r} // G, x ∈ W^{r} // G be a point away from the principal stratum and v ∈ W^{r} be a point with closed G-orbit, such that x = π(v). Our goal is to show that W^{r} // G is not smooth at x. As in the proof of Theorem 1.2(a) and (b) in the previous section, we shall do this by showing that the slice representation of H = Stab_{G}(v) on Slice(v, W^{r}) is not coregular. Our strategy will be to express this representation as a direct sum of three H-representations,

Slice(v, W^{r}) = V_{1} ⊕ V_{2} ⊕ V_{3} ,

where V_{1} has trivial principal stabilizer and V_{2} is not fixed pointed, then appeal to Corollary 4.6.

Since we are assuming that x does not lie in the principal stratum, H = Stab_{G}(v) ≠ {e}. As an H-representation, the tangent space T_{v}(G·v) is isomorphic to W/ Lie(H) (recall that here W = Lie(G)). Thus the complement Slice(v, W^{r}) to T_{v}(G · v) in W^{r} can be written as

W^{r−1} ⊕ Lie(H) ⊕ S

for some linear representation S of H. By Lemma 6.2(b), the principal stabilizer for the ρ(G)-action on W ⊕ W is trivial. Hence, the same is true of the H-action on W ⊕ W, since ρ(H) is a reductive subgroup of ρ(G); see Corollary 2.5(a). We will now consider two cases.

Case 1. H acts non-trivially on Lie(H). By Lemma 6.2(a), the H-action on Lie(H) is stable. Since we are assuming that this action is non-trivial, it is not fixed pointed; see Example 4.4. By Corollary 4.6, the H-representation

Slice(v, W^{r}) = \underbrace{W^{r−1}}_{trivial principal stabilizer} ⊕ \underbrace{Lie(H)}_{not fixed pointed} ⊕ S

is not coregular, as desired. (Recall that we are assuming throughout that r ≥ 3.)

Case 2. H acts trivially on Lie(H). Since H is reductive this is only possible if H^{0} is a central torus in H. By Lemma 6.2(b) the G-action (and hence, the H-action) on W^{2} has trivial principal stabilizer. Lemma 2.6 now tells us that the H-action on W also has trivial principal stabilizer. Since H ≠ {e}, no such action can be fixed pointed. Thus by Corollary 4.6

Slice(v, W^{r}) = \underbrace{W}_{trivial principal stabilizer} ⊕ \underbrace{W}_{not fixed pointed} ⊕ (W^{r−3} ⊕ Lie(H) ⊕ S)

is not coregular. □

The above lemma shows that the family Λ_{c} satisfies condition (ii) of Definition 3.2. Thus Λ_{c} is an acceptable family. The proof of Theorem 1.2(c) is now complete. □

Remark 6.4. Lemma 6.3 may be viewed as a variant of a result of Richardson [Ri_{2}, Theorem 8.1], which asserts that Lie(G)^{r} // G is singular away from its principal stratum for any r ≥ 2, if G is connected, semisimple and has no factors of rank 1. These assumptions cannot be made in our setting, because even if we make them for G, they may not remain valid for the quotient of N_{G}(H) which comes up at the next step in the induction process. For this reason we were not able to appeal to [Ri_{2}, Theorem 8.1] directly. However, our proof of Lemma 6.3 is very much in the same spirit.

7. Representation types

Consider the action of the general linear group GL_{n} on the variety

V_{l,n,r} = 𝔸^{lr} × M_{n}^{r}

given by

g · (a_{1}, …, a_{l}, A_{1}, …, A_{r}) ↦ (a_{1}, …, a_{l}, gA_{1}g^{−1}, …, gA_{r}g^{−1})

for any a_{1}, …, a_{l} ∈ k and any A_{1}, …, A_{r} ∈ M_{n}. Let X_{l,n,r} be the quotient variety V_{l,n,r} // GL_{n}. Our goal in the next two sections will be to prove Theorem 1.3 in the following slightly more general from.

Theorem 7.1. Suppose r ≥ 3. Then every Luna stratum in X_{l,n,r} = V_{l,n,r} // GL_{n} is intrinsic.

Of course, the case where l = 0 is of greatest interest to us; in this case Theorem 7.1 reduces to Theorem 1.3. For l ≥ 1 the variety X_{l,n,r} is only marginally more complicated than X_{0,n,r}. Indeed, since GL_{n} acts trivially on 𝔸^{lr}, X_{l,n,r} is isomorphic to 𝔸^{lr} × X_{0,n,r}, and every Luna stratum in X_{l,n,r} is of the form

(4)  S = 𝔸^{lr} × S_{0} ,

where S_{0} is a Luna stratum in X_{0,n,r}. We allow l ≥ 1 in the statement of Theorem 7.1 to facilitate the induction step (on n) in the proof.

The Luna stratification in X_{l,n,r}, has a natural combinatorial interpretation, which we will now recall, in preparation for the proof of Theorem 7.1 in the next section. Let v = (a_{1}, …, a_{l}, A_{1}, …, A_{r}) ∈ V_{l,n,r} be a point in the unique closed GL_{n}-orbit in the fiber over x ∈ X_{l,n,r}. Here each a_{i} ∈ k and each A_{j} ∈ M_{r}. We will view an r-tuple (A_{1}, …, A_{r}) ∈ M_{n}^{r} of n × n-matrices as an n-dimensional representation

ρ: k{x_{1}, …, x_{r}} → M_{n}

of the free associative algebra k{x_{1}, …, x_{r}} on r generators, sending x_{i} to A_{i}. By a theorem of Artin [A, (12.6)], the orbit of v is closed in M_{n}^{r} if and only if ρ is semisimple. (Strictly speaking, Artin’s theorem only covers the case where l = 0; but since V_{l,n,r} = 𝔸^{lr} × V_{0,n,r} and GL_{n} acts trivially on 𝔸^{lr}, the general case is an immediate consequence.) If ρ can be written as ρ_{1}^{e_{1}} ⊕ ⋯ ⊕ ρ_{s}^{e_{s}}, where

ρ_{i}: k{x_{1}, …, x_{r}} → M_{d_{i}}

is an irreducible d_{i}-dimensional representation, we will say that the representation type of x is

(5)  τ = [(d_{1}, e_{1}), …, (d_{r}, e_{r})] .

The square brackets [ ] are meant to indicate that that τ is an unordered collection of pairs (d_{i}, e_{i}); permuting these pairs does not change the representation type. Note also that d_{i}, e_{i} ≥ 1 for every i = 1, …, s. Following Le Bruyn and Procesi [lBP], we shall denote the set of representation types (5) with d_{1}e_{1}+⋯+d_{s}e_{s} = n by RT_{n}. If τ = [(d_{1}, e_{1}), …, (d_{s}, e_{s})] ∈ RT_{n} and μ = [(d′_{1}, e′_{1}), …, (d′_{s′}, e′_{s′})] ∈ RT_{n′} then we will sometimes denote the representation type

[(d_{1}, e_{1}), …, (d_{s}, e_{s}), (d′_{1}, e′_{1}), …, (d′_{s′}, e′_{s′})] ∈ RT_{n+n′}

by [τ, μ].

The Luna strata in X_{l,n,r} = (𝔸^{lr} × M_{n}^{r}) // GL_{n} are in a 1-1 correspondence with RT_{n}; cf., e.g., [lBP, Section 2]. If x ∈ X_{l,n,r} has representation type τ, as in (5), then the associated stabilizer subgroup

(6)  H_{τ} = Stab_{G}(v) ≃ GL_{e_{1}} × ⋯ × GL_{e_{s}} ,

embedded into GL_{n} as follows. Write

k^{n} = V_{1} ⊗ W_{1} ⊕ ⋯ ⊕ V_{s} ⊗ W_{s} ,

where dim(V_{i}) = d_{i} and dim(W_{i}) = e_{i} and let GL_{e_{i}} act on W_{i}. Then H_{τ} = GL_{e_{1}} × ⋯ × GL_{e_{s}} is embedded in GL_{n} via

(g_{1}, …, g_{s}) ↦ (I_{d_{1}} ⊗ g_{1} ⊕ ⋯ ⊕ I_{d_{s}} ⊗ g_{s}) .

where I_{d} denotes the d × d identity matrix; cf. [lBP, Section 2]. For notational convenience we shall denote the Luna strata in X_{l,n,r} = M_{n}^{r} // GL_{n} by X_{n,r}^{τ}, rather than X_{n,r}^{(H_{τ})}. Note that if τ = [(d_{1}, e_{1}), …, (d_{s}, e_{s})] ∈ RT_{n} then

(7)  dim X_{l,n,r}^{τ} = (r − 1)(d_{1}^{2} + ⋯ + d_{s}^{2}) + s + lr

for any r ≥ 2; cf. [lBP, p. 158].

Definition 7.2. An elementary refinement of τ = [(d_{1}, e_{1}), …, (d_{s}, e_{s})] ∈ RT_{n} consists in either

(1) replacing one of the pairs (d_{i}, e_{i}) by two pairs, (a_{i}, e_{i}) and (b_{i}, e_{i}), where a_{i}, b_{i} ≥ 1 and a_{i} + b_{i} = d_{i}

or

(2) replacing two pairs (d_{i}, e_{i}) and (d_{j}, e_{j}), with d_{i} = d_{j}, by the single pair (d_{i}, e_{i} + e_{j}).

Given two representation types τ and τ′, we will say that τ′ ≺ τ if τ′ can be obtained from τ by a sequence of elementary refinements. This defines a partial order ⪯ on RT_{n}.

Note that while operations (1) and (2) are defined in purely combinatorial terms, they are, informally speaking, designed to reflect the two ways a representation

ρ = ρ_{1}^{e_{1}} … ρ_{s}^{e_{s}}: k{x_{1}, …, x_{r}} → M_{n}

can “degenerate”. Here ρ_{1}, …, ρ_{s} are distinct irreducible representations of dimensions d_{1}, …, d_{s} respectively. In case (1), one of the representations ρ_{i} “degenerates” into a direct sum of irreducible subrepresentations of degree a_{i} and b_{i} (each with multiplicity e_{i}). In case (2) we “degenerate” ρ by making ρ_{i} and ρ_{j} isomorphic (of course, this is only possible if their dimensions d_{i} and d_{j} are the same). The following lemma gives this a precise meaning.

Lemma 7.3. X_{l,n,r}^{μ} lies in the closure of X_{l,n,r}^{τ} if and only if μ ⪯ τ.

Proof. In view of (4), we may assume l = 0. In this case Lemma 7.3 is proved in [lBP, Theorem II.1.1]. □

Remark 7.4. Note that V_{l,n,r} = W^{r}, where W = 𝔸^{l} × M_{n} is the Lie algebra of G = (GL_{1})^{l} × GL_{n}. The (GL_{1})^{l} factor acts trivially on W (via the adjoint action), so we may drop it without changing the quotient W^{r} // G or the Luna strata in it. Theorem 1.2(c) now tells us that the Luna stratification in X_{l,n,r} = V_{l,n,r} // GL_{n} = W^{r} // G is intrinsic (provided r ≥ 3). Thus an automorphism σ of X_{l,n,r} induces an automorphism σ^{∗} of the set RT_{n} of Luna strata in X_{l,n,r} given by σ^{∗}(τ) = ν if σ(X^{τ}) = X^{ν}. By Lemma 7.3 σ^{∗} respects the partial order ⪯ on RT_{n}. Theorem 7.1 asserts that σ^{∗} is always trivial. In some cases this can be deduced from the fact that the partially ordered set (RT_{n}, ⪯) has no non-trivial automorphisms. For example, the partially ordered sets RT_{n}, for n = 1, 2 and 3, pictured below have no non-trivial automorphisms.

\begin{matrix}RT & 1 \\ [(1, 1)] \end{matrix}  \begin{matrix}RT & 2 \\ [(2, 1)] \\ [(1, 1), (1, 1)] \\ [(1, 2)] \end{matrix}  \begin{matrix}RT & 3 \\ [(3, 1)] \\ [(2, 1), (1, 1)] \\ [(1, 1), (1, 1), (1, 1)] \\ [(1, 2), (2, 1)] \\ [(1, 3)] \end{matrix}

This proves Theorem 7.1 for n ≤ 3.

Note however, that for larger n the partially ordered set (RT_{n}, ⪯) does have non-trivial automorphisms. For example, a quick look at RT_{4} (pictured on p. 156 in [lBP]), shows that the permutation α of RT_{4} interchanging

τ = [(1, 1), (1, 1), (1, 1), (1, 1)] and ν = [(2, 1), (1, 2)]

and fixing every other element, does, indeed, respect the partial order on RT_{4}. For this reason we cannot hope to prove Theorem 7.1 by purely combinatorial arguments, without taking into account the geometry of the strata X_{l,n,r}^{τ}. Nevertheless, the following combinatorial proposition will play a key role in the proof of Theorem 7.1 in the next section.

Proposition 7.5. (a) Let α be an automorphism of RT_{n} (as a partially ordered set). If α([(1, 1), μ]) = [(1, 1), μ] for every μ ∈ RT_{n−1} then α = id.

(b) Let l ≥ 0, n ≥ 1 and r ≥ 3. Suppose we know that Luna strata of the form X_{l,n,r}^{[(1,1),μ]} are intrinsic in X_{l,n,r} for any μ ∈ RT_{n−1}. Then every Luna stratum in X_{l,n,r} is intrinsic.

Proof. Part (b) is an immediate consequence of part (a) and Remark 7.4; we shall thus concentrate on proving part (a). Given a representation type τ = [(d_{1}, e_{1}), …, (d_{s}, e_{s})], let m(τ) denote the minimal value of d_{i} + e_{i}, as i ranges from 1 to s. Note that since d_{i}, e_{i} ≥ 1 for each i, we have m(τ) ≥ 2. We will now show that α(τ) = τ by induction on m(τ). By our assumption this is the case if m(τ) = 2, since in this case d_{i} = e_{i} = 1 for some i.

For the induction step, assume that m(τ) = m ≥ 3 and α(ν) = ν for every ν ∈ RT_{n} with m(ν) < m. After renumbering the pairs (d_{i}, e_{i}), we may assume that d_{1} + e_{1} = m. Suppose

(8)  α(τ) = τ′ = [(d′_{1}, e′_{1}), …, (d′_{t}, e′_{t})] .

Our goal is to show that τ = τ′. We will consider two cases, where d_{1} ≥ 2 and e_{1} ≥ 2 separately. Since we are assuming that d_{1} + e_{1} = m ≥ 3, these two cases cover every possibility.

Case 1: d_{1} ≥ 2. Let

τ_{0} = [(1, e_{1}), (d_{1} − 1, e_{1}), (d_{2}, e_{2}), …, (d_{s}, e_{s})] .

Since m(τ_{0}) = e_{1} + 1 < d_{1} + e_{1} = m, the induction assumption tells us that α(τ_{0}) = τ_{0}. Now observe that τ_{0} immediately precedes τ in the partial order on RT_{n}, i.e., τ_{0} is obtained from τ by a single elementary refinement; see Definition 7.2. Consequently, τ_{0} can also be obtained from τ′ by a single elementary refinement. Schematically,

\xymatrix{τ \ar@{-->}[dr] \ar[rr]^{α} &  & τ′ \ar@{-->}[dl] \\  & τ_{0} ,}

where the broken arrows denote elementary refinements. Now observe that

(d′_{i}, e′_{i}) ≠ (1, e_{1}) or (d_{1} − 1, e_{1}) ∀ i = 1, …, s .

Indeed, otherwise we would have m(τ′) < m and thus α(τ′) = τ′ by the induction assumption. Combining this with (8), we obtain α(τ) = τ′ = α(τ′). Since α is a permutation of RT_{n}, we conclude that τ = τ′, which is impossible, since m(τ′) < m = m(τ).

To sum up, τ_{0} “contains” two pairs, (1, e_{1}) and (d_{1}−1, e_{1}), that are not “present” in τ′. It now follows from Definition 7.2 that the only possible elementary refinement taking τ′ to τ_{0} is of type (1), consisting of ”splitting up” (d_{1}, 1) into (d_{1} − 1, 1) and (1, 1). That is, τ′ = [(d_{1}, 1), μ] = τ, as claimed.

Case 2: e_{1} ≥ 2. The argument here is very similar (or more precisely, “dual”; in the sense of Remark 7.6) to the argument in Case 1. Let

τ_{1} = [(d_{1}, 1), (d_{1}, e_{1} − 1), (d_{2}, e_{2}), …, (d_{s}, e_{s})] .

Since m(τ_{1}) = d_{1} + 1 < d_{1} + e_{1} = m, the induction assumption tells us that α(τ_{1}) = τ_{1}. The relationship between τ, τ′ and τ_{1} is shown in the following diagram

\xymatrix{ & τ_{1} \ar@{-->}[dr] \ar@{-->}[dl] \\ τ \ar[rr]_{α} &  & τ′ ,}

where the broken arrows denote elementary refinements. Once again, we see that τ_{1} “contains” two pairs, (d_{1}, 1) and (d_{1}, e_{1} − 1) both of which “disappear” after we perform an elementary refinement (and obtain τ′). This is only possible if the elementary refinement taking τ_{1} to τ′ is of type (2) (cf. Definition 7.2) and consists of replacing (d_{1}, 1) and (d_{1}, e_{1} − 1) by (d_{1}, e_{1}). This shows that τ = τ′, thus completing the proof of Proposition 7.5. □

Remark 7.6. The two elementary refinement operations of Definition 7.2 are dual to each other in the following sense. Given a representation type

τ = [(d_{1}, e_{1}), …, (d_{s}, e_{s})]

let \overline{τ} = [(e_{1}, d_{1}), …, (e_{s}, d_{s})]. Then α is obtained from β by an elementary refinement of type (1) (respectively, of type (2)) if and only if \overline{β} is obtained from \overline{α} by an elementary refinement of type (2) (respectively, of type (1)). Consequently, the map τ ↦ \overline{τ} is an isomorphism between the partially ordered sets (RT_{n}, ⪰) and (RT_{n}, ⪯). The statement and the proof of Proposition 7.5 are invariant with respect to this map (in particular, Case 2 is dual to Case 1).

8. Proof of Theorem 1.3

In this section we will prove Theorem 7.1. This will immediately yield Theorem 1.3 (for l = 0). We will continue to use the notations introduced in the previous section; in particular, V_{l,n,r} stands for 𝔸^{lr} × M_{n} and X_{l,n,r} denotes the categorical quotient V_{l,n,r} // GL_{n}.

We will argue by induction on n. The base cases, n = 1 and 2, are proved in Remark 7.4. (Theorem 7.1 is also proved there for n = 3 but we shall not need that here.) For the induction step assume n ≥ 3 and σ is an automorphism of X_{l,n,r}. Recall that σ maps each stratum in X_{l,n,r} to another stratum; cf. Remark 7.4. In particular, σ preserves the maximal (principal) stratum X_{l,n,r}^{[(n,1)]} (which is the unique stratum of maximal dimension) and permutes the ”submaximal” strata X_{l,n,r}^{[(d,1),(n−d,1)]}, 1 ≤ d ≤ \frac{n}{2}, among themselves. By the dimension formula (7),

dim X^{[(d,1),(n−d,1)]} = rl+2+(r−1)(d^{2} +(n−d)^{2}) = rl+2+2(r−1)((d− \frac{n}{2})^{2} + \frac{n^{2}}{4} ) .

Thus the submaximal strata X_{l,n,r}^{[(d,1),(n−d,1)]} have different dimensions for different values of d between 1 and \frac{n}{2}. Hence, σ preserves each one of them.

Of particular interest to us is the submaximal stratum X_{l,n,r}^{[(1,1),(n−1,1)]}. Since σ

preserves this stratum, it preserves its closure \overline{X_{l,n,r}^{[(1,1),(n−1,1)]}} and thus lifts to an

automorphism σ̃ of the normalization of \overline{X_{l,n,r}^{[(1,1),(n−1,1)]}}. The rest of the argument

will proceed as follows. We will identify the normalization of \overline{X_{l,n,r}^{[(1,1),(n−1,1)]}} with X_{l+1,n−1,r} and relate Luna strata in X_{l+1,n−1,r} and X_{l,n,r} via the normalization map. By the induction assumption σ̃ preserves every Luna stratum in X_{l+1,n−1,r}; using the normalization map we will be able to conclude that σ preserves certain Luna strata in X_{l.n,r}. Proposition 7.5 will then tell us that, in fact, σ preserves every Luna stratum in X_{l.n,r}, thus completing the proof.

We now proceed to fill in the details of this outline. First will explicitly describe the normalization map for \overline{X_{l,n,r}^{[(1,1),(n−1,1)]}}. The stabilizer H_{τ}, corresponding to τ = [(1, 1), (n − 1, 1)] consists of diagonal n × n matrices of the form

diag(a, \underbrace{b, …, b}_{n − 1 times}) ,

where a, b ∈ k^{∗}; cf. (6). The natural projection

V_{l,n,r}^{H_{τ}} // N_{G}(H_{τ}) → \overline{X_{l,n,r}^{τ}}

is the normalization map for \overline{X_{l,n,r}^{τ}}; cf. [PV, Theorem 6.16]. Here

V_{l,n,r}^{H_{τ}} = 𝔸^{lr} × M_{1}^{r} × M_{n−1}^{r} = 𝔸^{l(r+1)} × M_{n−1}^{r} = V_{l+r,n−1,r}

and

(9)  N_{G}(H_{τ}) = GL_{1} × GL_{n−1} ,

where GL_{1} acts trivially on 𝔸^{l(r+1)} ×M_{n−1}^{r} and GL_{n−1} acts on the second factor by simultaneous conjugation. Since GL_{1} acts trivially, we may replace GL_{1} × GL_{n−1} by GL_{n−1} without changing the categorical quotient or the Luna strata in it. That

is, the normalization V_{l,n,r}^{H_{τ}} // N_{G}(H_{τ}) of \overline{X_{l,n,r}^{[(1,1),(n−1,1)]}} is (canonically) isomorphic to V_{l+1,n−1,r} // GL_{n−1} = X_{l+1,n−1,r}.

The following lemma gives a summary of this construction. Here, as before, we identify an r-tuple A = (A_{1}, …, A_{r}) of n × n-matrices with the n-dimensional representation ρ_{A}: k{x_{1}, …, x_{r}} → M_{n} of the free associative k-algebra k{x_{1}, …, x_{n}}, taking x_{i} to A_{i}.

Lemma 8.1. Suppose n ≥ 3 and r ≥ 2 and let f: V_{l+1,n−1,r} → V_{l,n,r} be the morphism given by

f: (t_{1}, …, t_{lr}, ρ) ↦ (t_{1}, …, t_{l(r−1)}, ρ_{t} ⊕ ρ)) ,

where t = (t_{l(r−1)+1}, t_{l(r−1)+2}, …, t_{lr}) ∈ M_{1}^{r}. Then (a) f descends to the normalization map

\overline{f}: X_{l+1,n−1,r} → \overline{X_{l,n,r}^{[(1,1),(n−1,1)]}} ,

where X_{l,n,r} = V_{l,n,r} // GL_{n}.

(b) \overline{f} maps \overline{X_{l+1,n−1,r}^{μ}} onto \overline{X_{l,n,r}^{[(1,1),μ]}} for every μ ∈ RT_{n−1}.

Proof. Part (a) follows from the discussion before the statement of the lemma. To prove part (b), observe that every semisimple representation k{x_{1}, …, x_{r}} → M_{n} of type [(1, 1), μ] can be written in the form ρ_{0} ⊕ ρ, where ρ is an n − 1-dimensional representation of type μ and ρ_{0} is a 1-dimensional representation (of type (1, 1)). This shows that the image of X_{l+1,n−1,r}^{μ} contains X_{l,n,r}^{[(1,1),μ]}. Since

dim X_{l+1,n−1,r}^{μ} = dim X_{l,n,r}^{[(1,1),μ]} ,

see (7), and \overline{f} is a finite map (in particular, \overline{f} takes closed sets to closed sets), part (b) follows. □

Remark 8.2. Lemma 8.1 uses, in a crucial way, the assumption that n ≥ 3. If n = 2 then (9) fails; instead we have N_{G}(H_{τ}) = (GL_{1} × GL_{1}) {> \atop ◁} S_{2}, and the entire argument falls apart.

We are now ready to finish the proof of Theorem 7.1. Restricting σ to the closure of the stratum X_{l,n,r}^{[(1,1),(n−1,1)]} and lifting it to an automorphism σ̃ of the normalization, we obtain the following commutative diagram:

\xymatrix{X_{l+1,n−1,r} \ar[r]^{σ̃} \ar[d]_{\overline{f}} & X_{l+1,n−1,r} \ar[d]^{\overline{f}} \\ \overline{X_{l,n,r}^{[(1,1),(n−1,1)]}} \ar[r]^{σ} & \overline{X_{l,n,r}^{[(1,1),(n−1,1)]}} .}

By our induction assumption σ̃ preserves every Luna stratum X_{l+1,n−1,r}^{μ} in X_{l+1,n−1,r}. Hence, by Lemma 8.1(b), σ preserves the closure of every Luna stratum in X_{l,n,r} of the form X_{l,n,r}^{[(1,1),μ]}. Since X_{l,n,r}^{[(1,1),μ]} is the unique stratum of maximal dimension

in its closure, we conclude that σ preserves X_{l,n,r}^{[(1,1),μ]} for every μ ∈ RT_{n−1}. By Propostion 7.5(b), we conclude that σ preserves every Luna stratum in X_{l,n,r}. This concludes the proof of Theorem 7.1 and thus of Theorem 1.3. □

Remark 8.3. Theorem 1.3 fails if (a) r = 1 or (b) (n, r) = (2, 2), because in this case the ring of invariants R = k[M_{n}^{r}]^{GL^{n}} is a polynomial ring or equivalently, X = M_{n}^{r} // GL_{n} is an affine space. In case (a) R is freely generated by the coefficients of the characteristic polynomial of A ∈ M_{n}, viewed as GL_{n}-invariant polynomials M_{n}^{1} → k and in case (b) by the five GL_{2}-invariants M_{n}^{2} → k given by (A_{1}, A_{2}) ↦ tr(A_{1}), tr(A_{2}), det(A_{1}), det(A_{2}), and det(A_{1} + A_{2}), respectively; cf. [GY, VIII, Section 136].

Acknowledgements

We would like to thank the referee for a thorough reading of the paper and a number of constructive suggestions. We are also grateful to J.-L. Colliot-Thélène, Ph. Gille, M. Thaddeus and I. Zakharevich for helpful comments and to the Pacific Institute for the Mathematical Sciences (PIMS) for making this collaboration possible.

References

[A] M. Artin, On Azumaya algebras and finite dimensional representations of rings, J. Algebra 11 (1969), 532–563.

[BH] H. Bass, W. Haboush, Linearizing certain reductive group actions, Trans. Amer. Math. Soc. 292 (1985), no. 2, 463–482.

[B] A. Borel, Linear Algebraic Groups, second edition. Graduate Texts in Mathematics, 126, Springer-Verlag, New York, 1991.

[CS] J.-L. Colliot-Thélène and J.-J. Sansuc, Fibrés quadratiques et composantes connexes réelles, Math. Ann. 244 (1979), no. 2, 105–134.

[DF] V. Drensky, E. Formanek, Polynomial identity rings. Advanced Courses in Mathematics, CRM Barcelona, Birkhäuser Verlag, Basel, 2004.

[F] E. Formanek, The polynomial identities and invariants of n×n matrices. CBMS Regional Conference Series in Mathematics, 78, American Mathematical Society, Providence, RI, 1991.

[GY] J. H. Grace, A. Young, The Algebra of Invariants, Cambridge University Press, 1903.

[K] H. Kraft, Geometrische Methoden in der Invariantentheorie, Aspects of Mathematics, D1, Friedr. Vieweg & Sohn, Braunschweig, 1984.

[KR] J. Kuttler, Z. Reichstein, Is the Luna stratification intrinsic?, math.AG/0610669.

[lBP] L. Le Bruyn, C. Procesi, Étale local structure of matrix invariants and concomitants, in Algebraic groups Utrecht 1986, Lecture Notes in Math., 1271 (1987), 143–175.

[lBR] L. Le Bruyn, Z. Reichstein, Smoothness in algebraic geography, Proc. London Math. Soc. (3) 79 (1999), no. 1, 158–190.

[Lo] M. Lorenz, On the Cohen-Macaulay property of multiplicative invariants, Trans. Amer. Math. Soc. 358 (2006), no. 4, 1605–1617

[Lu] D. Luna, Slices étales, in Sur les groupes algébriques, 81–105. Bull. Soc. Math. France, Paris, Mémoire 33, Soc. Math. France, Paris.

[LR] D. Luna, R. W. Richardson, A generalization of the Chevalley restriction theorem, Duke Math. J. 46 (1979), no. 3, 487–496.

[MFK] D. Mumford, J. Fogarty, F. Kirwan, Geometric invariant theory. Third edition, Springer-Verlag, Berlin, 1994.

[Mum] D. Mumford, The red book of varieties and schemes, Lecture Notes in Mathematics, 1358, Springer-Verlag, Berlin, 1988.

[Po_{1}] V. L. Popov, Criteria for the stability of the action of a semisimple group on a factorial manifold (Russian) Izv. Akad. Nauk SSSR Ser. Mat. 34 (1970), 523–531; English transl.: Math. USSR-Izv. 4 (1971), 527–535.

[Po_{2}] V. L. Popov, Generically multiple transitive algebraic group actions, Proceedings of the International Colloquium on Algebraic Groups and Homogeneous Spaces (January 2004), TIFR, Mumbai, India, to appear. Preprint available at www.arxiv.org/math.AG/0409024

[PV] V. L. Popov and E. B. Vinberg, Invariant Theory, Algebraic Geometry IV, Encyclopedia of Mathematical Sciences 55, Springer, 1994, 123–284.

[Pr] D. Prill, Local classification of quotients of complex manifolds by discontinuous groups, Duke Math. J. 34 (1967), 375–386

[Pro] C. Procesi, The invariant theory of n × n matrices, Advances in Math. 19 (1976), no. 3, 306–381.

[Re_{1}] Z. Reichstein, On automorphisms of matrix invariants, Trans. Amer. Math. Soc. 340

(1993), no. 1, 353–371.

[Re_{2}] Z. Reichstein, On automorphisms of matrix invariants induced from the trace ring, Lin-

ear Algebra Appl. 193 (1993), 51–74.

[RV] Z. Reichstein, N. Vonessen, Group actions on central simple algebras: a geometric approach, J. Algebra 304 (2006), no. 2, 1160–1192.

[Ri_{1}] R. W. Richardson, Jr., Principal orbit types for algebraic transformation spaces in char-

acteristic zero, Invent. Math. 16 (1972), 6–14.

[Ri_{2}] R. W. Richardsion, Jr., Conjugacy classes of n-tuples in Lie algebras and algebraic

groups, Duke Math J. 57 (1988) no. 1, 1–35.

[S] G. W. Schwarz, Lifting smooth homotopies of orbit spaces, Inst. Hautes Études Sci. Publ. Math. 51, (1980), 37–135.

Department of Mathematics, University of British Columbia, Vancouver, BC V6T 1Z2, Canada, Current address: Department of Mathematical and Statistical Sciences, University of Alberta, Edmonton, AB T6G 2G1, Canada

   E-mail address: jochen.kuttler@ualberta.ca

Department of Mathematics, University of British Columbia, Vancouver, BC V6T 1Z2, Canada

   E-mail address: reichst@math.ubc.ca