Relative trace ideals and Cohen-Macaulay quotients of modular invariant rings

Let \(g\) be a finite group, F a field whose characteristic \(p\) divides the order of \(g\) and \(A^G\) the invariant ring of a finite-dimensional FG-module \(v\). In nalogy to modular representation theory we define for any subgroup \(H\leq G\) the (relative) trace-ideal \(A_H\) \(^G\triangleleft A^G\) to be the image of the relative trace map \(t^G_H : A^H \to A^G, f\mapsto \sum_{g\in [G:H]} g(f)\). Moreover, for any family \(\chi\) of subgroups of \(G\), we define the relative trace ideals \(A^G_{\chi} := \sum_{X\epsilon\chi} A^G_X\triangleleft A^G\) and study their behaviour.

If \(\chi\) consists of proper subgroups of a Sylow \(p\)-group \(P\leq G\) and then \(A\) \(_G\) \(^{\chi}\) is always a proper ideal of \(A^G\);in fact, we show that its height is bounded above by the codimension of the fixed point space \(V^P\). But we also prove that if \(V\) is relatively (\chi\)-projective, then \(A\) \(_G\) \(^{\chi}\) still contains all invariants of degree not divisible by \(p\). If \(V\) is projective then this result applies in particular to the (absolute) trace ideal \(A\) \(_G\) \(^{\lbrace e\rbrace}\).

We also give a 'geometric analysis' of trace ideals, in particular of the ideal \(A^G_{ < P} :=\sum_{Q < P} A^G_{Q^O}\) and show that \(I^{G,P} :=\sqrt{A^G_{< p}}\) is a prime ideal which has the geometric interpretation as 'vanishing ideal' of \(G\)-orbits with length coprime to \(p\). It is shown that \(A^G/I^{G,P}\) is always a Cohen-Macaulay algebra, if the action of \(P\) is defined over the prime field. This generalizes a well known result of Hochster and Eagon for the case \(P = 1\) (see [13]). Moreover we prove that if \(V\) is a direct summand of a permutation module (i.e. a 'trivial source module'), then the \(A\) \(_G\) \(^{\chi}\) are radical ideals and \(A^G_{< p}/I^{G,P}\). Hence in this case the ideal and the corresponding Cohen-Macaulay quotient can be constructed using relative trace maps.