Bulletin of the
Korean Mathematical Society BKMS

Some modular representations of reflection groups related to Weyl groups are considered. The rational cohomology of the classifying space of a compact connected Lie group $G$ with a maximal torus $T$ is expressed as the ring of invariants, $H^*(BG; \Q)\cong H^*(BT; \Q)^{W(G)}$, which is a polynomial ring. If such Lie groups are locally isomorphic, the rational representations of their Weyl groups are equivalent. However, the integral representations need not be equivalent. Under the mod $p$ reductions, we consider the structure of the rings, particularly for the Weyl group of symplectic groups $Sp(n)$ and for the alternating groups $A_n$ as the subgroup of $W(SU(n))$. We will ask if such rings of invariants are polynomial rings, and if each of them can be realized as the mod $p$ cohomology of a space. For $n=3, 4$, the rings under a conjugate of $W(Sp(n))$ are shown to be polynomial, and for $n=6, 8$, they are non--polynomial. The structures of $H^*(BT^{n-1}; \F_p)^{A_n}$ will be also discussed for $n=3, 4$.

Keywords: Invariant theory, unstable algebra, pseudo--reflection group, Poincar\'e series, Lie group, $p$--compact group, classifying space

MSC numbers: Primary 55R35; Secondary 13A50, 55P60