We initiate a study of the rings of invariants of modular representations of elementary abelian $p$-groups. With a few notable exceptions, the modular representation theory of an elementary abelian $p$-group is wild. However, for a given dimension, it is possible to parameterise the representations. We describe parameterisations for modular representations of dimension two and of dimension three. We compute the ring of invariants for all two dimensional representations; these rings are generated by two algebraically independent elements. We compute the ring of invariants of the symmetric square of a two dimensional representation; these rings are hypersurfaces. We compute the ring of invariants for all three dimensional representations of rank at most three; these rings are complete intersections with embedding dimension at most five. We conjecture that the ring of invariants for any three dimensional representation of an elementary abelian $p$-group is a complete intersection.