We consider moduli spaces of plane quartics marked with various structures such as Cayley octads, Aronhold heptads, Steiner complexes and Göpel subsets and determine their cohomology. This answers a series of questions of Jesse Wolfson. We also count points of these moduli spaces over finite fields of odd characteristic.