The match turned on a controversial red card after 20 minutes for Garry Richards.
The Gills centre-half was sent off by referee Mike Thorpe for a tackle on Brighton's French midfielder David Martot.
Gillingham were 1-0 up at that stage, captain Andrew Crofts giving them a fourth minute lead with a spectacular volley from 30 yards.
It all went wrong for them though with the dismissal of Richards. From the resulting free-kick Brighton were awarded a penalty for push on Glenn Murray and Nicky Forster levelled with his second spot-kick in as many games.
Brighton capitalised further on their numerical advantage with two goals in the final two minutes of the first half.
Adam El-Abd put them ahead in the 43rd minute with an angled drive after a free-kick by Matt Richards had been pushed onto the upright by Gillingham goalkeeper Derek Stillie.
El-Abd's central defensive partner, Tommy Elphick, sent the home side in with a 3-1 cushion at the break with a header from a Tommy Fraser corner.
The game was over as a contest by that stage and Brighton went further in front midway through the second half from another Fraser corner.
Andrew Whing's shot was saved by Stillie, but substitute Jake Robinson rifled in the rebound.
Gillingham kept persevering and were rewarded with a second goal for Crofts 14 minutes from time.
The Wales international midfielder fired in the loose ball after Brighton goalkeeper Michels Kuipers made a complete hash of punching clear Adam Miller's corner.
Gillingham's misery was compounded with another red card for Sean Clohessy eight minutes from time. He had already been booked for a foul on Forster when he received a second yellow card for kicking the ball away.