Full-back Virgo scored a stunning last minute header with virtually the last touch of extra-time to inspire Brighton to a thrilling penalty shoot-out victory.
Swindon were unlucky to go in to the deciding leg trailing by a goal from Sunday's first match at the County Ground. Andy King's side again dominated during a one-sided match before taking a deserved lead nine minutes from the end of normal time.
Veteran striker Tommy Mooney had the first real opportunity to claw back the deficit when his half-volley was smartly saved at the near post by Brighton goalkeeper Ben Roberts.
Roberts, chasing a club record equalling seventh consecutive clean sheet, reacted quickly to keep out Brian Howard's cross-cum-shot.
Mooney came within inches of ending Swindon's frustrations when he hit the crossbar in the 76th minute.
Former Birmingham and Watford striker Mooney turned smartly on the edge of the penalty area before drilling a right-foot shot against the top of the woodwork.
Monney's strike partner Sam Parkin did finally end Swindon's goal drought five minutes later with his 23rd of the season. Parkin reacted the quickest to stab the ball home at the back post when Sammy Igoe's cross from the right was only half-cleared.
Swindon seemed to have booked their place in Cardiff when substitute Rory Fallon chipped in his seventh goal for the club seven minutes into extra-time. Roberts could only push the New Zealander's first-time shot into the net after Fallon had been picked out by Mooney's pinpoint cross.
But just as the Swindon fans were beginning to celebrate their passage into the final, Virgo popped up at the far post to head in substitute John Piercy's left-wing cross and take the tie into a penalty shoot-out.
Parkin, Matthew Heywood, Richard Carpenter and Chris Iwelumo all successfully converted before Mooney hit his spot-kick to the left, only for former Middlesbrough keeper Roberts to make a great save.
Piercy, Fallon and that man Virgo, all found the net meaning that Andy Gurney had to score to keep Swindon in the Shoot-out.
He struck the outside of Roberts left post to send Brighton into a Millennium Stadium clash against Swindon's west country neighbours Bristol City on May 30.