Richard Carpenter's free kick was headed across the face of goal by Chris Iwelumo for Knight to notch his 24th of the season with a diving header from close range.
It was hard on Colchester, who had enjoyed the better of the first half despite falling behind.
Iwelumo, signed on a free transfer from Stoke, back-headed his second goal in as many games for Brighton when he beat Colchester keeper Simon Brown to a Nathan Jones cross.
Brown made amends after the break with a superb one handed stop from another Iwelumo header to keep the visitors in contention. They equalised on 69 minutes, Rowan Vine nodding back a cross from Craig Fagan for midfielder Kemal Izzet to head in from six yards.
It looked like earning United a deserved point until Knight struck and, to make matters worse, they finished the match with ten men. Defender Wayne Brown, on loan from Watford, was sent off on the stroke of full time for a second bookable offence. He had already been cautioned in the second half for a foul on Knight when he scythed down the match winner again.
Brighton manager McGhee was happier with the result than the performance. "We did not play well and I thought Colchester deserved at least a point," he admitted.
"Chris Iwelumo was the difference for us. We have been missing a big striker who can hold the ball up."Colchester boss Phil Parkinson, who played under McGhee at Reading, took comfort from the fact that Izzet's goal ended an away drought in the league stretching all the way back to their 1-0 win at Wrexham at the end of October.
"That will be a relief to the players," said Parkinson. "That sort of thing plays on people's minds. "I was delighted with the way we played. We passed the ball well and we certainly deserved something out of the game."