With Norwich playing at Arsenal later and Sunderland drawing at Stoke, the Magpies moved into 17th – a point clear of the Black Cats.
Townsend gave Newcastle the lead after the break with a magnificent 20-yard free-kick.
Karl Darlow then came to the rescue when he saved Cabaye’s poor penalty.
They had been written off by pundits and even some of their fans had stopped believing but, on a sunny spring day at St James’ Park, Newcastle provided another twist in this fascinating Premier League season.
Until the second half it had been an error-ridden display by the Magpies, whose midfield were overrun by their Palace counterparts.
However, as they have in recent matches, Newcastle improved tenfold after the break and it was their January signing Townsend who caused the ground to erupt in the 58th minute when he cracked home a stunning free-kick from the right.
It was his fourth in 11 games for his new team – more than the three in 50 he managed for Tottenham.