The Netherlands international swept in an equaliser for the Seagulls after Jean-Philippe Mateta had earlier backheeled Palace in front from their first real attack.
But while Graham Potter's side dominated possession and had 25 shots on goal, a remarkable conclusion saw Christian Benteke volley a dramatic winner for Palace deep into added time with only their second shot on target.
The result sees Palace move up to 13th with Brighton in 16th, just four points above a resurgent Fulham in 18th and at risk of being drawn into a relegation battle.
"It is unbelievable. They are getting away with it," said Veltman. "It feels like a robbery of our points. The chances we didn't finish and they had half a chance and Benteke scored. We have to score more.