A contentious decision to award the home side a penalty on the quarter-hour gave Forest an advantage they never relinquished.
Henri Lansbury went down under little pressure from Grant Hanley but the referee pointed to the spot, with the Forest man converting.
It got worse for Rovers Henri Lansbury scored his second from the spot following a clumsy Adam Henley challenge on Jamie Mackie.
Rovers ended the half strongly with Ben Marshall pulling a goal back with a fine effort, and in the second half it was the visitors pressing for an equaliser.
But decisions went against them and Forest hit Rovers on the break twice at the end, Jamie Paterson and then Andy Reid netting to give the hosts all three points.
"We didn't get started, so we'll put our hands up to that, we were a little bit slow," said boss Gary Bowyer.
"But then the game changes on that first goal and I've got to be careful because I don't want to get into trouble, but we've just had a look at it again and if that is a penalty then we should have had two penalties in the second half.
"Chris Taylor has been grabbed round as he is going for a cross, then Grant Hanley gets pulled down with Greg Halford not even looking for the ball. So that changes the game.
"I've no complaints about the second penalty, we should have defended better, Adam knows that.
"Then we come back into the game and we score a great goal with a fantastic finish from Ben Marshall, then in the second half I think it was just us really, we were camped in their half.
"We get caught on the counter with the third goal, again we should have defended better.
"Then we lose Scott with a bad injury which we felt should have been spotted again by the referee. So we had to end the game with ten men, but you wouldn't have known that with the way we played.
"We kept going, had half chances, but they grab a further goal at the end."