Lucas Joao struck his first goals for the club but it wasn't enough for all three points at Carrow Road.
After Cameron Jerome had tapped the home side in front on 19, the hosts were reduced to ten men when Mitchell Dijks saw red for a terrible challenge on Marvin Emnes.
Rovers pinned the hosts back for long spells but it wasn't until the mid-point of the second half that the pressure eventually told.
After Joao had slammed a shot against the foot of the post, the on-loan striker poked home a Connor Mahoney centre, before rising high to head in a cross from Graham.
However the Rovers couldn't hold onto their lead with the Canaries grabbing an equaliser on 81. A shot from Jerome taking a huge deflection to find the corner of the net.
The unbeaten start to Tony Mowbray's reign continues, but the frustration is Rovers should have been leaving Norfolk with all three points.
Ahead of the game, boss Tony Mowbray was forced into making three changes, as Charlie Mulgrew and Sam Gallagher missed out with tight hamstrings, whilst Craig Conway didn't travel to Norfolk due to the birth of his daughter on Friday.
That meant a first start since October for defender Tommie Hoban, whilst Danny Graham and Elliott Bennett returned to the starting line-up.
It was a steady start from those in blue and white at a sunny Carrow Road, controlling the early proceedings, enjoying plenty of play down the flanks.
The first chance of note came on nine when good work from Ryan Nyambe on the right saw him find Bennett. The former Canary crossed to the back post to where Liam Feeney attacked the ball but he couldn't keep his header down.
But despite looking untroubled, Rovers found themselves behind on 19 minutes.
Josh Murphy attacked the penalty area from the left, but his low shot slipped under the dive of Jason Steele and that allowed Cameron Jerome to tap into the net from a yard out. A poor goal to concede.
Two minutes later the hosts were reduced to ten men after a poor challenge from Mitchell Dijks. The defender flew into a challenge with Marvin Emnes, catching the Rover in the face with an elbow and scraping a boot down his shin.
Referee Simon Hooper no hesitation in reaching for his top pocket and issuing the Norwich man a straight red card. After a spell of treatment, Emnes was able to continue.
For the remainder of the half Rovers dominated the play but they couldn't make the most of the extra man and get an equaliser.
Bennett drove narrowly wide after he was fed by Feeney, before a clever ball from Emnes to Graham saw him in on goal but he put his effort the wrong side of the post.
Feeney then fired inches over the bar after a free-kick came out to him, before he turned provider with a great cross to Emnes, but the Dutchman couldn't keep his header down.
Connor Mahoney was introduced in place of Nyambe at the start of the second half with Bennett dropping back into defence. But it was the home side with an early chance when Steele saved Jerome's clever header.
Rovers replied with a free-kick from Bennett that fizzed over the bar, before Emnes and Graham exchanged passes in the box but they were just crowded out.
Mowbray wasted little time in making his two final substitutions as Hope Akpan and Lucas Joao were introduced, Guthrie and Feeney the men to make way. Yet it was Norwich with another decent chance as Ivo Pinto struck the outside of the post after a surging run into the box.
After Joao had put a header over the bar, the on-loan striker then went even closer. Meeting a drilled ball across the box from Willams, he fired low and hard but was unlucky to see his effort smack the foot of the post.
The hosts then should have been reduced to ten men when Steven Whittaker felled Mahoney for the second time in a matter of minutes. Carded earlier, the referee ducked issuing the Scot with a second card.
But after knocking on the door for so long, Rovers finally got their reward of an equaliser on 73 minutes.
It was two of the subs that combined to full effect as Mahoney fired the ball into the box and Joao stuck out a leg to divert it into the back of the net for his first Rovers goal.
Within five minutes the striker was celebrating his second of the game. This time Graham crossed from the right and to the back post where Joao lept high to head home.
However Rovers would hold the lead for just three minutes as the home side hit straight back.
Substitute Jacob Murphy drove in from the flank and as his ball found Jerome the striker's shot took a huge deflection off Elliott Bennett and into the corner of the goal.
Joao had a couple of half chances in the closing stages as Rovers pressed for a third goal, but at the other end they were grateful for a late block by Darragh Lenihan as threw himself in front of a Jonny Howson shot.
A point on the road is always welcome, but the frustration is that all three were there for the taking at Carrow Road.
NORWICH: McGovern, Dijks, Martin (c), Howson, Jerome, Hoolahan (sub Wildschut 71 mins), Pritchard (sub Whittaker 25 mins), Bennett, Ivo Pinto, Tettey, Jo. Murphy (sub Ja. Murphy 71 mins).
Subs not used: Ruddy (GK), Bassong, Naismith, Maddison.
Goals: Jerome 19, 81 mins.
Booked: Hoolahan 43 mins, Whittaker 62 mins, Tettey 88 mins.
Off: Dijks 21 mins
ROVERS: Steele, Nyambe (sub Mahoney 46 mins), Lenihan, Hoban, Williams, Bennett, Lowe (c), Guthrie (sub Akpan 54 mins), Feeney (sub Joao 59 mins), Emnes, Graham.
Subs not used: Raya (GK), Ward, Tomlinson, Brown.
Goals: Joao 73, 78 mins.
Booked: Lowe 56 mins, Lenihan 76 mins, Akpan 90+4 mins.
Attendance: 26,224 (including 384 Rovers supporters).