Rovers let it slip late on as they had to settle for a 1-1 draw at Pride Park.
Jordan Rhodes had put Eric Black's side in front with a superb 35th minute strike, his fourth for the club.
The visitors looked in control but Derby rallied late and deservedly equalised with just a minute to go when Theo Robinson fired home on the rebound after Paul Robinson had saved Craig Bryson's first-time shot.
The draw sees Rovers keep up their unbeaten record on their travels, but frustration will be the overriding feeling after letting three points slip through their grasp.
Eric Black made three changes from the side that tasted defeat last time out at Wolverhampton Wanderers.
The three were all in midfield, with David Dunn, Dickson Etuhu and Morten Gamst Pedersen all returning to the starting eleven as Danny Murphy, Mauro Formica and Grant Hanley dropped to the bench.
Those changes saw a change formation for the game, Colin Kazim-Richards and Morten Gamst Pedersen operating on the flanks in more of a 4-5-1 as David Dunn supported Jordan Rhodes up top.
The game started at a furious pace with both sides giving no quarter in the opening exchanges. The home side seeing the first goal opportunity on 13 minutes when Connor Sammon crossed from the right but a mis-control from Theo Robinson saw the striker side-foot wide of Paul Robinson's goal.
Rovers responded with an excellent chance of their own as Morten Gamst Pedersen fired in a free-kick from the left, Scott Dann rose to head powerfully goalwards but Adam Legzdins was equal to the defender's efforts with an impressive save.
Play switched back to the other end with Rovers having to clear off the line from a Rams corner, Jake Buxton's header hacked away by a combination of Robinson and Dann.
Legzdins was then called into action once more with a second impressive save of the first half, diving full-length to tip a Morten Gamst Pedersen effort away from goal. The Norwegian's free-kick was arrowing towards the top corner of the net until the Rams' 'keeper's timely intervention.
Roared on by yet another fantastic away following the travelling support were rewarded on 35 minutes with the opening goal of the game.
Colin Kazim-Richards started the move with a ball in from the right, Morten Gamst Pedersen, who had vacated his left-wing position, headed back on the edge of the area into the path of Jordan Rhodes and the young striker took a touch before lashing an unstoppable 20-yard strike into the corner of the goal. Legzdins having no chance this time as Rhodes celebrated his fourth goal in Rovers colours.
Derby threatened with a couple of corner kicks as the half drew to a close but it was the unlikely figure of Bradley Orr who would have two efforts before half-time was up.
A break from a corner kick saw Colin Kazim-Richards spread play out wide to Orr on the right but his effort was scuffed and Craig Bryson. Then on 44 minutes a lovely first-time cross from Martin Olsson was flicked on by Kazim-Richards to where Orr arrived by his low shot was saved.
Rovers were bright at the start of the second half as they went looking to grab that all-important second goal. Barely seconds after the restart Rovers were threatening when Morten Gamst Pedersen sent Martin Olsson away on the left, the Swede powered forward before delivering a low cross into the box that Rhodes couldn't get a clean connection with.
The Rovers goalscorer went close again moments later with a shot from a tight angle, letting fly from the right corner of the area it looked easy for the Derby 'keeper to claim, but he let it bounce on the line before it was grasped.
Derby responded with chances of their own, neat approach play down the left saw the ball arrive at Craig Bryson but he fired over the bar. Then a short corner routine caught Rovers napping, Paul Coutts' shooting onto the roof of the net.
Then it was Rovers' turn to create. Dunn, making just his second start of the season, showing quick feet to trick his way into the area but his near post shot was blocked for a corner. From Pedersen's inswinger Scott Dann rose high to head cleanly but the ball went over the bar.
Disappointingly it was the home side who then went on to dominate the last twenty minutes of the match with the introduction of substitutes Michael Jacobs and Nathan Tyson making the required impact.
Jacobs went close on 83 minutes when his 22-yard free-kick deflected just past the post, then just seconds after his introduction, Tyson was forcing a near-post save from Paul Robinson as the pressure continued to mount.
Aside from an effort from sub' Mauro Formica that flashed just wide the attacking flow was firmly towards Paul Robinson's goal with the pressure eventually telling with just ninety seconds left on the clock.
Jacobs, so impressive on the left after his introduction, fired over a low ball that was hit goalwards first-time by Craig Bryson, Paul Robinson got down superbly to make the save but despite his best efforts he couldn't keep out his namesake Theo who fired home the rebound.
Five minutes of added time was shown but there was to be no further late drama as Rovers maintained their unbeaten start to the season on the road but they'll be left to lament the dropping of two points.
DERBY: Legzdins, Brayford, Keogh, Buxton, O’Connor (sub Tyson 84), Coutts, Hendrick, Bryson, Hughes, Robinson, Sammon (sub Jacobs 75 mins).
Subs not used: Deeney (GK), Freeman, Bennett, Gjokaj, O’Brien.
Goals: Rhodes 35 mins.
ROVERS: Robinson (c), Orr, Dann, Givet, Olsson, Lowe, Etuhu, Dunn (sub Formica 67 mins), Pedersen, Rhodes, Kazim-Richards.
Subs not used: Kean (GK), Murphy, Nuno Gomes, Rochina, Hanley, Ribeiro.
Goals: Robinson 89 mins.