After a goalless first half in which Dean Rittenberg saw a penalty saved, it was the Shakers who grabbed the lead, Richard Hinds heading home a Lewis Young corner kick.
Rovers hit back with 20 minutes to go thanks to a stunning strike from Markus Olsson, but Bury had the final say as Temitope Obadeyi struck in added time to send them through.
With Joshua King getting a runout along with Markus Olsson, Bryan Dabo and Jake Kean, Rovers were the stronger of the two sides early on.
King was seeing plenty of the ball, the Norwegian international dragged a 7th minute shot wide, before he fired straight into the arms of Bury goalkeeper Reice Charles-Cook soon afterwards.
Striker Dean Rittenberg lofted a shot over the bar when through but then missed the chance to give Rovers the lead after he was fouled in the area by Andy Procter.
Rittenberg took the resulting penalty but Charles-Cook saved comfortably.
The visitors ended the half the stronger, Temitope Obadeyi showing good pace and strength to race through but Jake Kean saved well with his legs.
Ten minutes into the second half and it was Bury who grabbed the opening goal. Lewis Young sent over a corner from the right and Richard Hinds rose unmarked to head home.
Obadeyi could have doubled the Shakers' lead with a vicious shot on the half-volley, the ball flashing narrowly over the bar,
On 71 minutes Rovers drew level thanks to a sensational strike from defender Markus Olsson.
The Swede burst down the left flank and with all expecting a cross, he blasted an unstoppable into the far corner of the goal.
Rittenberg saw a shot deflected into the side netting as Rovers pressed for a winner. From the resulting corner kick from Bradley Bauress Bryan Dabo rose highest but his header was well saved by the Bury stopper.
Navid Nasseri fired over as Bury hit Rovers on the break, and a minute into added time they struck a decisive blow.
The lively Temitope Obadeyi advanced from the flank to the corner of the area and he steered the ball beyond Jake Kean into the far corner of the net to put Bury back in front.
In the final throes Dean Rittenberg saw an effort tipped around the post but it was Bury who secured a semi-final tie with Burnley.
BURY: Charles-Cook, Poscha, Obadeyi, Hinds (c), Cameron, Procter, Cooke, Burgess, Young, Grimes, Nasseri (sub Brown 90 mins).
Subs not used: Ciiffe, Dickinson, Barber.
Goals: Harris 56 mins, Obadeyi 90+1 mins
ROVERS: Kean, Thurston, Olsson, Tanner (sub Beesley 49 mins), Lavelle, Lenihan (c), Green, Bauress, Rittenberg, Dabo, King (sub Preston 61 mins).
Subs not used: Raya, Wall, Pero.
Goals: Olsson 71 mins