A Jordan Preston strike just before half time had given Rovers the lead at St James Park, but two goals early in the second period saw the Grecians take control of the match to earn a place in the last eight of the competition.
It was a young side that had travelled to the south west for the Tuesday night test, just John O'Sullivan and Paulo Jorge with any first team experience to their name. In contrast, the Exeter City side featured no less than ten from the starting XI having played senior football.
That showed as the home side, who had knocked out Crystal Palace and Southampton in earlier rounds, took control of the game from the off, but up against David Raya in inspired form in the Rovers goal, they couldn't find a way through.
Attacks from Rovers were rare but a well worked move just before the break saw them take the lead against the run of play. Raheem Hanley playing a deep cross into the box to where Jordan Preston added to his impressive goal tally by nodding the ball home.
It was the Scottish Under-19 international's 21st goal of the campaign in all competitions.
Rovers went in at the break in front but mistakes at the back would prove costly as the Grecians turned the game around in the space of five second half minutes.
From an Exeter corner kick an under-hit back pass from Hugo Fernandez allowed Scot Bennett to steal in and fire home.
Moments later and the comeback was complet abd again it was Rovers' making. John O'Sullivan with a poor ball across his own area allowed Exeter to attack, Jimmy Keohane applying the finishing touch.
With their tails up the Grecians went looking for further goals, but Raya enhanced his growing reputation by making several outstanding saves, the Spaniard keeping the scoreline to a more respectable figure.
Academy first year's George Pierce and Sam Lavelle were both introduced as the second half wore on as a younger Rovers looked to get back into the match but Connor Mahoney and Jordan Preston were both thwarted in the closing stages as Exeter clinched a place in the quarter-finals.
Speaking after the match, coach Eric Kinder said: "Fair play to them, they were better than us on the night.
"In the first half we got away with a few moments in our box, but we did okay and Jordan Preston has scored a great goal.
"But everything we talked about at half time we didn't do and we were put under tremendous pressure in the first 20 minutes of the second half and conceded silly goals.
"But for David Raya it could have been more, he made five outstanding saves.
"If we had done things right in the first 15 minutes of the second half then we might have hung on but full credit to Exeter, they have a decent group, they beat Palace, Southampton and now us. They deserved to win tonight."
ROVERS: Raya, Fernandez, R. Hanley, Paulo Jorge (sub Pierce 60 mins), Daly, Tumwa, Mahoney, O'Sullivan, Preston, Payne, Bauress (sub Lavelle 75 mins).
Subs not used: Edinho, Urwin, Pero.