David Lowe's side surged into a two-goal lead thanks to goals from Jordan Preston and Dean Rittenberg, but strikes either side of the break saw the Foxes draw level.
Rovers edged in front once more thanks to a second Preston goal, but the visitors equalised a minute from time to deny them victory.
The home side were strong from the off, taking a deserved lead after 19 minutes.
Neat passing play saw the ball moved wide right and Ian Vicars centred for Jordan Preston to head home.
Chances came and went for the young Rovers before they added a second just past the half hour.
Smart play in and around the area saw the ball reach Dean Rittenberg and he finished nicely with a right-footed effort.
Rovers were in control but a 42nd minute penalty allowed the visitors back into the game when Ian Vicars was harshly deemed to have fouled his man in the box.
That gave the Foxes the incentive and they came out much the stronger of the two sides at the start of the second half before evening up the scores after poor defensive play from Rovers allowed the offside trap to be beaten with ease.
With ten minutes remaining Rovers thought they had clinched all three points when excellent work from Brice Wassi saw the midfielder regain possession on the edge of the Leicester box and Jordan Preston finished smartly for his 13th goal of the campaign.
Unfortunately Rovers couldn't hold onto their lead and a minute from time their defensive fragilities came to the fore once more as a long free-kick from the goalkeeper saw a Leicester man to steal in to score.
"To be fair our play was good, particularly in the first period when we should have scored more goals," said coach David Lowe.
"Leicester started the second half strongly and we struggled to come to terms with their physicality in the first part after the break.
"There were some good performances, Jordan Preston impressed throughout, Dean Rittenberg looked a threat and Brice Wassi worked hard in midfield."
ROVERS: Killip, Vicars, Lavelle, Langford, Bauress, Green, Wassi, Torres, Doyle (sub Tanner 72 mins), Rittenberg, Preston.
Subs not used: Butterwick, Williams, Joel, Cham.