Dan Jones sparked the Spireites' first attack, when his cross was headed into the box by Liam Grimshaw but Cameron Burgess beat Sylvan Ebanks-Blake to the loose ball.
Having taken over earlier in the week, it was Lee Clark’s first match in charge of the Shakers and while they looked defensively solid, they were struggling in attack.
Osman Kakay’s lofted ball upfield was brought down by the toes of Ebanks-Blake, the forward looked to slip it past Joe Murphy but the Bury keepermade himself large and palmed wide.
The hosts continued to edge closer to an opening goal and Dan Gardner’s in-swinging free-kick looked to be curling into the top corner, but was punched away by Murphy
A moment of magic from Ebanks-Blake created the opener, when his chipped through ball was latched onto by Kristian Dennis (31) who slid it underneath Murphy.
Brought to life by Dennis’ strike, Ryan Lowe battled through a stubborn Spireites defence for a sight of goal and cut inside before blasting his shot wide.
It had been a quiet debut for Stuckmann before the break, but Bury were looking to make more of an impact in the second half. The visitors' attack was being led almost single-handedly by Lowe and the forward chased Greg Leigh’s ball into the box, however could not head it on target.
Tom Anderson came to the rescue for the Spireites as Bury’s influence on the game grew. Tom Pope was picked out in the box by Scott Burgess but Anderson intervened as he was about to strike.
Chesterfield remained pinned back into their own half but a dominating performance from Ian Evatt was stifling the away side’s attack. Miller unleashed a vicious strike that was blocked by the skipper.
Gary Caldwell’s men were fighting for their lives and Evatt had thumped yet another ball clear before a cross back into the box deflected off Anderson and hit the woodwork before being cleared away by Hird.
Twenty minutes remained and the Bury onslaught was relentless. Pope backed into Anderson and went to ground but the referee waved play on.
Rai Simons charged down the left wing on the counter for the home side and his low cross found Grimshaw but the midfielder blazed his effort over the bar.
Bury’s charges continued and Miller broke free once more, firing in a shot at the Spireites stopper who palmed wide.
Anderson was then sent off after bringing down Miller in the penalty area and conceding a penalty. With 15 minutes remaining Bury had a lifeline but debutant Stuckmann made a heroic penalty save.
For all their chances, it was a sensational strike from Tutte (78) that drew the away side level. The midfielder pulled the trigger from 25 yards out and smashed the ball past Stuckmann to stun the home crowd.
Having led for 78 minutes, the Spireites were shocked in injury time as Stuckmann failed to deal effectively with Hird's back-pass and Miller fired into the empty net.
Chesterfield: Stuckmann, Jones, Hird, Evatt (C), Gardner (Brown), Ebanks-Blake (Simons), Faupala, Dennis (Mitchell), Anderson, Kakay, Grimshaw.
Subs not used: Allinson (gk), Angel, Humphreys, El Fitouri
Bury: Murphy, Leigh, C. Burgess, Kay (C), Tutte, Pope (Miller), Mellis, S. Burgess (Styles), Barnett, Moore, Lowe (Hope).
Subs not used: Williams (gk), Mackreth, Pennant, Deslandes.