Caldwell’s men were a goal down in the opening minute thanks to Ollie Clarke’s strike, before Rory Gaffney added a second in the 18th minute.
Goalkeeper, Thorsten Stuckmann prevented further pain, by saving a 30th minute penalty from Luke James, while Sylvan Ebanks-Blake added a late consolation for the Spireites in the 85th minute.
Speaking exclusively to Player HD, Caldwell said: “We just didn’t give ourselves the chance to win the game, you have to do the basics of football in any game, but especially away from home and we didn’t do that.
“We tried to fire them up, and be ready for what I knew would be a home crowd atmosphere early in the game, we then gift them a goal inside a minute and it’s an uphill battle from there.
“I thought we passed the ball well at times in the first half and created problems with Kristian [Dennis] and Rai [Simons] in behind, but basic footballing errors have cost us.”
Going into the game, Caldwell wanted to see a similar level of attacking performance, as displayed in the 3-3 draw with Peterborough United on Tuesday night, however, any such plans were thrown out of the window thanks to Clarke’s opener.
He said: “It’s crazy, like I say you have to do the basics of football and you can’t allow a team a two-goal start especially in our position, and too often we do that.
“We gift teams’ goals, and shots at our goal, it’s far too easy to attack us and we find how difficult it is the other way to create opportunities.”
The manager made one change to the XI from the match with the Posh, with Paul McGinn returning to the team in place of Connor Dimaio, and Caldwell was pleased with his fellow Scot’s performance.
“I wanted to go back to three at the back to give us a bit of solidity in the game, but obviously that went out the window in the opening minute,” he said.
“But I thought that Paul’s performance was outstanding, and showed a lot of people what it takes to be a footballer. He was brave in and out of possession, he won headers, tackles and did the basics well.
“To come back from such a long injury and to play like that in his first game, I was delighted with him.”
Next up for the Spireites is a home match with Rochdale on Saturday, and despite the result at the Memorial Stadium leaving the Spireites further away from safety, Caldwell still feels his team can get out of trouble.
He added: “We’ve got to keep going, and can’t let our heads drop. We need to keep trying to build from week to week and game by game, and keep putting things into the players’ heads that’s going to make them better.
“As long as there’s points left then it can turn quickly, we just need to get a win and back that up with another win.
“As soon as you put back to back wins together the optimism will raise right through the club so we have to keep going.”