Simon Weaver's men were the architects of their own downfall at the CNG Stadium on Saturday afternoon, throwing away an early lead before going on to concede a succession of awful goals in the freezing cold.
The opening quarter of an hour saw Town knock the ball about neatly, but there was little action of any real note until the game exploded into life after 18 minutes.
Jack Emmett got down the right and into the away box where he delivered a dangerous cross that Jordan Keane couldn't help but divert into his own net from just a couple of yards out.
Taking the lead seemed a just reward for a decent start where the hosts had dominated possession, though they were pegged back within three minutes.
Emmett initially did well to win the ball deep in his own territory, but then immediately gave it away again, allowing County to shift the play right where Darren Stephenson found Adam Thomas and the latter shot low across James Belshaw and inside the far post.
Less than 60 seconds later, a Liam Kitching error afforded Stockport striker Frank Mulhern the opportunity to cut in from the right and run in on goal before he calmly slotted past Belshaw.
The visitors, unbeaten in 13 matches prior to kick-off, had barely had a kick prior to this, but all of a sudden they found themselves very much in the ascendancy.
Mulhern headed high and wide when well-placed, yet having withstood a spell of pressure, Town rallied, Jack Muldoon rifling into the side-netting from a narrow angle.
They were then gifted the chance to level matters with almost the last kick of the first half when a loose pass sent George Thomson scampering towards the away goal.
He opted to shoot early, and although his strike had plenty behind it, it was too close to Ben Hinchliffe, who managed to parry.
The second period did not begin well for Weaver's men as they once again shot themselves in the foot.
Warren Burrell under-hit an attempted back-pass and Matty Warburton was able to stroll in on goal, dance around the despairing dive of Belshaw and roll into an empty net.
A flat 20-minute spell followed as the Hatters appeared to be cruising into the quarter-finals, however Town did eventually begin to show some signs of life.
Josh Falkingham played in Thomson on the left side of the penalty area where he had time to pick his spot, however Hinchliffe did enough to divert the winger's effort past the far post.
The introduction of Aaron Williams from the substitutes' bench then sparked a spot of head tennis and a scramble in the away box, but that was about as good as it got for Harrogate.
Play soon switched to the other end, County left-back Scott Duxbury getting forward from defence to extend Belshaw before the visitors' fourth goal arrived in the 79th minute.
Thomson gifted away possession and Mulhern was sent clean through, rounding Belshaw before coming back inside and firing beyond covering defenders on the goal-line.
There was time for a consolation strike for the men in yellow and black, Falkingham picking out Ryan Fallowfield who drew Hinchliffe then pulled back for Thomson to fire home, though Town's hopes of a trip to Wembley had already long evaporated.