Preston North End gave Arsenal an almighty scare at Deepdale but bowed out of the FA Cup, beaten by a last minute goal.
The Lilywhites took a seventh-minute lead through Callum Robinson and produced a fine first-half performance to shake the Gunners who had never lost a third-round tie under Arsene Wenger's management.
Those 45 minutes were as well as North End have played this season and the Gunners could have had no complaints had they gone in further behind.
Chances came and went to stretch the advantage but then back came the visitors after the interval to level and then score their winner in the 89th minute.
Aaron Ramsey fired them level just 50 seconds into the second half, then Olivier Giroud found the net as the clock ticked towards the final whistle.
The Premier League side's pressure had been mounting but the manner in which the game finished was still so cruel on North End.
PNE manager Simon Grayson had made three changes to the North End starting XI, Aiden McGeady, Paul Gallagher and Marnick Vermijl coming into the side.
They replaced Eoin Doyle, Alan Browne and Bailey Wright, centre-half Wright having been sold to Bristol City on Friday.
Deepdale was rocking before kick-off and North End responded with a high octane start which continued throughout the first half.
Inside the first two minutes, Robinson saw a shot blocked in the box after good approach play from Ben Pearson had played him in.
North End went in front in the seventh minute, great play from McGeady at the axis of the build-up.
McGeady collected possession centrally and drove forward, a smart pirouette turn taking him beyond a defender before he slid a pass to Jordan Hugill in the box.
As Hugill reached it, a challenge from Nacho Monreal knocked him off his feet but the ball spun across the box into the path of Robinson who held off Shkodran Mustafi to side-foot home from six yards.
The Gunners' first sight of goal was Lucas Perez's first-time shot from the edge of the box which went straight into Chris Maxwell's gloves.
But it was Preston in the main bossing the play, some good chances falling their way as the first half progressed.
Robinson's close range header from a Hugill cross was blocked by an Arsenal defender, then Robinson sprung the offside trap down the inside right channel, his low cross just evading Hugill who appeared to slip as he moved to meet it.
The impressive Pearson tried his luck with a low shot from 25 yards which keeper David Ospina seemed to slightly misjudge and could only shovel it past the post.
As the half-hour mark approached, McGeady got down the left-wing, his cross finding Vermijl beyond the far post who volleyed wide.
Pressure from the visitors saw Aaron Ramsey get the ball in the box but good pressure from the PNE defence crowded him out and when the ball broke to Perez, he was forced out to the side of the box.
Giroud got clear down the left side of Preston's box in the 34th minute, his chip beat Maxwell but dropping across goal and bouncing beyond Alex Iwobi who was sliding to try and meet it.
Within sixty seconds, North End almost doubled their lead at the other end as Hugill flicked a pass through to Gallagher, the midfielder lifting a shot over Ospina which Monreal got back to clear off the line.
They had the ball in the net five minutes before half time when Robinson headed home from close range but the whistle went for a foul on the keeper by Paul Huntington.
Arsenal came out all guns blazing for the second half and levelled within 50 seconds of the re-start.
The ball was worked in from the left to Iwobi who laid a low pass into the path of Ramsey, the Wales international taking the ball into his stride before rifling a right-foot shot past Maxwell from just inside the box.
A McGeady cross was plucked out of the air by Ospina as Tom Clarke closed in to try and get his head on it.
Razor-sharp reactions from Maxwell denied Arsenal the lead in the 73rd minute, the North End goalkeeper saving superbly from Mustafi at the near post after the centre-half had met a corner.
Giroud volleyed the ball into the PNE net in the 76th minute after getting beyond the home back line but the flag had gone up for offside against the French front man.
Daryl Horgan came off the bench for his Preston debut with nine minutes remaining, the former Dundalk winger replacing Johnson.
Former North End loan man Danny Welbeck joined the action soon after in Arsenal's first change, coming on for Alex Oxlade-Chamberlain.
The visitors' pressure which had been mounting throughout the second half, paid off in the 89th minute.
A ball into the PNE box was headed on by Giroud and chased by Perez to the byline.
Perez back-heeled it back into the path of Giroud who fired home a low shot at the near post from close in, possibly with then help of a deflection.
Three minutes into stoppage-time, Welbeck let fly with a shot from 25 yards which Maxwell tipped over the bar.
PNE: Maxwell, Vermijl, Clarke, Huntington, Cunningham, Pearson, McGeady, Gallagher (Browne 59), Johnson (Horgan 81), Robinson, Hugill (Makienok 75). Subs (not used): Doyle, Spurr, Pringle, Lindegaard.
Arsenal: Ospina, Maitland-Niles (Holding 90), Gabriel, Mustafi, Monreal, Xhara, Ramsey, Oxlade-Chamberlain (Welbeck 83), Perez (Reine-Adelaide 90), Iwobi, Giroud. Subs (not used): Holding, Jenkinson, Reine-Adelaide, Dasilva, Willcok, Martinez.
Referee: Robert Madley (West Yorkshire)