Match report: PNE 1 Hartlepool 0
The game looked destined for 30 extra minutes and possibly a penalty shoot-out, when Doyle struck to end the brave resistance of League Two outfit Pools.
He steered home a great cross from substitute Chris Humphrey to book North End a place in the second round of the competition.
North End had some great chances in the second half to break the stalemate, Jordan Hugill seeing a header cleared off the line while Doyle hit the post and Callum Robinson squandered a great late chance before the winner finally came.
Both sides had the ball in the net earlier only to be denied by offside flags - Joe Garner for PNE in the first half and Lewis Alessandro in the second.
Alan Browne and Doyle had come into the starting XI in place of Tom Clarke and Jermaine Beckford.
Grayson switched to a 4-4-2 formation, with Liam Grimshaw at right-back after his use as a wing-back on the opening day.
Browne came into the midfield after being a second-half substitute at Reading, with Doyle brought in to partner Garner up front.
Beckford moved to the bench with Clarke ruled out with a hamstring injury.
The match kicked-off with several hundred supporters still in long queues outside the two home stands which were open, a problem which must be addressed with urgency.
It took a time for the action to spark into life and when it did, it was North End who bossed the first half without being able to carve a route through the Pools defence.
Paul Gallagher, skipper in place of the absent Clarke, curled a 16th minute free-kick into the wall after Doyle had been fouled.
In the same passage of time, Garner tried his luck with a powerful shot from 20 yards which struck the big frame of visiting centre-half Aristote Nsiala.
Bailey Wright earned a yellow card for a sliding challenge on the speedy Nathan Woods, once which halted a rare first-half break up the pitch for Pools.
North End maintained their pressure and had the best chance of the opening period when Gallagher's right-footed cross from the left channel travelled across to Ben Pringle beyond the far post.
Pringle showed good technique to meet it with a cushioned side-foot volley, one which Pools goalkeeper Adam Bartlett pushed behind at the near post.
A Doyle header looped over the bar from Pringle's cross, then a shot from Gallagher clipped Nsiala on the head and deflected behind for a corner.
Hartlepool's one opening of the first half came when Nicky Deverdics' free-kick into the box was inches away from being met by the in-rushing Nsiala.
Garner had the ball in the net after half an hour, meeting a free-kick from Pringle with a powerful volley, however the striker had strayed just ahead of the last defender and up went the assistant's flag for offside.
Another sight of goal for Garner saw him meet Gallagher's cross with a stooping header, once which he directed across goal and wide of the far post.
In the 35th minute, Doyle got room down the right side of the box and drove a powerful centre across goal, one which hit Garner and flew wide.
Six minutes before the interval, Gallagher's cross was met by Browne who had timed a late run into the box, the Irishman's header dropping just the wrong side of the post.
Early second-half pressure from Pools saw Wright having to make a last-ditch clearance to stop a bouncing ball reach the visitors' strike force.
Then Nicky Deverdics put a free-kick too high, Browne having given it away with a foul on Thomas.
Good work by Gallagher, first in keeping a high ball in play on the touchline and then cutting inside, set up a chance for Johnson - his low curling shot from 20 yards going just wide.
Twice Hartlepool went close in the space of 90 seconds, Matthew Bates heading over the bar, then Josh Laurent chose to take a shot early when space opened up for him behind the Preston defence, Anders Lindegaard getting behind it to save comfortably.
Grayson made a double change just past the hour, Hugill and Robinson replacing Garner and Johnson, the switch seeing Gallagher move into the centre of midfield to accommodate Robinson on the left.
Former Morecambe man Alessandra had the ball in the net for the visitors in the 66th minute but an offside flag saw it ruled out.
North End went straight down field and forced a corner, one which almost led to the deadlock being broken.
Robinson delivered the flag-kick from the left, Hugill connecting with a header which beat the keeper but not Michael Woods who cleared acrobatically off the line.
The woodwork denied Doyle with 70 minutes played, the ball bouncing off a Hartlepool leg and spinning into the path of the striker who from a tight angle, shot against the post.
Hugill got away down the left side of the box to chase a Robinson pass with 10 minutes remaining, his low effort not really a shot or a cross, with the keeper diving to cut it out.
Humphrey replacing Pringle in the 81st minute was PNE's last throw of the dice in terms of their substitutes.
They came so close to scoring with four minutes remaining, a ball in from the left finding Doyle whose shot was pushed out by keeper Bartlett, the follow-up by Hugill blocked by Nsiala.
North End then should have scored on the break in the first minute of stoppage-time, Robinson lifting a shot too high over the bar from eight yards after a Doyle shot had been blocked.
But there was still time for them to find a winner, the game 28 seconds beyond the three minutes of added time signalled when Humphrey's low cross from the right into the six-yard box was guided home by Doyle.
PNE: Lindegaard, Grimshaw, Wright, Huntington, Cunningham, Pringle (Humphrey 81), Browne, Johnson (Robinson 62), Gallagher, Doyle, Garner (Hugill 62). Subs (not used): Beckford, Spurr, Welsh, Maxwell.
Hartlepool: Bartlett, Magnay, Bates, Nsiala, Carroll, Woods, Deverdics, Laurent, Alessandra, Thomas, Amond. Subs (not used): Featherstone, Jones, Oates, Hawkins, Richards, Smith, Dudzinksi.
Referee: Andy Woolmer (Northants)
Attendance: 4,509 (180 away)