They always had the upper hand over the Shakers but couldn't quite shrug them off, the sound of the final whistle coming as a relief.
North End came out of the blocks quickly, finding the net twice in the opening five minutes.
Midfielder Terry Gray game them the lead from close range, with David Johnson adding the second soon after.
Most Popular
-
1
Luton Town boss says best team didn't win after Championship defeat to Preston North End
-
2
PNE player ratings vs Luton Town as Ben Whiteman and Liam Lindsay steal the show in hard-fought win
-
3
John Smith's PNE Fans' Panel verdict: Sunshine all the way
-
4
Longridge Cricket Club will play at Emirates Old Trafford in a bid to win the Lancashire Knockout Cup Final
-
5
Match Report: Luton Town 0-1 Preston North End - Brad Potts stunner gives North End first Championship win
Wayne Entwistle reduced the arrears, the Bury striker finding the net in the 19th minute.
Early in the second half, PNE put more daylight back between themselves and the visitors when Johnson netted his second of the game.
However, the Shakers pulled it back to 3-2 when Craig Madden scored just beyond the hour mark.
Willie Naughton scored Preston's fourth goal in the 77th minute, only for Trevor Ross to net seven minutes from time to set up a nervous finish.
The game was watched by a crowd of 5,013, that a decent figure when league crowds at Deepdale had been struggling to hit the 4,000 mark.
North End's reward for beating Bury was a second-round tie with non-league side Telford United at Deepdale.
That proved to be a torrid day for PNE, Telford winning 4-1. It was the second game of a 13-match run without a win for PNE.