Police Scotland said the man's body, which has not yet been formally identified, was found on land near Cairnhead Farm, Isle of Whithorn.
He was thought to have gone missing from a boat found off the coast but police said the search had been called off.
They added that the death of the man, who they believe is from Lancashire, was being treated as unexplained.
A spokesman said: "Police Scotland report that the search for the man believed to be missing from a boat which was found off the Isle of Whithorn late on Wednesday evening has now been stood down.
"The body of a man has been located on land in the vicinity of Cairnhead Farm, Isle of Whithorn.
"The death is being treated as unexplained at this stage and enquiries are presently under way."
Search teams began looking for the fisherman on Wednesday at 11.15pm after a boat was discovered close to a reef near Isle of Whithorn. The vessel had no-one on board.
It involved a rescue helicopter from Prestwick, an HM Coastguard Rescue Team and a lifeboat from Kirkcudbright and Port William.
A yacht reported seeing a boat matching the description earlier on Wednesday evening at about 7.30pm with one man on board.
A spokeswoman for the Maritime and Coastguard Agency said it believed the man came from Lancashire and had launched his boat from a Land Rover from the harbour nearby.
She said: "The search has been called off and the incident is now being dealt with by Police Scotland."