Paul Anthony Timmis, 54, formerly of Springfield Avenue, Bacup, was jailed for 14 years after being found guilty - following a retrial - of a string of rapes and assaults.
He tried to bring his conviction and sentence to the Court of Appeal in October but a judge rejected his case.
He has since made a second attempt, but another judge has thrown out his bid.
It is understood the second appeal related to three charges during his retrial, in which his lawyers questioned the prosecution being able to use Mr Pilsworth's evidence from the 13 charges Timmis was acquitted of during his original trial.
But High Court judges said the use of the evidence was permitted to explain the background leading up to the charges.
His victim Wayne Pilsworth, who previously waived his legal right to anonymity to raise awareness of sexual abuse, was able to attend on a video link.
Today he said: " They said that the judge that sentenced him acted 100% lawfully and the judge from first appeal, rejected his appeal correctly."
His harrowing ordeal dates back to the 1980s, when he was just 11 and his neighbour introduced him to Timmis, who was in his 20s.
Timmis groomed the impressionable school boy with cigarettes, alcohol and money, trips to gaming arcades, and lavished attention on him that he didn't have from his parents.
Timmis would sneak him into his parents' house and hide him in his wardrobe.
The youngster came from a challenging background, and when his parents were evicted from their home in Manchester, Timmis forced him to move in with him.
The adult went on to repeatedly rape and indecently assault his vulnerable victim in various locations that they moved to including Kirkham, Freckleton, Blackpool, and Greater Manchester between 1987 and 2000.
But in 2000, Mr Pilsworth refused to move to Scarborough with his abuser and fled.
He got a job at Tiggis in Guildhall Street, Preston, but unknown to his colleagues was sleeping rough on benches in the city.
Eventually he obtained shared accommodation off Fishergate Hill.
In 2004 he met his wife Teresa in Blackpool, and the couple, who now have three children, moved to Morecambe.
In 2017 he disclosed his ordeal to Lancashire Police, and has since depicted his traumatic story in unusual tattoos daubed all over his upper body.
They include the face of the judge who presided over Timmis' first trial at Preston Crown Court, and a lone wolf 'looking back' on his life
Anyone affected by issues raised in this story can call police on 101.
The Lancashire Post is more reliant than ever on you taking out a digital subscription to support our journalism. For unlimited access to Lancashire news and information online, you can subscribe here.