Sir Amyas Morse, head of the National Audit Office, flagged up the rise as he published a highly critical report on a controversial part-privatisation of probation services.
From 2015 every criminal given a custodial sentence became subject to statutory supervision and rehabilitation upon release into the community.
Prior to the change, which was designed to drive down re-offending rates, convicts who had served less than one year did not have to be supervised by probation services.
The NAO's report said that, between January 2015 and September 2018, the number of offenders recalled to prison for breaching their licence conditions increased by almost half (47%), from 4,240 to 6,240.
Over the same period, the percentage of offenders recalled to custody who had received sentences of less than 12 months increased from 3% to over a third (36%).
"Offenders serving short sentences often find it difficult to comply with licence conditions as they are more likely to have chaotic lives, with accommodation difficulties, mental health problems and alcohol and drug dependency," the report said.
It also noted that the number of reviews carried out when a supervised offender is charged with a serious sexual or violent crime increased by almost a quarter, from 507 in 2015-16 to 627 in 2017-18.
Whitehall's spending watchdog cited the figures in its latest assessment of an overhaul of arrangements for managing offenders in England and Wales.
In 2013, under then-justice secretary Chris Grayling, the Ministry of Justice embarked on major reforms of probation services.
Known as Transforming Rehabilitation, the scheme saw the creation of the National Probation Service to deal with high-risk cases, while remaining work was assigned to 21 community rehabilitation companies (CRCs).
Sir Amyas said: "The ministry set itself up to fail in how it approached probation reforms.
"Its rushed roll-out created significant risks that it was unable to manage.
"These have had far reaching consequences.
"Not only have these failings been extremely costly for taxpayers, but we have seen the number of people on short sentences recalled to prison skyrocket."
Overall, between 2011 and March 2017, there was a 2.5 percentage point reduction in the proportion of proven re-offenders, the NAO's review said.
However, the average number of re-offences per re-offender increased by 22% over the same period.
Last year, the MoJ announced plans to end the CRC contracts 14 months early, in December 2020.
But the report warned this "comes at a cost".
It said: "When added to previous, unsuccessful, efforts to stabilise CRCs, the ministry will pay at least £467 million more than was required under the original contracts.
"It now estimates it will pay up to £2.3 billion under the contracts. This is still less than the £3.7 billion expected at the outset of the reforms, but little progress has been made on transforming probation services."
Sir Amyas welcomed proposals to address some of the issues that have caused problems.
But he added that "risks remain", and called on the department to "pause and think carefully about its next steps".
Meg Hillier, chairwoman of the Commons Public Accounts Committee, said: "In its haste to rush through the reforms the Ministry of Justice has failed to deliver the transformation it promised."
Unison national officer Ben Priestley described the report as a "devastating critique of a catastrophic privatisation".
As of September, just over a quarter of a million offenders were on probation in England and Wales.
Prisons and Probation Minister Rory Stewart said: "Transforming Rehabilitation has meant that we are now monitoring 40,000 more offenders than we were in the past. This is good for public safety.
"I am pleased that the report recognises the strong performance of the National Probation Service in looking after our higher risk offenders.
"But the performance of the CRCs, which look after our lower risk offenders, is too often deeply disappointing.
"That is why we have stepped in to end contracts early and invested an extra £22 million a year in services for offenders on release.
"We take the NAO's findings very seriously and will set out our detailed proposals for the future of probation later this year."
The MoJ added an inspection report found "almost all" NPS and CRC recall decisions were "good decisions".
Ian Lawrence, general secretary of the Napo probation union, told The Guardian: "Not only have the CRCs been unable to deliver services, the NPS also has significant failings.
"Staff shortages have resulted in unmanageable workloads across the board," he added.
"A (Transforming Rehabilitation) mark 2 will not resolve the issue of the first reform programme and will only perpetuate the ongoing risks and issues we have seen over the last four years."