TalkTalk customers are less likely to recommend their provider to a friend and their overall satisfaction is below average, according to the "most comprehensive" survey the regulator has carried out into telecoms customer service.
TalkTalk scored poorly on most customer service measures. Ofcom said its customers are less satisfied with reliability, more likely to have a reason to complain and less satisfied with how their complaints are handled than any other provider.
Overall, eight in 10 broadband customers (80%) said they were satisfied with their service.
Virgin Media was the only provider with above-average customer satisfaction for broadband speeds and its broadband customers are more likely than average to recommend the company to a friend.
However, Ofcom said the broadband industry could improve its handling of complaints, with the amount of time people are kept waiting on the phone to customer service lines varying widely.
Plusnet customers had to wait the longest at an average of almost eight minutes, while EE customers had an average wait of just 48 seconds.
Among mobile providers, Vodafone and Virgin Mobile scored below average in a number of satisfaction rankings, particularly when it came to handling complaints.
Overall, nine in 10 customers (91%) said they were satisfied with their mobile phone service, with just 4% having a reason to complain.
Tesco Mobile and giffgaff customers were more likely to recommend their provider to a friend, while the two providers as well as EE generally scored highly compared to the sector average for the quality of their service.
Ofcom encouraged customers to compare providers and switch if they are "not getting the service they deserve".
Lindsey Fussell, Ofcom's consumer group director, said: "People often focus on price when they're choosing a phone or broadband provider. But there are big differences in the customer service offered by providers.
"We're encouraging people to look beyond the price and consider customer service too. In such a competitive market, companies simply can't afford to let their service standards slip.
"If they don't up their game, customers can vote with their feet."
A TalkTalk spokesman said: "While these results are disappointing, we are already seeing more customers staying with us than ever before as we roll out major service improvements.
"We've closed down our call centres in India, we have introduced new online tools to help customers resolve issues quickly and conveniently and we are launching new wi-fi routers to strengthen network reliability.
"All these initiatives are starting to deliver material improvement in satisfaction levels."
Vodafone said its own, independently verified consumer satisfaction score increased 12 points in the last year to its best ever level.
A spokeswoman said: "But we know there is still more to do. This year, we're recruiting even more UK-based customer service employees and are empowering our front-line team to resolve more customer inquiries first time.
"We are also continuing to invest in new, market-leading digital technologies to make it even easier for customers to interact with us."
Citizens Advice chief executive Gillian Guy said: "Consumers deserve services that meet the demands of modern life, yet these new figures show that not all providers are delivering. Citizens Advice research also reveals that loyal customers face higher prices in the telecoms market.
"There are clearly some issues that need tackling. The Government's Consumer Green Paper offers an opportunity to strengthen consumers' hands by establishing an independent telecoms consumer advocate. This would help address poor service and ensure the consumer voice is heard in this increasingly important market."
Minister for digital Margot James said: "Broadband and mobile phones have become modern-day necessities, and we all know how frustrating it is when it doesn't work.
"We've introduced a raft of measures with Ofcom to help protect consumers, including receiving compensation when their service falls short, and making it easier for customers to switch provider if they're not getting what they've paid for and are entitled to."