The Government-funded scheme, which provides the licences to the elderly, ends in June 2020 and the responsibility will be passed on to the BBC.
The BBC has to decide whether to continue the benefit but has said that shouldering the cost would "fundamentally change" the broadcaster.
Speaking at a protest arranged by the National Pensioners Convention outside the Department for Digital, Culture, Media and Sport, Ellen Lebethe, 81, chairwoman of the Lambeth Pensioners Action Group told the Press Association: "A TV for older people especially is absolutely essential, especially for those of us who are lonely, those of us who are housebound.
"It's one of what we call the universal benefits and universal means it's free for all over-75s and we feel it should be funded by the Government and not the BBC.
"What this means is slowly and slowly there will be cuts on our Freedom pass and all the other universal benefits.
"It (the free TV licence) should be kept and paid for by the Government. Once it goes it will go for good, it won't come back."
Allen Hawley, 71, said: "The BBC is targeting youngsters with their apps and free TV, everybody on a computer or with a phone can receive free TV and they are targeting pensioners because they sit at home with a static TV.
"They are being targeted because they are an easy target."
At a rally at Portcullis House, Caroline Abrahams, charity director at Age UK, read out letters she had received from pensioners saying how important the free licence is to them.
She added: "We are calling on the Government to change its mind and live up to its responsibility and stop hiding behind the BBC. It's not their fault, it's the Government's fault and older people are stuck in the middle.
"The Government should do the decent thing and stump up the cash."
Shadow public health minister Sharon Hodgson said it was "unconscionable" to take away free TV licences from pensioners, adding it was "an absolute disgrace".
She continued: "I hope the BBC will be taken out of this appalling situation they have been put in."
A Government spokesman said: "We know people across the country value television as a way to stay connected with the world.
"The BBC will take on responsibility for free licences for the over-75s from 2020 and it is right that they have consulted the public before making any decisions.
"We've been clear that we would want and expect them to continue with this important concession."