Wyre and North Preston MP Mr Wallace said the UK's intelligence agencies and armed forces "do what we have to" to keep the country safe but did so "within the rule of law".
His comments came after Mr Williamson was condemned by political opponents after saying Britons who have fought for Islamic State should never be allowed to come back to this country and "a dead terrorist can't cause any harm to Britain".
The Defence Secretary used a Daily Mail interview to say those who were intent on bringing "destruction, death and bloodshed" on to the streets of Britain were being "hunted down" and that threat "eliminated".
Asked if Mr Williamson's comments risked encouraging extremists to carry out attacks in the UK, Mr Wallace said: "I don't think there's a danger of it but I do think ... I'm a former soldier. Taking life is a very serious thing and, at the end of the day, this country, this Government and our intelligence services and armed forces do what we have to do to keep us safe at home and abroad.
"But we do it within the rule of law and the oversight of Parliament."
He told BBC Radio 4's Today programme: "We are the United Kingdom, we don't relish violence in anything we do. We do what we have to do to keep us safe."
Mr Wallace also said he was worried about the increased threat from far-right extremists in the UK.
"We are worried that it is increasing, and it is using all the grooming methods and the networking methods that we have seen used by other terrorist groups recently," he said.
"That is not good for our safety and we have got to take steps to deal with it."
He added: "Extremists on all sides of arguments would love to dominate the ground and antagonise their opponent, pushing them to the extreme to ultimately cause some form of conflict. That is part of the methodology of some of these people and we have got to make sure that we counter that."