Afghan authorities said Friday they had started to release 400 Taliban prisoners, the final hurdle in long-delayed peace talks between the two warring sides, even as President Ashraf Ghani warned they were a "danger to the world".

The Afghan government and the Taliban are set to meet within days of the prisoner release being completed, in a move that has drawn widespread condemnation after it emerged many of the inmates were involved in attacks that killed scores of Afghans and foreigners.

A group of 80 prisoners had been released Thursday, National Security Council spokesman Javid Faisal said, tweeting that it would "speed up efforts for direct talks and a lasting, nationwide ceasefire".

The release of 400 militants was approved at the weekend by a gathering of thousands of prominent Afghans who said they wanted to pave the way for talks to begin in Doha, Qatar, and a possible ceasefire.

But the decision has caused heartache for the families of those killed by the insurgents.

"If (the Taliban) can't bring peace and they attack again, thousands of people will be killed and their families will be tormented," said Bashir Naween, whose brother was killed in a 2017 truck bombing near the Germany embassy in Kabul, an attack involving one of the militants due to be released.

"But if the real peace comes, we won't have any problems because... our big dream is peace," he told AFP.

The prisoners include at least 44 insurgents of particular concern to the United States and other countries for their role in high-profile attacks, according to an official list seen by AFP.

Australian Prime Minister Scott Morrison said earlier this week he had lobbied for a former Afghan army soldier, who went rogue and killed three Australian colleagues, to stay in jail.

Ghani warned on Thursday that the hardened criminals were "likely to pose a danger both to us and to (America) and to the world".

"Until this issue, there was a consensus on the desirability of peace but not on the cost of it," Ghani said in a video conference organised by a US think tank. 

"We have now paid the major instalment on cost and that means peace will have consequences."

A prisoner swap formed part of a deal signed by the Taliban and the US in February, which saw Washington agree to withdraw its troops from Afghanistan in return for various commitments from the insurgents including a pledge to hold peace talks with the Afghan government.

The government was excluded from that deal, which stipulated Kabul release 5,000 militants in return for 1,000 Afghan government prisoners held by the insurgents.

While Kabul released the bulk of Taliban prisoners as agreed in the deal, it had baulked at freeing the final 400.

The "loya jirga" -- the traditional gathering of tribal elders and other prominent citizens -- was called by Ghani after the authorities initially refused to free the militants.

In a resolution, the jirga asked authorities to monitor the freed prisoners to ensure they did not return to the battlefield.

Ahead of the jirga, US Secretary of State Mike Pompeo had urged the gathering to release the prisoners, although he acknowledged the move was "unpopular".

The Taliban, meanwhile, have warned of possible attacks against the freed prisoners by Islamic State jihadists in coordination with Afghanistan's spy agency.