Ukraine says more than 260 soldiers from the Azov steelworks in Mariupol have been evacuated.
The regiment at Azovstal said its entire garrison is implementing a decision of Ukraine's military command.
Deputy Defence Minister Hanna Malyar said that 53 heavily injured fighters were taken to the town of Novoazovsk, held by Russian-backed rebels. Another 211 were evacuated using a humanitarian corridor to another rebel-held location, Olenivka.
Malyar said the 264 wounded soldiers would be exchanged for captured Russian soldiers.
Reuters news agency said about a dozen buses carrying Ukrainian fighters who were holed up beneath the besieged plant have been seen leaving the site.
"We hope that we will be able to save the lives of our guys," Ukraine President Volodymyr Zelensky said in a late night address. "There are severely wounded ones among them. They're receiving care. Ukraine needs Ukrainian heroes alive."
Ukraine's military said it had "ordered the commanders of the units stationed at Azovstal to save the lives of the personnel" and that troops there had fulfilled their combat mission.
Efforts to rescue troops still inside were under way, the military added. It did not say how many troops remained.
Ukrainian troops say they held out in Azovstal for 82 days, buying time for the rest of Ukraine to battle Russian forces and secure Western arms needed to withstand Russia's assault.
But the evacuation likely marked the end of the longest and bloodiest battle of the Ukraine war and a significant defeat for Ukraine. Mariupol is now in ruins after a Russian siege that Ukraine says killed tens of thousands of people in the city.
Since Russia launched its invasion in February, Mariupol's devastation has become a symbol both of Ukraine's ability to withstand Russia's invasion and of Russia's willingness to devastate Ukrainian cities that hold out.
The evacuation came hours after Russia said it had agreed to evacuate wounded Ukrainian soldiers to a medical facility in Novoazovsk.
Azovstal's last defenders had been holding out for weeks in bunkers and tunnels built deep underground to withstand nuclear war. Civilians were evacuated from inside the plant, one of the largest metallurgical facilities in Europe, earlier this month.
The wife of an Azov Regiment member described conditions at the plant earlier on Monday: "They are in hell. They receive new wounds every day. They are without legs or arms, exhausted, without medicines," Natalia Zaritskaya said.
'Heroes of our time'
The Ukrainian military's general staff has said that fighters evacuated from the Azovstal steel plant in Mariupol are "heroes of our time".
In a message posted to Facebook, army chiefs said the determination of the Ukrainian troops had forced Russia to keep around 20,000 troops entrenched around the city and "hindered the implementation of the plan for the rapid capture of Zaporizhzhia".
"Forging the enemy's core forces around Mariupol has given us the opportunity to prepare and create the defensive frontiers on which our troops are still present today and give a decent counterpoint to the aggressor," the post said.
"We got the critically needed time to build reserves, regroup forces, and get help from partners," military chiefs went on.
Azovstal is one of the largest metal works factories in Europe, covering a 10 sq km site.
Russian troops pushed back to border - US official
Ukrainian officials say their forces have managed to push Russian troops back in the area around Kharkiv, the second largest city in Ukraine.
For the last few weeks, the Ukrainian army has retaken around a dozen villages in and around Kharkiv, making it harder for Russia to attack.
A senior US defence official has given an assessment of the situation, saying Ukrainian forces have pushed back Russian troops to within 3km-4km of the Russian border, Reuters reported.
Earlier today, a group of those soldiers were pictured alongside a restored border sign in the colours of the Ukrainian flag - having reportedly reached the frontier with Russia.
The BBC has not been able to independently verify these claims.
-BBC / Reuters