The news is a major setback for the Lancashire 37-year-old, who was just two games into his comeback following a calf problem sustained on the first morning of last summer's Ashes.
Anderson spent almost five months working back to full fitness and showed signs of being back to his rhythmical best as he helped the tourists level the series with a 189-run win, taking exemplary match figures of seven for 63.
He was in visible discomfort on the final day, though, clutching his side at times and grimacing through two painful overs after tea.
An MRI scan on Wednesday morning confirmed the bad news.
"James Anderson has been ruled out of the remainder of the Test series against South Africa after sustaining a left rib injury in England's second Test victory at Newlands," read a statement from the England and Wales Cricket Board.
"Anderson felt tightness and discomfort at the end of the morning session on day five and was only able to bowl eight overs during the day. He will return to the UK in the next few days."
Anderson's tally of 584 Test scalps make him England's record wicket-taker as well as the most prolific seamer in history, while his appearance in the Boxing Day Test at Centurion saw him become just the ninth man to win 150 Test caps. He used the occasion to suggest he was "not even thinking about retirement".
England were already giving consideration to sparing Anderson a thankless job on the spin-friendly pitches in Colombo and Galle but the issue will now be taken out of the selectors' hands, with a bone-related injury likely to require around two months of recovery work.
With that in mind he may now focus on getting ready for the English season, with a view to becoming just the fourth man in history to take 600 Test wickets after Muttiah Muralitharan, Shane Warne and Anil Kumble.
Somerset seamer Craig Overton will remain with the group having originally arrived as illness cover, but England should also have pace duo Jofra Archer and Mark Wood back in contention for next week's third Test at Port Elizabeth.
Anderson tweeted: "Frustrating to be missing the rest of this series with a broken rib but hopefully will be healed in a few weeks! Will be supporting the boys from home."