Humans lived so long that they had too much to loose for colonizing space. Humanity was in 51 baskets (50 Spacer-World and Earth). In less than a million years, calamity would wipe out all of humanity.

The robots rolled back medical technology and artificial intelligence (the robots made certain that they are the only sapient AIs) for making the risk/benefit-analasis for colonization worth it to humans. The robots probably held back technology generally for preventing dangerous disruption.

For the Foundation to survive, it needed to advance for protecting itself against the dying Galactic Empire and newly independent factions. The robots allowed the Foundation to have technological progress, except in the fields of medicine and AI. After the Mule conquered the Foundation, his SpaceNavy, if displaced back in time 500 years, when the Empire, although politically rotting, with economic upheavals, still had the best military it ever had, the MuleFleet could easily defeat the Imperial StarNavy and conquer the whole Galactic Empire.

In Foundation's Edge, the Mayor of Terminus wanted to throw out the SeldonPlan and just conquer the whole Galaxy just 500 years into the SeldonPlan (the 2nd Foundation and Gaia stop her).