The short answer is: first earthquakes, then the popes. For more on this fascinating building, check out my guided virtual tour of the Colosseum.
Further Reading
Nathan T. Elkins, A Monument to Dynasty and Death: The Story of Rome’s Colosseum and the Emperors Who Built It (Johns Hopkins University Press, 2019)
Ada Gabucci (ed.), The Colosseum (trans. Mary Becker) (J. Paul Getty Museum, 2001)
Keith Hopkins and Mary Beard, The Colosseum (Profile, 2005)