The Baroque college of the New Town Jesuits was built in several phases. In 1658–1665 and after 1674 the southern part with St Francis Xavier Church was built with the participation of the architects Martin Reiner and Carlo Lurago (?). During the following phase in 1687–1702, Paul Ignaz Bayer took part, which was completed in 1751 and after 1759 probably under the direction of Jan Josef Wirch. After the Jesuit order was abolished in 1773 a military hospital was established there in 1776. The site has been used by the general hospital (today faculty hospital) since 1789. The college is adjacent to St Ignatius Church (1665–1678) on the northern side.
During the bombing on 14 February 1945 the college was hit and mainly the central wall was destroyed. St Ignatius Church was not hit directly but corpses were deposited in the interior (including Josef Lada's daughter Eva, among others). In his photographs Josef Sudek captured the ruins of the college. Compared to photographers who took pictures of the place immediately after the raid and often photographed the dead bodies in St Ignatius Church (e.g. Stanislav Maršál ), in Sudek's photographs the ruins of war are presented rather through the emptiness of the usually crowded location.
After the war the college was reconstructed, including the complete renovation of the central wall to its original appearance.