The castle was founded in 1348. Matthias of Arras is often credited with being the architect, but he already died in 1352, and Holy Roman Emperor Charles IV personally supervised the construction works and the decoration of interiors. The construction was finished nearly twenty years later when the "heart" of the treasury – the Chapel of the Holy Cross situated in the Great tower – was consecrated in 1365.
Following the outbreak of the Hussite Wars, the Imperial Regalia were evacuated in 1421 and brought via Hungary to Nuremberg. In 1422, during the siege of the castle, Hussite attackers used Biological warfare when Prince Sigismund Korybut used catapults to throw dead (but not plague-infected) bodies and 2000 carriage-loads of dung over the walls, apparently managing to spread infection among the defenders. Later, the Bohemian coronation jewels were moved to the castle and were kept there for almost two centuries, with some short-time breaks.
The castle underwent several reconstructions: in late Gothic style after 1480, in Renaissance style in the last quarter of the 16th century. During the Thirty Years' War in 1619, the coronation jewels and the archive were brought to Prague, and in 1620 the castle was turned over to