The castle is first mentioned in 1219 as in arce Clanx. It was built by the Abbot of St. Gall Ulrich von Sax (1204–20) and his brother, the Abbey-Vogt Heinrich. It was named after the Sax family holding in the Calanca valley. As the center of the Abbey of Saint-Gall's property in Appenzell, the castle was of great importance. The castle was first destroyed in 1289 and then rebuilt. It was destroyed again in 1402 and not rebuilt.
In 1949 the foundations of a strong square tower, a smaller corner tower on the highest point of the hill and parts of the walls were excavated.