Probably built in 1132 by Norman King Roger II, it was destroyed in 1156 by king William I of Sicily and rebuilt and reinforced in 1233 by the Holy Roman emperor Fredrick II. During the Angevin domination, it went through several transformation, and after being acquired by Duke Ferdinand of Aragon, was donated to the Sforza family and passed to Bona Sforza, Queen of Poland. After Bona's death, it was returned under the King of Naples and transformed into a prison and barracks.
The castle is surrounded by a moat on all sides, except the northern section, which was bordering the sea and can be accessed from the bridge and the gate on the southern side. It is mainly composed of the Aragon walls and the main Swabian tower, and is currently used for exhibitions.
There is a story that, in the year 1221, Emperor Frederick II met Saint Francis of Assisi in this castle. According to tradition, the emperor had a courtesan sent to Francis's room and watched through a peephole to see what would happen. When Francis sent the woman away, Frederick was impressed with his principles; the two spent the rest of the night in conversation. This story is not confirmed beyond doubt, but it is considered believable.