The original castle was constructed in one summer, in the year 1492. It was named after Moscovian Grand Prince Ivan III. Its purpose was to fend off the Livonian Knights. The castle is strictly quadrilateral, measuring Шаблон:Convert, with walls 14 meters tall.
Ivangorod was won back later in the year by Muscovite forces, under the command of Prince Ivan Gundar and Mikhail Klyapin. Three thousand troops arrived to retake the castle, rebuild it, and construct a new barracks and stronger bastions. For almost 10 years, the land around the castle was in constant warfare. The fortress and the land around changed hands repeatedly. The castle was reconstructed and fortified many times, becoming one of the strongest defensive structures in the 16th century. The castle was in development until the 17th century, becoming a large, sprawling fortress with several lines of defense.
In 1704, Peter the Great captured the castle from Swedish troops, bringing the fortress back into Russian control. Inside the fortress, there are two churches: one is dedicated to the Virgin's Assumption (1496) and the other to St. Nicholas (built in the late 16th century but later reconstructed).
After the early 18th century, the military role of the fortress dwindled due to technological advances. In 1728, a review was carried out of the fortresses in this area, which concluded that the installation had been neglected, and had a low fighting efficiency. An order was issued for restoration of Ivangorod fortress, but after the inspection of 1738 the fortress was designated not adequate for defence purpose.
In 1840, some improvements were carried out in the fortress (roofs were changed), further improvements took place in 1863 and 1911-1914. During World War I, the fortress was captured by Germans on 25 February 1918. From 1919 to 1940, the fortress belonged to Estonia. Despite changing hands several times in the first half of the 20th century, the fortress played no significant role in fighting.
During World War II, the fortress was first controlled by the Soviet Union (1940-1941) and then by Nazi Germany (1941-1944). The Germans established two POW camps within the fortress and left many of its buildings damaged after their retreat.
Currently, the fortress serves as a museum.
The history and art museum of Ivangorod fortress exhibits paintings of Ivan Bilibin, Alexandra Pototskaya and other authors. As well, local history and tradition are explained by some of museum's exhibits, such as archeological finds encountered in the surrounding areas. There is a permanent exhibition devoted to the Northern War and the Livonian War. As well, there are models of fortresses in the surrounding region and armaments, and documents and letters related to famous people such as Fyodor Dostoevski, exhibited in the fortress museum.