The castle was built by John de Courcy in 1184 and it was later occupied by the Savage family and their dependents, the Smiths. In the 17th century Sir James Montgomery, then living at Rosemount, Greyabbey, purchased the Quintin estate from Dulaltaigh Smith. His son William, built a walled courtyard and other smaller towers, a large house adjacent to the central tower and a great kitchen to the seaward side of the castle structure. They then re-roofed the castle and added new floors, all probably before 1659.
The Montgomerys sold the castle to George Ross, but he never lived in the castle, allowing it to become a near ruin. In the early 19th century the castle was inherited by Elizabeth Calvert (née Blacker), a descendant of Ross, and restored in 1850. The central keep was raised, a walkway made within the battlements, a drawing room opened into the inner gardens, and a dining room constructed on the lowest floor of the great tower. Most of the grounds were also enclosed by a massive stone wall.
Eventually the estate was purchased by the Burgess family, who lived in Quintin from the early 1930s until the 1950s. Skeets Martin, an auctioneer from Belfast, and his wife, then occupied the castle until the early 1970s. In 1978 the castle was opened as a private old people’s home, which it remained as for many years until sold into private ownership. The castle has been extensively refurbished by McGimpsey and Kane Builders, changing hands most recently in late 2006.