The palace was commissioned in 1715 (some say 1730) by Don José Tagle and Brachio (Marquis Torre Tagle), who at the time was treasurer of the Royal Spanish fleet, for his own personal use as his home.
The exterior of the palace has a baroque stone doorway. The main facade is made from stone in the first wing and plaster in the second. The style is Sevillian baroque with a strong Mudejar influence. The materials used in its construction were brought from Spain, Panama and Central America.
Apart from carved columns, the palace is distinguished by two finely worked balconies in dark wood. These balconies (or miradors) adapt the European architecture to vernacular Peruvian tradition. The interiors feature Sevillian tiles, plasterwork, wooden columns, lobed Moorish arches and soaring coffered ceilings. It is considered to have a true "Limeño" architectural originality, harmoniously combining Andalusian, Moorish, Criollo and Asian features.
The public cannot easily visit the inside, but some visits can be made by appointment only at the office of 'Imagen Institucional del organismo público'.
Across the street from Torre Tagle, Casa Goyeneche (also called Casa de Rada) is another impressive 18th-century mansion, with distinct French influences. Other interesting similar sites that are nearby (in downtown Lima) are:
- Casa Negreiros, Jr. Azángaro 532
- Casa de las Trece Monedas, Jr. Ancash 536
- Casa Barbieri, Jr. Callao at Rufino Torrico
- Casa de Pilatos, Jr. Ancash 390
- Casa la Riva, Jr. Ica 426.