The castle of Latour was first mentioned in a charter of 1066. The discovery in the foundations of the castle of sculpted blocks dating back to the Gallo-Roman era lets us suppose an earlier occupation of the site. The castle belonged to the La Tour family until the middle of the 15th century, when the ruins of the building (destroyed after a siege in 1433) became, by marriage, property of Henri Bayer de Boppard. La Tour passed, once again by marriage, to the de Créahanges family who sold the land and the seigneury to Mathieu Baillet on 7th October 1659.
The most famous Lord of Latour is without any contest Maximilian Anton Karl, Count Baillet de Latour, Field Marshal of the Austrian Imperial Armies and owner of the Dragoon Regiment of Latour. He was the last Lord of Latour as the castle was burnt down by the French revolutionaries on 17th April 1794. Since 1962, the site has started a new life. After having been a dancing and a recreational area in the sixties, the ruins have been partially rebuilt to accommodate a hotel and a restaurant.