The Museum is situated in the old part of the town, in the representative baroque palace called Vojković-Oršić-Rauch, which was built at the end of the 18th century. In the course of the history the palace frequently changed its owners and its tenants and mirrored the social and the financial status of the aristocratic families.

On the facade of the palace, there is the coat of arms of the Vojković family. It is a coincidence that it is similar to the coat of arms of the earls Oršić, who were the second owners of the palace.

After the Oršić family, the palace was owned by the baronial family Kulmer who were followed by the Rauchs, also barons. The interior of the palace from their time has been preserved. In the big ballroom on the first floor balls, concerts and theatrical performances were held, in which the representatives of the Croatian social elite of the 18th, the 19th, and the 20th centuries took part.

After the Zagreb town had bought the palace from its last private owners in the 1930s, it became the residence of Zagreb mayors. In 1959 the palace was temporarily assigned to The History Museum of Croatia. After it had been merged with The Museum of the Revolution of the Peoples of Croatia in 1991, a new institution was founded - The Croatian History Museum.