This Hanroku Garden was made through the maintenance and renovation of the garden in the old residence of Hanroku Nakano. The Hanroku Nakano family was one of the prominent, long-standing landowners in this area and wealthy merchant who conducted extensive business in the shipping and brewing trade.

Handa Canal, which runs east of the garden, was the estuary harbor of Aui River in those days where from the late 17th century, shipments of brewing products such as sake, which was made in this area and called "CHugoku-shu," and vinegar wre busily loaded on ships to Edo. The renovation work of Handa Canal in the late 19th century enhanced the prosperity of this area and becme a driving force for the development in Handa threafter.

The present residence and the garden were built in 1889, and in the old days there were a lot of buildings such as a tea room and nice storehouses and so on as well as the main house. It said that the water in the pond in the circuit-style Japanese garden was seawater drawn from the Canal and its water level fluctuated with the tide.