In 1824, Lieutenant John Cotton, a West Point graduate, was stationed at Fort Howard in Green Bay. While there, he met Mary Arndt, the daughter of local judge John Penn Arndt. In 1825, the two married, and the next year, Cotton was transferred to another army post in a different part of the country. John and Mary Cotton moved several times between 1826 and 1848 according to John’s army orders.

In the early 1840s, Judge Arndt built a house for his daughter and son-in-law in Green Bay. The house is a temple style Greek Revival home which features a two-story central portico. This portico is distyle-in-antis in style, meaning that it has two central columns and two outer pilasters connected to the main section of the house. The house also has single story wings on either side of the central portico.

The house stayed in the Cotton family until 1893, when they sold it to J. W. Woodruff. He in turn sold it in 1896 to the Catholic Diocese of Green Bay, which used the home as an orphanage until 1933. In 1941, the house was moved from its original location to the top of a hill overlooking the Fox River. The Cotton House became one of the first buildings incorporated into the Heritage Hill State Historical Park in the 1970s.

The Cotton House is still part of Heritage Hill and is open during normal visiting hours.