Joseph Rolette built this house around 1837, as a separation settlement agreement for his wife Jane Fisher. Michel and Dometille Brisbois raised Jane Fisher until she married Joseph Rolette in 1818, but by the 1830s Jane and Joseph's marriage had failed, resulting in the decision to live separately. Jane and B.W. Brisbois, her cousin and trustee for the separation, lived in this house until Jane's second marriage to fur trade baron, Hercules L. Dousman in 1844.

The Brisbois house is constructed of locally quarried limestone and situated on St. Feriole Island, overlooking the Mississippi River. The house is a rare pre-statehood example of the Federal Style, then considered to be the height of fashionable taste. Two and a half story, rectangular shaped homes were typical of Federal Style houses, which usually featured central halls and dominant staircases. Some unique aspects of this house include the white pine interior woodwork and the lime mortar and plaster made of lime extracted from burned clamshells harvested from the Mississippi River.

The Wisconsin Historical Society acquired the house around 1955; it is now part of the Villa Louis historic site. Visit the Villa Louis website for more information.