Cudahy’s first Catholic congregation—St. Frederick’s--was established in 1896. Despite the existence of St. Frederick, which primarily consisted of Germans, the area’s Polish population wanted a church of their own. Four years later, Holy Family Roman Catholic Church was established as a mission church of South Milwaukee’s, St. Adalbert’s Roman Catholic Church (est. 1898). In 1901, the Holy Family congregation built their first edifice at the northeast corner of E. Underwood and S. Swift avenues at a cost of $3,000. In 1905, a rectory was built and, in 1910, a school was erected; the latter structure of which also provided accommodations for the Felician Sisters who were called upon to teach the Holy Family students.

Although the growing congregation was in need of a larger church by the early 1920s, their new, Neo-Gothic Revival-style house of worship, built at the same location as the 1901 structure, was not completed until 1931. The need for additional classroom space grew such that it prompted the 1954 construction of a $75,000 purpose-built, Colonial Revival-style convent along E. Hammond Avenue. Finally, in need of modernized educational facilities, the 1910 school was demolished and a new 1950s Contemporary-style structure was completed in 1958.

The convent was sold in the early 1980s, but the congregation continued to worship in the church until 2000, while the school was utilized as a parochial school until closing in 2004 and the former rectory was used as a rental. Despite the 2018 razing of the rectory by the congregation, the remaining three Holy Family Roman Catholic Church buildings—church, convent and school--combine to stand as a very good and highly intact religious complex (in Cudahy), both inside and out.