One of the first settlements in the Town of Bayview was known as the Village of Houghton. Complete with its own train depot and post office, the village sprang up above the brownstone it began to quarry. The Hartley Quarry, Smith Babcock Quarry and Prentice Quarries all opened in the 1880s. Men with stone cutting skills were necessary laborers in the quarries, and the community grew. The "Great Monolith" stone was cut from the Prentice Quarry and destined as an exhibit at the Chicago Worlds Fair. Many local buildings in the City of Washburn are made from these brownstone quarries. The quarries closed in the mid 1890s. Many were filling with water, and many automatically filled with water as springs were tapped while cutting. The swimmable waters of the Bodin Brownstone Bowl led to the creation of the World's Log Rolling Championships during the early 1920s. This event brought thousands of people to the quarry's edge for several years. People also settled along the lakeshore, fishing for their livelihoods. Some of their old homesteads, boathouses, docks and cribbing remain visible on the water's edge.

In the 1890s, immigrants from Finland began to settle among the stumpage of the forests west of the Sioux River. Farming was the primary occupation in the "Finn Settlement." The men worked the fields sharing horses to make a team, while the women tended the herds and children. The Finns were great for building halls for meetings and gathering places, and one was erected where the Finn Memorial stands today. Initially a wooden bridge was built to cross the Little Sioux River as it was the primary passage out to the railroad (basically following the lakeshore north and south). The present bridge built in 1928, was built with labor from the Finn Settlement.

Early settlers of the Town of Bayview had to work long, hard days to make ends meet. When the week's work was done, there was always room for fun, and the need for communion of neighbors always evident. The Town schoolhouses and Finnish Workers Hall were often the sites of community dances, pie socials and school programs. Women's groups formed, not only to provide fellowship, but often were clubs with focus. The shoreline women between the Sioux River and Houghton Point formed the Sunshine Club. They gathered to visit, but were also there to support their community and those of greater need. Similarly, the women of the Finn Settlement also met as the Homemakers Club. Many of the children were also organized and continue to be a part of the Friendly Valley 4-H Club.

The educational needs of the children were met through the creation of four rural schools, years before the formal school district was created in 1913. The Gibson School (also called the Finn School) started in 1903-1904. The Houghton School also was used as a Town Hall and is still standing on its original site. The Sioux River School was built on the corner of Kjarvick and McCulloch Roads. And after the closing of the Roosevelt School, the building was used as a Town Hall until it was replaced by the current Town Hall in 1974. Note is made of a Pike Quarry School operating 1916-1917, but no further references exist except Esther Olson as teacher. Records show that these schools closed about 1938-39 when the City of Washburn took over the education system.

The Town of Bayview was formally created by Chapter 89, Laws of 1913 from sections of the Towns of Washburn and Bayfield along with a section of the City of Washburn. The first Town meeting was held in 1913 in the Houghton School and officers were elected. The Town's assessed value was set at $270,000. For the next few years, petition requests from freeholders within the Town were heard, and the laying out of roads began.