Mount Hermon Baptist Church and Berlin Baptist Church both were once united as one single church back in 1876. On December 28, 1925 the Mount Hermon church building caught fire and burned down causing the church family to hold services in the old Berlin Schoolhouse located on the right side of what is known today as Highway 302.

Controversy over where the new church building should be built sprung up. The decision was between two different locations. One location was between the railroad crossing and the field on the left of the road near Dead Man's Branch. The other location was what is now known as Highway 302 intersects Cedar Creek Road. The first location obtained 24 votes and the second location obtained 18 votes. On the second Sunday in May of 1927, Reverend K.D. Smith and the deacons were elected to submit the building plans for the new church building, but the pastor was given the final say in the location of the building. He chose the site where Highway 302 intersects Cedar Creek Road and this caused more controversy within the church. Unable to come to an agreement, a fraction of the members remained at the schoolhouse and the remaining members built Old Mount Hermon on the location where it sits today.

As the years went on, it became difficult to distinguish one church from the other, so New Mount Hermon changed their name in 1964 to Berlin Baptist Church.