Historical Marker #2367 in Petersburg highlights the history of Bullittsburg Baptist Church. In 1794, the church was organized by Elders Joseph Redding and John Taylor along the north bend of the Ohio River. The congregation consisted of eight members, most of which had moved from Clear Creek Baptist Church near Versailles, Kentucky. Originally called The Great Bend of the Ohio Church, the name then changed to The Baptist Church of Christ at Bullittsburg, and finally to its current name. It was the first Baptist church in the area and was received as a member of the Elkhorn Association in 1795. In 1803 Bullittsburg along with 8 other churches became charter members of the North Bend Association.
Well-known pastor John Taylor moved to the area in April 1795 and joined the membership. Though invited to join the congregation as a pastor, he declined, in deference to the then current pastor, Brother Deweese. Taylor wrote about his experiences at the little church in his History of Ten Churches published in Frankfort, Kentucky, in 1823.
The church is located near the log home of George Gaines, who originally donated the land on which the church stands. . The present structure was built in 1819 at a cost of $1,850. The church cemetery, bounded by a wrought-iron fence, was established in the 1860s. The church continues to be active with worship services weekly.