Most of the land that today encompasses Milltown was owned by Spoke Oke Harjo, a Creek Indian. His beautiful valley was located on a stream he called “Cohoasanocsa” and later twisted by the white man to Chickasanoxee. His small log house was on the south bank of the stream just west of Hutton’s Ferry. Spoke Oke Harjo was spared the ordeals of removal. He died around 1836 and was buried on the banks of his beloved stream. White men administered his estate and the Federal Grant for his land was issued to Ware Daugherty & Company which was an infamous land speculator of that day.

The first white settlers in the area settled one mile west of the present town in the vicinity of todays Mount Hickory Church. In 1832 and 1833 came Reverend Isiah Durham, Booker Lawson, Aaron Strother, Michael A.J. Carlisle, Lazirus Atkinson, Thomas Smith, William Blackburn, Elijah Phillips, Gideon Leverett and James Webb Sr. In the spring of 1835 William Graggs built a log house just south of the creek. In partnership with William W. Carlisle he constructed a dam and grist mill on the creek adjacent to the present town.

Originally Milltown was called “Graggs Mill” but a split developed and the members of Mount Hickory Church withdrew from Graggs Mill and constructed a “meeting house” across the creek from the mill.

In 1839 Jesse B. Robinson married a daughter of William Carlisle and built a saw mill downstream from the grist mill. The town built around the church and the mills of Robinson, Graggs & Company. Among the businesses that opened in the 1840’s were Taylor & Robinson, Gideon Leverett, Robinson & Carlisle, Dozier & Thompson, J.D. Cook, Robinson & Greer, McCune B. Taylor, Meriwether & Wood, Abner Penton, and Lewis Schuessler. Shortly after this came James Cheek who was a tailor, Richard Edge who was a carriage maker and Brayton Johnson a blacksmith.

In 1846 the name of the town was changed from Gragg’s Mill to Woodville, but since there was another Woodville in Alabama the name was changed once again in 1848 to Milltown.

Just prior to the Civil War Rev. David Moon built a cotton gin on the creek and on the tanyard branch which borders the business district on the west, Jesse B. Robinson constructed a shoe manufactory in conjunction with his tannery. David Greer operated this shop for several years and examples of his handiwork are preserved.

On a crest overlooking the town, William Carlisle built a rambling Greek revival mansion in the fall of 1839. For over twenty years he operated his home as a hotel and tavern. This home still exists. A picture of that home is shown below.