William Ellison was one of the wealthiest men in the South as well as being a black, former slave. He owned cotton gins, plantations, and 68 slaves… and from accounts of the time, he wasn’t very nice…

At the peak of slavery in the United States, large numbers of free Negroes owned black slaves; in fact, in numbers disproportionate to their representation in society at large. In 1860 only a small minority of whites owned slaves. According to the U.S. census report for that last year before the Civil War, there were nearly 27 million whites in the country. Some 8 million of them lived in the slaveholding states. The census also determined that there were fewer than 385,000 individuals who owned slaves. Even if all slaveholders had been white, that would amount to only 1.4 percent of whites in the country (or 4.8 % of southern whites owning one or more slaves, however, around 30% for free blacks owned slaves).

In the rare instances when the ownership of slaves by free Negroes is acknowledged in the history books, justification centers on the claim that black slave masters were simply individuals who purchased the freedom of a spouse or child from a white slaveholder and had been unable to legally manumit them. Although this did indeed happen at times, it is a misrepresentation of the majority of instances, one which is debunked by records of the period on blacks who owned slaves. These include individuals such as Justus Angel and Mistress L. Horry, of Colleton District, South Carolina, who each owned 84 slaves in 1830. In fact, in 1830 a fourth of the free Negro slave masters in South Carolina owned 10 or more slaves; eight owning 30 or more.