Spain and the colonies are not the same. As the Spanish colonies (relatively) prospered, the Spanish monarchy used that not to strengthen the Spanish economy but to fund a series of destructive wars. Colonial wealth did not make Spain a stronger country in the long run, like lotto winners who end up in bankruptcy. The habits and institutions of a healthy modern economy did not necessarily do better because of slave-based colonial wealth. And England did start late -- the Spanish had what was considered the best part of the New World for a hundred years, and the French the most valuable part of continental North America, when the English got started as a serious New World colonial power (and I believe the first permanent English settlement in the New World was at Jamestown, although you might be referring to something else). That the English seized colonial possessions from rivals doesn't really change that.
"You say that, but yet the industry went from being the most profitable in America to collapsing in the span of a couple of decades. It's about on par with the oil industry collapsing overnight today. "
Can you explain what you mean? All Americans farmers faced agricultural problems as over-productivity drove prices down. That's part of what caused the Populist revolt, which wasn't just a southern thing. Cotton had particular problems -- growing competition internationally, etc etc -- but it wasn't uniquely bad except to the extent that the South, with its tenancy and lack of capital and race tension and so on, was uniquely bad. The first AAA contracts in the New Deal (admittedly a bit after what we are talking about) included cotton but also wheat and hogs, with corn and tobacco and maybe something else added later. If anything, slavery hid cotton's lack of profitability pre-Civil War by limiting those who participated in them.
"No you took what I said out of context. It is curious why the handful of slave-owners produced most of the ag production in an upper southern state like Kentucky if it supposedly was not profitable. "
I don't quite understand, But maybe to clarify, the key distinction is not simply profitability, but relative profitability. Free laborers are (usually) more profitable than slave ones, but southerners continued to use slaves for a variety of non-economic, as well as economic, reasons.