1 JSM probably was the found of modern liberalism in my mind
2 You read a lot of earlier liberals and you'll find ideas that you can argue are modern liberal ideas (some readings of John Locke have him as a defender of an extensive welfare state, for instance)
3 classical liberals are more like libertarians or neoliberals, yes, but still you may find some of the racism surprising, given lots of the stuff on human rights, and various other general treatment of humans
4 A lot of other historical thinkers, that some would describe as important founders of modern liberalism, that one might associate with ideas about general mankind, such as Kant, Hume, whoever, said horribly racist things too.
5 I'm rambling and going on, but if you're interested you can read Charles Mills on "herrenvolk ethics"