Being "smart" isn't a binary, and can't describe someone in any all-encompassing ways. Someone can be smart about investing in startups but stupid about understanding social discourse around marginalized groups.
I am not surprised at all that Graham is both of those things.