It would be easily dismissed if said by someone much less accomplished. In this case, I just take it as a frank statement of belief.
Regardless, I think it's safe to say Minsky was a great intellect, of a caliber not often seen.
In one of the lectures, Minsky told a story about how when he was at Princeton, Oppenheimer invited him to lunch. When Oppenheimer brought Minsky to the lunch, there were two other people there - Gödel and Einstein. Talk about brainpower, that must have been an interesting table conversation.
I would have preferred Asimov to say something like "Sagan and minsky are the brightest minds I've encountered in my life." And if you think Asimov is super smart, then you'll respect Sagan and Minsky that much more. And you'll respect Asimov too for not being so egotistical. It doesn't even matter if it's actually true or not.
Minsky, in my experience, demonstrated humility when talking with or listening to him. For that I gain admiration beyond the talent.
But that wouldn't make good banter.
...He also said that he thought McCarthy was an idiot. :)