In all honesty I think this mindset is precisely why we've ended up in this massively polarized situation. When you "take the speaker's identity and character into account", you're obviously ("you" here being people in general not you in particular) to lend more credence to someone who's priors match your own. In other words, you give the benefit of the doubt to people like you, and interpret more uncharitably the words of someone who you think isn't like you. This creates a destructive cycle where everything eventually devolves into an echo chamber, increasing polarization and creating more echo chambers.
That's why I like how one of the principles behind Hacker News is to employ the "principle of charity" – try to interpret people's words in the best possible light, regardless of their priors or your own.
If you don't have the bandwidth to process all the arguments you're receiving... receive fewer arguments. Get involved in fewer shitposting threads on the internet. Have fewer arguments about politics at work. The solution is not to assume / reduce / summarize until the arguments become tidy little things you can stick in boxes, it is to just reduce your workload so that you can give the arguments you care most about the attention they deserve.
Likewise, you talk about devoting your attention to the most important arguments -- but how do you decide which arguments deserve that attention? You can't be doing that 100% independently, you must at some point be allowing other people to raise issues to your attention, and to shift your priors a bit by virtue of the credibility they have earned via their track record (of being right, of being honest, of caring about things you care about).