I faced a similar problem when I didn’t find a sanely sized flagship non-bloated (though that’s essentially an oxymoron in Android world including the Pixels) Android phone sometimes back and I decided to vote with my wallet and I bought the first iPhone SE, then 7, and now 12 Mini. I am an Android developer and the way the OS is designed (UX and privacy wise; even the stock one) I don’t really see myself moving back to Android anytime soon but I wish there was a a flagship normal sized phone.
I dislike Apple for a lot of things but in this duopoly the size of a phone is such a fundamental characteristic that you’re out of options anyway.
I think most of the people are “swiper users” — the “content consumers” — so they want big phones and OEMs are simply making what the near 100% majority (yup!) wants.