Smaller size means smaller battery, but that's mitigated by the above. I want utilitarian. I don't want a phablet. I want practical and unobtrusive. The smartwatch was meant to replace the phone, but doesn't hit the right notes for me.
Today, bluetooth works quite well for me (I love not having cables... but it sucks that performance with a microphone is trashfire). 3.5mm adapters are cheap and easy when needed (rarely. I also have a $10 bluetooth->3.5mm in my travel kit that does get used once a year!). And with usb-c providing fast charging, I rarely feel like I'd benefit from battery swaps. I can give myself 50%+ in 30 minutes, with a portable battery that will power not just my phone, but any other device I run into. With Qi 2.2 releasing with 25W wireless charging, and magnetic coupling being standard now, you don't even need wires anymore. Carrying a bespoke phone-only battery seems like a massive downgrade today. (It also felt like a massive fire hazard!) Time to update your expectations!
Worth mentioning that battery swaps make water-resistance much much trickier to pull off. There' a real cost to battery-swappability.
I do wish we saw something like Ara, some phone modularity & extensibility. Fairphone has some modular parts, but it doesn't feel like an open ecosystem, and the parts dont seem super designed for expansion but more just replacement. I guess maybe Framework is doing the best work, albeit in a bigger form factor space, with their Expansion Cards, which are basically just a card form factor USB-C. Licensed CC-BY-4. https://github.com/FrameworkComputer/ExpansionCards