Like the OP, I switched from Android (Pixel 3a) to an iPhone SE 3 specifically for the smaller form factor. After using it for over a year, I've found the trade-offs in battery life and camera quality are too significant for my daily use.
These limitations aren't an issue when I'm at home or my desk with easy access to a charger. However, they become acute the moment I'm out for the day. For example, using GPS for navigation or connecting Bluetooth accessories becomes a liability. I can't rely on the phone to last. Also, photos are noticeably more pixelated, and the quality drop-off is clear compared to larger, contemporary phones.
This thread is evidence that the niche for small phones exists. But it's for people willing to accept these compromises by carrying a dedicated camera, a power bank, and using wired peripherals. For me and as the market suggests for most consumers, small phones just doesn't work out as reliable all-in-one devices. I'll probably wait till early next year to pick up one of the new iPhones after they iron out the initial kinks.
I'm a small-phone person, and I don't think these _should_ be necessary. I'm fine with wired peripherals (and prefer them), but in 2025, with efficient chips, I don't see why we can't power a device much longer than 24 hours. What if it had decade-old hardware, and -- this is the bit I think is the problem -- the operating system and apps were efficient?
Same with a camera. It seems more about thickness than width; I don't believe it should be impossible to put a large-phone-format camera in a smaller phone. It may take battery space, but see above, we should be ok there these days.
That is only possible if we don't write the software with dozen layers of abstraction and gimmicky features (looking at you, Liquid Glass!).