Like the OP, I switched from Android (Pixel 3a) to an iPhone SE 3 specifically for the smaller form factor. After using it for over a year, I've found the trade-offs in battery life and camera quality are too significant for my daily use.
These limitations aren't an issue when I'm at home or my desk with easy access to a charger. However, they become acute the moment I'm out for the day. For example, using GPS for navigation or connecting Bluetooth accessories becomes a liability. I can't rely on the phone to last. Also, photos are noticeably more pixelated, and the quality drop-off is clear compared to larger, contemporary phones.
This thread is evidence that the niche for small phones exists. But it's for people willing to accept these compromises by carrying a dedicated camera, a power bank, and using wired peripherals. For me and as the market suggests for most consumers, small phones just doesn't work out as reliable all-in-one devices. I'll probably wait till early next year to pick up one of the new iPhones after they iron out the initial kinks.
My conclusion is the same as the author’s: it’s a matter of you get what you pay for. The demand here is for a small premium phone. This would come with a good set of cameras.