Compared to your ideal specifications, my wishes are: support for microSD card storage; battery that easily and reliably lasts a full day with moderate phone usage; fingerprint sensor, not necessarily on the power button; camera decent, not necessarily great (I don't care that much about low light performance, for example).
I'm tempted to sign up even with the specifications as you list them though. Missing microSD card support could be the major dealbreaker. Or alternatively some other user-friendly reliable method of getting lots of files from my PC to the phone's storage, but so far I haven't found any. Early Android versions supporter USB mass storage and that worked pretty well, but the transfer method implemented on newer versions is very slow and never works reliably for me.
And for the other guy a 3.5mm jack and for a third a physical off switch and look at that we have too many dealbreaker features for the form factor.
Power users tend to have more dealbreakers than the average consumer. Anecdotally, it seems power users prefer smaller phones. This might be what kills the small phone factor.
Women are also overrepresented in the Really Big Phone market, and wield them two-handed.
They also trend heavily iPhone in the US market, but that leaves plenty of alpha for the manufacturer who serves the actual market for small-form-factor Android phones.
But if you have to keep your phone in a purse anyways, why not just get a big one?
So mostly the people in that market who still care are the ones who can’t or don’t want to carry a purse, which is also a smaller market. (I’m in this market though, so i am sad)
I’m not saying that this is people’s preferred choice, I’m saying it’s a logical decision given the choices available that seems counterintuitive from first principles (and assuming a market with real choices).
I am the small-phone-lover this article is addressing, and I did sign up to their list - I have an Xperia XZ1 Compact and no plans to upgrade because there's nothing to upgrade it to - but my biggest complaint about the Compact is that it's too big already. I'm a not-quite-six-foot man and I can't reach to buttons in the corners one handed. So why bother? It seems that my preference is not entirely rational after all.
The actual market for a truly one handle-able phone is enormous. It's just not possible to fit modern phone functions into a package that small though.
Who will pay flagship prices for a phone with 3 hours of battery life?