Security and privacy concerns don't even come in to play (although these do make the apps even worse)
This is a shame, because I had good experiences with my old Hue, when they still did remotes.
Something like this: https://ae01.alicdn.com/kf/HTB1bweoSVXXXXbHXFXXq6xXFXXXG/New...
Maybe I missed it, but from the material that was available to me without actually buying one of these things, this seemed impossible: the boxes and stores didn't have any remotes, and I couldn't find anything on the website. If it is possible, then it's hidden pretty well and people are steered towards that app pretty strongly.