For every authoritative-sounding, in-depth explanation, there is an equally plausible, yet conflicting and contradictory alternative.
One is gyroscopic forces. Ever picked up a spinning hard drive? Notice that it feels strangely hard to turn in some directions? Same idea.
The other is the feedback loop consisting of the bicycle and its rider.
If the bike is stationary, it's hard to keep it upright because you have no assistance from gyroscopic forces. At low speeds, you have some assistance but not enough. At higher speeds, the bike wants to maintain its current orientation, and it's easy to feed in the slight corrective forces needed to keep it that way. Hop off the bike and it will keep going until something causes it to veer off course.
You can throw a ton of math at it, as in the paper mentioned elsewhere in the thread, but at the end of the day, gyroscopic forces and negative feedback are all that's necessary. The Schwab paper appears to show that the gyroscopic forces aren't necessary, but no bicycle in the real world is ever going to work that way except in rare corner cases, e.g., if you're one of those riders who can stay upright at a standstill.
For example, gyroscopic-forces-as-stabiliser don't need the Schwab paper's "ton of math" to be undermined. A simple counter-rotating wheel was used empirically at Cambridge to show as much, alongside notes that gyroscopic forces are relevant to the dynamics of a loaded bicycle, but misconstrued; far from assisting to hold it upright whilst ridden, they induce instability at the beginning of a change in direction, and more so at speed, a phenomenon (counter-steering) familiar to cyclists and relied upon by motorcyclists.
Then there's a simpler observation that can be made: people have to learn to ride a bicycle. The fact it stays upright when rolling unloaded, but not when loaded, is indicative of how small the gyroscopic effect is, not how significant it is. Ergo, that argument would suggest, it is tiny shifts body position that contribute all of the stability.
And then others, proposing further explanations, etc etc ad nauseum.
I have come around to the view that in fact they don't stay upright, and they are almost always falling over, but in a many-branched configuration of the universe our observer effect sends us preferentially down the vanishingly unlikely path where they didn't, and there are an uncountably many alternatives of Me that have nothing but knee scars to show for it.
When you learn to ride a bike, you're simply training the feedback mechanism. Much like a PID controller, your brain has to keep track of the amount of error and null it out with proportional and integral terms (at least). Once those constants are dialed in, it's "just like riding a bike" -- they're yours for life.
Then there's the matter of learning which way to lean so that the gyroscopic instability inherent in turning doesn't send you into the nearest ditch...