> sudo systemctl disable phosh.service
Or something like that. I'm sure it would be pretty easy with a bit of fiddling. However, you're right that you'd probably need a keyboard plugged in because the default TTY probably doesn't have a soft keyboard.