So, you can't trust the address in the "From" on an SMS or the "From" of a phone call.
That means a voice call to Signal would not work to validate phone numbers.
Problem: A system that enforces a monetary penalty to prevent sign up abuse while also not tying a users identity to said system.
Without doing some pain in the a crypto stuff it seems like there are no easy solutions other than the #