Signal encryption is its main feature (I think) and how easy it makes it (abstracts handling key transfer and all that), I'm just trying to think through... if I wanted nobody to read what I was saying , would I use an app/target as popular as Signal or something homegrown?
If you want to talk to one person, you can give them a USB key in person with a set of crypto keys and then use that to encrypt your messages over any transit method and it will be secure.
The hard part is the key exchange.
We could stack a hundred layers of encryption algorithms, and if just one of them works, then the whole stack is secure.
More layers of encryption doesn't really solve those problems.
In the meantime, the added complexity adds new places for errors.