EDIT: and would you then also review every commit to make sure nothing bad gets introduced? No, at some point you have to place trust in the vendor, the developers, independent audits, etc.
https://tests.reproducible-builds.org/debian/reproducible.ht...
We're making great strides into software being completely deterministic. The Bitcoin project for many years has had completely deterministic binaries and a ceremony process for GPG signing the output with many individual parties.
Assuming all three match, you know that the binary matches the source.
Someone who is more technically inclined can probably go into more detail on this.
How do you know you're not living in a computer simulation in which the operators can access your data without any backdoors whatsoever?
[1]: https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref...
For the curious: we actually were intentional about finding these, by compiling many programs with the same parameters on different machines. One with a 32 bit OS and toolchain, the other one on a 64 bit machine, and we would get alerted when we produced binaries with a different checksum.
Trying to get a bit-to-bit equivalent of a binary lifted from the app store sounds challenging to say the least.
Cartesian doubt becomes pointless at some point. If you're worried that the deep state has implanted microchips in your brain to prevent you from analyzing signal, it probably doesn't matter because at that point they wouldn't need to hack signal to get to you.
A less snarky and more realistic answer is: threat models and risk assesment. (Non-divine) adversaries generally have limited resources. The limit may be high, but its still there. You can realistically worry about a government coercing a service to hand over keys, because that's easily within their power. On the other hand, having a giant conspiracy-trusting trust style-where every compiler & microchip has a backdoor that is inserted into every tool ever compiled, is a bit unrealistic. It would take thousands of people to be in on it to pull it off, spread across many countries (who hate each other) over at least 50 years. Having that many people, especially academics, keep that type of secret for that long is basically impossible. If they could do that, it would be child's play to have most of the protestors be gov agents, so if you think this is realistic, worry about that first. Anyways, in my judgement governments don't have that kind of power, so its probably not something to worry about.
So, to conclude, estimate the level of power and influence you think your enemies have, and then take steps to rule out the possibilities that your enemies have done the things that are theoretically in their power to do. Start with the possibilities that are most likely multiplied by how bad it would be for you (liklihood*severity = risk)
See also Guix, which provides tools to challenge servers providing binary packages to see if they match a locally-built version: https://guix.gnu.org/manual/en/html_node/Invoking-guix-chall...