EDIT: and would you then also review every commit to make sure nothing bad gets introduced? No, at some point you have to place trust in the vendor, the developers, independent audits, etc.
https://tests.reproducible-builds.org/debian/reproducible.ht...
We're making great strides into software being completely deterministic. The Bitcoin project for many years has had completely deterministic binaries and a ceremony process for GPG signing the output with many individual parties.