That's my point, you can't establish trust by checking everything yourself. So you delegate to other things as an approximation. In this case, Signal seems to be reputable, have competent developers and afaik no history of leaks or malevolence so I would rely on that rather than a half-assed source code review.