If I was evil, I wouldn't have a totally separate source tree and binary that I shipped; I'd have my CI process inject a patch file. As a result, everything would work as expected - including getting any changes from the public source code - but the created binaries would be backdoored.