It's not strictly required by the definition of open source, but....
A) If you don't provide builds and successful building is more involved then ./configure && make && make install, then you're pretty user-unfriendly.
B) If you aren't providing builds for target platforms then you probably aren't building for target platforms, which means part of your software has zero test coverage. Again, not a requirement, but it's fair for people to count that as a negative.
But it is required by Free Software licenses. GPLv2 §3[0]:
> The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable.
GPLv3 §1[1]:
> The “Corresponding Source” for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities.