It's not strictly required by the definition of open source, but....
A) If you don't provide builds and successful building is more involved then ./configure && make && make install, then you're pretty user-unfriendly.
B) If you aren't providing builds for target platforms then you probably aren't building for target platforms, which means part of your software has zero test coverage. Again, not a requirement, but it's fair for people to count that as a negative.
This QT version or that? Hunt down some weird dependency. Find that that dependency clashes with a more recent version, but you can't downgrade. Oh, oops your app depends on a quirk in glibc v x but your system only has v y so now you have to figure out how to run two different glibc's without conflicting with each other, and better not make a mistake during that installation or your system may never boot again.
Complex software can be quite a pain to build properly, more-so if you want to target multiple different architectures or operating systems.