The point is that if you have a bad development process, people are going to have issues. And these issues are real.
Even though you have no "obligation" to solve these problems that you have created for other people, you shouldn't be surprised if people bring them up, or perhaps even fork the project with a bunch of other people who also have problems with you.
Sure, the community does not own your time, but neither do you own the community. A community is fully within it right to do something else with their time, or convince other people to contribute to a different project.
Publishers of open source code don't create problems for other people. People who accept that code into their projects assume those problems for themselves.
If an open source package has bugs in the forest and nobody is around to install it...
After yesterday's NPM fiasco sorry but it is your project. You should fix the problems or don't release it out in the world.