I've been using XCode for 10 years. For me, it's only improved and I don't have any real pain points. They are definitely fixing bugs. I make software for iOS, macOS, car play, and apple watch.
Sure sometimes I've got to reset or clear a cache, but this has never stopped my day.
What is so horrible about XCode?
it constantly disconnects, requires restarts and other nonsense techniques. i legit do not know how you can not be running into these problems if you are developing on those platforms.