It can be also a source of bugs where you hang onto something for longer than intended - considering there's no indication of something that might block in Go, you can acquire a mutex, defer the release, and be surprised when some function call ends up blocking, and your whole program hangs for a second.
But I do definitely agree that the dynamic nature of defer and it not being block-scoped is probably not the best