If the black bloc was all of antifa, it would just be called antifa. Outside the black bloc, direct action covers a lot of things, from just organizing all sorts of things (not just protests), art and being loud, to vandalism and even violence.
Just take all the sorts of things that were done in human history while claiming it was for freedom, both bad and good. Does that make any person doing thing X in the name of freedom responsible for what a person doing Y in the name of freedom? You can call those people a "group" all day long, but that doesn't mean they're actually a group in the sense of voting on their values and actions.