The old Something Awful forums did something similar. If someone posted something that was unacceptable, the comment would generally stay and the comment would get a tag saying “the user was banned for this post”. They also had a moderation history so you could go back and see mod comments on why they gave bans/probations.