This is a great idea. The first version of this could be a human-assisted AI, where an ai makes moderation choices (with a confidence interval) and the choices it makes can be supervised and overriden by human moderators when it gets things wrong. Over time the AI can be retrained to make better choices. Kind of like a spam filter with more knobs.
The hard thing early on might be getting getting started with good training data. But chatgpt might already be good enough to make reasonable choices today with a good system prompt.