It is absolutely a criminal violation. Also there are 2 different acts: the crossing of the border (which even a US citizen can be charged for crossing illegally) and being in the country without permission.
Also misdemeanors are still considered serious crimes. I don't get where this hand-waving "because it's just a misdemeanor" comes from.