The reason is audiobooks: Having shorter chapters is a better user experience and audiobooks are really hot (money makers) right now.
This is just an anecdote though. I'm not an expert (only wrote one novel and my chapters were like ~2,500-3,500 words).
unless i found the right indie press or self-publish or already had a big platform.