It was a combination of things, the Perl 6 announcement was one of the smaller parts actually.
- TIMTOWDI lost against "batteries included", and it lost big: It's not so much that the simpleness of the Python language won over Perl, the real and absolutely major problem at that time was the success of CPAN, which "killed" Perl 5. Perl 5 would have needed ONE object system, ONE base exception class, ONE way to do function signatures. Instead at the time, it came bundled with a bunch of Perl 4 utilities which nobody dared unbundling, CGI and a handful math functions. Being able to pick is nice, but everybody else did pick, too, and so if you used any CPAN dependencies, you soon had 3 incompatible object systems, 4 type systems, 5 exception systems co-existing within your project. A problem Perl has to this day and the reason why my team tends to re-implement stuff instead of pulling in dependencies. Python's "batteries included" approach easily won out, because stuff worked with each other and you didn't burn then scarce RAM on having different implementations of the same thing.
- Perl started to lose at the low-cost hosters against PHP already pretty early. It was easier during a certain time period to have a halfway performant PHP installation than a performant Perl installation. Hosters usually only offered CGI for Perl, not mod_perl, while PHP usually was offered as FastCGI module or mod_php which supposedly was easier to install. No low cost hoster wanted to mess with mod_perl, because it always was a hassle to setup.
- Speaking of mod_perl, things in the web dev sector still stayed pretty stable as long as Apache 1 ruled. People had to use mod_perl, but then you had a pretty performant solution. Then Apache 2 came around but mod_perl 2 wasn't ready for a long time (honestly I don't know whether it ever became ready). The Perl community at that time appeared entirely unprepared for that. While Python users were accustomed to run standalone Python servers behind proxies, Perl always was relying mostly on mod_perl. But threading in Perl stagnated and ran against a wall, and Apache 2 preferred multi process multi threading modes versus the simpler prefork multiprocess model that Apache 1 and mod_perl 1 ran on. This was the time were you could basically feel the creaking and everything breaking apart: suddenly there was no clear path forward, with mod_perl2's stability issues and the general threading stability issues of Perl 5.
It took years for the community to recover from that and focus on things like the Python-inspired PSGI/Plack as a way forward.
So, how does Perl 6 fit into this? Yes, in the early 2000s we did think, it would be just around the corner and be a clear upgrade path and solve everything wrong with Perl 5. Nice fantasy at the time. But at the same time, people were busy with far more pressing immediate problems, like how to migrate away from mod_perl1. When Python's WSGI and soon after that, Ruby on Rails came, they pressed directly into the vacuum left open by the mod_perl 2/Apache 2 disaster and this is actually what sealed the deal, IMHO.
I am not aware of any way (back then) to run mod_perl, which was powerful enough to facilitate developing Apache modules directly in Perl, safely in the shared hosting environments that most people used in the 2000s. If you wanted Perl, your options were either classic CGI, with the overhead of script recompilation and a new perl process spawned for each individual HTTP request (even if initially handled by the same Apache process), or paying up for a dedicated server/VPS just so you could run mod_perl.
How the Perl community thought this was an acceptable state of affairs, especially given how big of a driver webdev had been of Perl uptake during the .com bubble, when Perl was affectionately called the "duct tape of the web," and how they just sat by and let PHP eat their lunch, I'll never understand. But the attitude of the Perl community then was this mix of old-school UNIX/USENET "RTFM" culture and a dubious sense of cockiness (instilled through catechisms and catchphrases, like JAPH), so they weren't going to bend-over backwards to attract and accommodate new blood, expecting Muhammad to come to the mountain.