zlacker

My entirely biased opinion is https://www.mojeek.com/ - a traditional search engine crawler (as in, follow links on the web) that identifies its user agent. Dead Simple. The open web, you can search on it.