zlacker

> it’s as if a great library, say the Library of Congress, refused to tell where they got their books and how they got their books and who chose the books and whether all the books they had were in the catalogue and available or some were held back, kept secret.

I think "proprietary" is a better descriptor for Google Search's inner machinations, than "secret". The general concept of engineering a search crawler is well-trodden. Many companies have done it, there are open-source examples, and Google themselves have written blogs about their own.

It would probably be more apt to say, we know where the books came from and how they were acquired, we just don't necessarily know how the archive shelves in the basement are arranged and we don't know which employee is responsible for organizing them and we don't have the source code to the library's LMS. (All of which is true, by the way, for the LOC.) Proprietary, not secret.