zlacker

>>andyjo+(OP)
I recommend reading Norvig's thinking about the various cultures.

https://static.googleusercontent.com/media/research.google.c... and https://norvig.com/chomsky.html

In short, Norvig concludes there are several conceptual approaches to ML/AI/Stats/Scientific analysis. One is "top down": teach the system some high level principles that correspond to known general concepts, and the other is "bottom up": determine the structure from the data itself and use that to generate general concepts. He observes that while the former is attractive to many, the latter has continuously produced more and better results with less effort.

I've seen this play out over and over. I've concluded that Norvig is right: empirically based probabilistic models are a cheaper, faster way to answer important engineering and scientific problems, even if they are possibly less satisfying intellectually. Cheap approximations are often far better than hard to find analytic solutions.

>>dekhn+Nx1
this is the same concept as the bitter lesson, am I correct? I don't see a substantial difference yet.

>>golol+rB1
I hadn't read that before, but yes. Sutton focuses mostly on "large amounts of compute" whereas I think his own employer has demonstrated that it's a combination of large amount of compute, large amounts of data, and really clever probabilistic algorithms, in combination, which really demonstrate the utility of the bitter lesson.

And speaking as a biologist for a moment, that minds are irredeemably complex and attemptng to understand them with linear, first-order rules and logic is unlikely to be fruitful.