General Machine Learning Notes

When they beat Kasparov in chess in 1997, it was accomplished via brute force search instead of using human understanding of how to play chess. Similarly, Go was solved via a combination of search and self learning. The same happened in NLP and CV were the dominant techniques now discarded the initial direction of trying to adapt human understanding to achieve performant models. Yet every time the best approaches have been via scaling search and learning.
“…the bitter lesson is that the actual contents of minds are tremendously, irredeemably complex; we should stop trying to find simple ways to think about the contents of minds, such as simple ways to think about space, objects, multiple agents, or symmetries. All these are part of the arbitrary, intrinsically-complex, outside world. They are not what should be built in, as their complexity is endless; instead we should build in only the meta-methods that can find and capture this arbitrary complexity. Essential to these methods is that they can find good approximations, but the search for them should be by our methods, not by us. We want AI agents that can discover like we can, not which contain what we have discovered.“