Looking to improve methods with just human knowledge is a waste of time

• AI researchers try to build knowledge into their agents, which helps in short term
• But plateaus in long term and inhibits progress

Eventually a breakthrough is achieved which scales computation by search and learning

• General purpose methods are powerful (methods that scale with increased computation even as available computation becomes great)
• The two methods that seem to scale in this way are search and learning

Overall We should invest in meta methods that allow computer to capture the arbitrary complexity, to find good approximations. We want AI to discover like we can, not contain what we discovered

My thoughts

Cool piece by richie here. I like the way he writes, and how he decomposed the original scaling laws into simply “more compute = better”. what he is saying is that its just about hardware - literally ANYTHING that can use (leverage) more compute is a good strategy.

So its a good thing to keep in mind that having some type of human experience that you resonate with might not work with AI. taking the chess example from richie here, people’s human intuition in building chess bots failed in comparison to a simple search based prorgram that ran on specialized hardware. People said if they had that much compute their theory would be better, but it did not do better.

The main point is to SCALE COMPUTE. just run a bigger search algorithm , and allow more learning to happen.

From this paper I get the path AGI happens when we make AI discover relentlessly.