Scaling does not equal capability, at least for now

• because compute is not standardized

But in the future imagine a universal kernel

• you could tell the kernel to deploy a model from a paper and it handles everything on its own
• has not happened because lack of incentive

But in the short term, everyone faces a market share loss if we standardize compute. important to prioritize long term.

how do i compete with less resources?

To do actual frontier reserach you need anywhere from 4-20 thousand chip cluster (couple million dollars).

Here is advice:

Fundamental algorithms research

• focus on research fundamentals / new architectures

Piggyback on open models

• take open weights and do your own extended mid training or RL with specialized datasets

Focus on new data

• Create unique data representations
• You can literally do it on a mac. here’s an idea: combine personal data (notes with sensor data or calendars), feed as context to AI and it creates new forms of intelligence

Understand why a model does what it does

• Currently models are vulnerable to jailbreaking and passing guardrails

• A better systems level understanding is needed.

Focus on verifiable domains

Material science, physics, robotics, superconductors

• Take general model and deploy it in specific physical or industrial system

• non verifiable domains (creative writing) is hard. but curated datasets help a lot

Action prediction is the new primitive

Instead of predict next token/video frame - predict next keyboard stroke or robot arm movement

• This is the bridge to physical world

in an era where open models are really good

• the context you provide is IMPORTANT
• If you give multimodal data, the model can reason across those modes too. For example model can reason what you were thinking, to how your body was reacting, and what you were doing
• You increase intelligence on these specific examples, something open models will never provide