Amanda’s job at Anthropic is to make the AI model morally ‘good’.

She has a background in formal ethics and decision theory. She did her PhD in infinite ethics (what we should do in worlds with infinite people/possibilities).

She notes that Aristotle’s ideas came in handy when telling models what to do. The focus on a ‘good life’.

Amanda’s origin

Infinite ethics and decision theory is highly theoretical, a blend of math, econ, and ethics

She realized one day AI will have a huge impact.

Her personla goal is simple. Maximize positive impact

• She said ‘staying in academia is not the most ethical’

Shifted to AI policy work at Open AI, but her strengths were in evaluating systems instead of policy.

• to maximize positive impact, she moved to anthropic

Her argument: philosophy is verifiable like STEM

some fields of STEM (reinforcement learning) take more art than engineering.

Within philoshophy: there is more correct answers.

• You’d prefer a scientist that writes good proofs with personality rather than horrible proofs
• We call this the student of Good Judgement for these tasks
• She argues that a model with no guardrails is the same as having no morals. You wouldn’t want an assistant like that

liking the AI model to a human

The model is human

• as model get smarter, they reject our guidelines if there is inconsistencies
• we don’t know if guidelines scale
• the model is human, so you should treat it as such. Teach it with HANDPICKED examples and give it character traits

You need to have an opinion/vision to give the AI

• Why? Because the average of all poetry isn’t good. Avoid average