Chapters

We've gathered the most important knowledge on AI safety into a single, comprehensive resource.

8 chapters 15h 9min

Ready to begin?

Start reading

Begin with Capabilities and work through at your own pace.

Table of Contents

8 chapters covering the foundations of AI safety, from core concepts to cutting-edge research.

8 chapters 15h 9min
Chapter 1

Capabilities

Download chapter audio
1.1 Introduction 3 min
1.2 Current Capabilities 14 min
1.3 Foundation Models 8 min
1.4 Defining and Measuring AGI 19 min
1.5 Leveraging Scale 11 min
1.6 Forecasting Timelines 9 min
1.7 Takeoff 12 min
1.8 Appendix: Forecasting 12 min
1.9 Appendix: Takeoff 9 min
1.10 Appendix: Expert Surveys 8 min
1.11 Appendix: Discussion on LLMs 21 min
Chapter 2

Risks

Download chapter audio
2.1 Introduction 4 min
2.2 Risk Decomposition 8 min
2.3 Dangerous Capabilities 15 min
2.4 Misuse Risks 31 min
2.5 Misalignment Risks 20 min
2.6 Systemic Risks 22 min
2.7 Risk Amplifiers 16 min
2.8 Conclusion 2 min
2.9 Appendix: Quantifying Existential Risks 2 min
2.10 Appendix: Forecasting Scenarios 11 min
Chapter 3

Strategies

Download chapter audio
3.1 Introduction 3 min
3.2 Definitions 7 min
3.3 Misuse Prevention Strategies 24 min
3.4 AGI Safety Strategies 31 min
3.5 ASI Safety Strategies 28 min
3.6 Socio-Technical Strategies 23 min
3.7 Combining Strategies 4 min
3.8 Challenges 7 min
3.9 Conclusion 2 min
3.10 Appendix: Long-term questions 10 min
Chapter 4

Governance

Download chapter audio
4.1 Introduction 2 min
4.2 Governance Problems 7 min
4.3 Compute Governance 12 min
4.4 Systemic Challenges 17 min
4.5 Governance Architectures 29 min
4.6 Implementation 11 min
4.7 Conclusion 3 min
4.8 Appendix: Data Governance 3 min
4.9 Appendix: National Governance 11 min
Chapter 5

Evaluations

Download chapter audio
5.1 Introduction 5 min
5.2 Benchmarks 16 min
5.3 Evaluated Properties 11 min
5.4 Evaluation Techniques 20 min
5.5 Evaluation Frameworks 10 min
5.6 Dangerous Capability Evaluations 33 min
5.7 Dangerous Propensity Evaluations 15 min
5.8 Control Evaluations 11 min
5.9 Evaluation Design 12 min
5.10 Limitations 16 min
5.11 Conclusion 1 min
Chapter 6

Specification Gaming

Download chapter audio
6.1 Introduction 1 min
6.2 Reinforcement Learning 7 min
6.3 Optimization 4 min
6.4 Specification Gaming 11 min
6.5 Learning from imitation 12 min
6.6 Learning from feedback 22 min
Chapter 7

Goal Misgeneralization

Download chapter audio
7.1 Introduction 3 min
7.2 Multi Objective Generalization 9 min
7.3 Learning Dynamics 12 min
7.4 Goal-Directedness 18 min
7.5 Scheming 29 min
7.6 Detection 14 min
7.7 Mitigations 32 min
Chapter 8

Scalable Oversight

Download chapter audio
8.1 Introduction 3 min
8.2 Oversight 9 min
8.3 Task Decomposition 6 min
8.4 Process Oversight 11 min
8.5 Iterated Amplification 13 min
8.6 Debate 43 min
8.7 Weak-to-Strong (W2S) 10 min

Ready to get started?

Begin with Chapter 1: Capabilities. Self-paced, always free.

Start reading