Chapters

We've gathered the most important knowledge on AI safety into a single, comprehensive resource.

8 chapters 11h 50min

Ready to begin?

Start reading

Begin with Capabilities and work through at your own pace.

Table of Contents

8 chapters covering the foundations of AI safety, from core concepts to cutting-edge research.

8 chapters 11h 50min
Chapter 1

Capabilities

1.1 Introduction 3 min
1.2 Current Capabilities 8 min
1.3 Foundation Models 7 min
1.4 Defining and Measuring AGI 11 min
1.5 Leveraging Scale 7 min
1.6 Forecasting Timelines 7 min
1.7 Takeoff 11 min
1.8 Appendix: Forecasting 6 min
1.9 Appendix: Takeoff 8 min
1.10 Appendix: Expert Surveys 1 min
1.11 Appendix: Discussion on LLMs 16 min
Chapter 2

Risks

2.1 Introduction 3 min
2.2 Risk Decomposition 5 min
2.3 Dangerous Capabilities 10 min
2.4 Misuse Risks 22 min
2.5 Misalignment Risks 16 min
2.6 Systemic Risks 17 min
2.7 Risk Amplifiers 12 min
2.8 Conclusion 1 min
2.9 Appendix: Quantifying Existential Risks 2 min
2.10 Appendix: Forecasting Scenarios 10 min
Chapter 3

Strategies

3.1 Introduction 1 min
3.2 Definitions 4 min
3.3 Misuse Prevention Strategies 8 min
3.4 AGI Safety Strategies 24 min
3.5 ASI Safety Strategies 19 min
3.6 Socio-Technical Strategies 15 min
3.7 Combining Strategies 4 min
3.8 Challenges 7 min
3.9 Conclusion 2 min
3.10 Appendix: Long-term questions 10 min
Chapter 4

Governance

4.1 Introduction 1 min
4.2 Governance Problems 6 min
4.3 Compute Governance 10 min
4.4 Systemic Challenges 13 min
4.5 Governance Architectures 22 min
4.6 Implementation 10 min
4.7 Conclusion 3 min
4.8 Appendix: Data Governance 3 min
4.9 Appendix: National Governance 10 min
Chapter 5

Evaluations

5.1 Introduction 5 min
5.2 Benchmarks 9 min
5.3 Evaluated Properties 11 min
5.4 Evaluation Techniques 16 min
5.5 Evaluation Frameworks 8 min
5.6 Dangerous Capability Evaluations 28 min
5.7 Dangerous Propensity Evaluations 11 min
5.8 Control Evaluations 10 min
5.9 Evaluation Design 8 min
5.10 Limitations 14 min
5.11 Conclusion 1 min
Chapter 6

Specification Gaming

6.1 Introduction 1 min
6.2 Reinforcement Learning 6 min
6.3 Optimization 4 min
6.4 Specification Gaming 9 min
6.5 Learning from imitation 11 min
6.6 Learning from feedback 19 min
Chapter 7

Goal Misgeneralization

7.1 Introduction 3 min
7.2 Multi Objective Generalization 6 min
7.3 Learning Dynamics 9 min
7.4 Goal-Directedness 11 min
7.5 Scheming 24 min
7.6 Detection 11 min
7.7 Mitigations 31 min
Chapter 8

Scalable Oversight

8.1 Introduction 3 min
8.2 Oversight 9 min
8.3 Task Decomposition 5 min
8.4 Process Oversight 10 min
8.5 Iterated Amplification 12 min
8.6 Debate 40 min
8.7 Weak-to-Strong (W2S) 10 min

Ready to start?

Begin with Chapter 1: Capabilities