Reflective Oracles: A Foundation

Benja Fallenstein and Jessica Taylor, Paul F. Christiano

Abstract

Classical game theory treats players as special: A description of a game contains a full, explicit enumeration of all players. This isn’t a realistic assumption for autonomous intelligent agents. Ideally, such agents would be based on a decision-theoretic foundation for game theory in which their coplayers are a non-distinguished part of the agent’s environment, but it is nontrivial to find such a foundation. Attempts to model both players and the environment as Turing machines, for example, fail for standard diagonalization reasons.
经典博弈论特殊对待参与人：一个博弈的描述包含一个完整的、显式的对所有参与人的枚举。这对于自主智能体来说并不是一个真实的假设。理想化地，这样的智能体会基于对博弈论的决策论基础，其中共同参与人是智能体环境的不可区分的一部分，但是找到这样的基础是非平凡的。比如，由于标准对角化原因，尝试定义参与人和环境为图灵机将会失败。