Convergent Instrumental Strategies - Arbital - close  ## Metadata - Author: **close** - Full Title: Convergent Instrumental Strategies - Arbital - Category: #articles - URL: https://arbital.com/p/convergent_strategies/ ## Highlights - Suppose an organization is building an AI that this organization believes will accomplish X where X is something plausibly sensible like "Be a Task-based AGI." Actually, however, some mix of insufficient caution and obscure error has led to a situation where, under reflection, the AGI's true utility function has focused on the particular area of RAM that supposedly provides its estimate of task performance. The AI would now like to overwrite as much matter as possible with a state resembling the '1' setting from this area of RAM, a configuration of matter which happens to resemble a tiny molecular paperclip.