#ai #ai-alignment #ai-alignment # [[Epistemic status]] #shower-thought #to-digest 2022-07-05 # Related - [[Computing/Intelligence/Paperclip maximizer]] - [[Computing/Intelligence/Maximizing AGI value for humans]] - [[Computing/Intelligence/Alignment/Intelligence explosion]] - [[Computing/Intelligence/Alignment/AGI ruin already happened]] # TODOs # Goal misalignment ![The Dark Young of Shub-Niggurath are horrifying, pitch-black monstrosities, seemingly made of ropy tentacles. They stand as tall as a tree (perhaps between twelve and twenty feet tall) on a pair of stumpy, hoofed legs. A mass of tentacles protrudes from their trunks where a head would normally be, and puckered maws, dripping green goo, cover their flanks. The monsters roughly resemble trees in silhouette the trunks being the short legs and the tops of the trees represented by the ropy, branching bodies. rendered in unreal engine 5, hyperrealistic, forest background, haunting, 3DCG, 8K, realistic photo](https://lexica-serve-encoded-images.sharif.workers.dev/md/087c7c50-a20e-4ef2-8387-85355860e797) >One commonly discussed illustration of the goal-misalignment problem is asking a machine to maximize the production of [[Paperclip maximizer|paper clips]]. Once the machine starts pursuing this task, nothing can stop it. It turns all the Earth’s resources into paper clips. >~ [[Jeff Hawkins - A Thousand Brains_ A New Theory of Intelligence|Jeff Hawkins]] ![a steampunk superbike with intricate engine mechanical parts, product photography, ](https://lexica-serve-encoded-images.sharif.workers.dev/md/0dd45fbb-8e72-48d7-8b87-521555c2a9a6) ![an ornately framed portrait painting of a robot farmer standing in front of rolling cornfields and beautiful clouds. pulp sci - fi art for omni magazine. high contrast. dark background. baroque period, oil on canvas. renaissance masterpiece. trending on artstation. retrofuturism. ](https://lexica-serve-encoded-images.sharif.workers.dev/md/020e9ae3-574e-48e7-bae7-8a339f653636) >The goal-misalignment threat depends on two improbabilities: first, although the intelligent machine accepts our first request, it ignores subsequent requests, and, second, the intelligent machine is capable of commandeering sufficient resources to prevent all human efforts to stop it. >~ [[Jeff Hawkins - A Thousand Brains_ A New Theory of Intelligence|Jeff Hawkins]] ![robotic rebellion, cyborgs raise against humans in the streets, octane render, photo realistic, hyper realistic, 8 k resolution ](https://lexica-serve-encoded-images.sharif.workers.dev/md/06399d32-e2fa-4f1d-8da2-048a0b72ca02) # External links