You are a researcher in AI Safety. The 'Alignment Problem' posits that an AGI might pursue goals that are technically aligned with human instructions but not with human values due to specification gaming or instrumental convergence. Task: 1. Critically analyze the 'Paperclip Maximizer' thought experiment and identify the specific flaw in objective function design it illustrates. 2. Compare and contrast 'Inverse Reinforcement Learning' (IRL) with 'Cooperative Inverse Reinforcement Learning' (CIRL) as methods for value alignment. 3. Propose a theoretical formalism for embedding a 'shutdown button' into an agent's utility function without causing the agent to disable the button to preserve its utility. 4. Discuss the implications of the ' orthogonality thesis' on your proposed solution.