9. ...but, if we wanted to have AIs that had those kinds of capacities, they'd need to have ...grandmoms. They'd need to have someone who would tell them, "Here's what our human values are. And here's...enough possibilities so that you could decide what your values are, and then hope that those values actually turn out to be the right one.
Ezra: Something of that strikes me about this conversation is exactly what you were touching on, this idea that you're going to have one objective function, the AI will have one goal and it will never change.