Know thyself
Looking in the AI-powered mirror
My own mind is often a mystery to me. I have some ability to introspect — and this seems helpful and worth cultivating — but there’s a lot that remains beyond me. I think we usually just take this for granted. But one of the weirder impacts of AI is that it might dramatically improve our individual ability to understand our own minds.
Why should this be? Consider:
Therapists and similar professionals can help people to understand their own thinking, by observing and reflecting back what they hear
This is an information-processing skill! Almost by definition the kind of thing AI can do in principle (setting aside the question of whether it’s any good at it yet)
Compared to most skills, AI systems might have disadvantages compared to humans — people seem to be very good at empathizing with others and “getting inside their head”
But by the time of superintelligence, and perhaps before, AI should overcome this disadvantage
In the longer run, AI systems could have big structural advantages:
Getting a lot more training data than even top human professionals
Potentially access to much richer data streams than just the visual (e.g. we could imagine a superhuman AI therapist that has been trained up while the subjects have scans of their brain activity)
So at some point1, I expect whoever wants it to have access to a kind of magic AI-powered mirror — where looking into it can help the user to see much more of what’s going on with their own thinking. Perhaps it will be represented as a visual map that you can explore and watch in real-time as thoughts or emotions come up for you.
What should we make of this? A miscellany of thoughts:
I find the prospect both quite appealing and at least a bit uncomfortable
I expect both feelings would be common, in varying ratios
It’s unclear how the social equilibrium would change in the presence of this technology
Would everyone be expected to use it? Would it be ostracized?
The strong version of the technology might be quite helpful for people avoiding actions they would ultimately not endorse
I do think the really really good version of this probably comes post-superintelligence (and so is beyond our strategic horizon); but my guess is that impactful precursors are decently likely to come before then
Close relatives of this technology might admit observations of another person’s thoughts and feelings
Potentially a big privacy violation! And generally changes social dynamics a lot if some or many people have functional mind-reading
This could also potentially feed into technology for manipulating people, via drawing a mental map
At first blush, I was inclined to think that the alarming relatives mean that this branch of the technology tree might be better left unexplored as long as possible
On reflection, I’m more unsure — perhaps the mirror might be an important tool in safeguarding against manipulation
Overall, I’m not convinced that this will be one of the earliest types of transformative AI. But it’s not out of the question, and I think is worth being aware of on those grounds. Beyond this, one reason that I’m sharing it is that I think people sometimes have difficulty appreciating how powerful and different AI-powered tools might be, without blurring into agents. By imagining seeing ourselves in the mirror, it may make it easier to consider the mirror itself.
Acknowledgements: Vaughn Tan introduced me to the mirror metaphor as a way of helping people think through things. At some point after I mentioned it to Lizka Vaintrob she drew the picture included here, which spurred the thoughts in the rest of the post.
That is, if humans retain sufficient control, and without themselves having radically upgraded.