Layers of Natural UI
The Natural User Interface is not about voice conversations taking over. It is about every modality natural for us, humans, adopted for human-computer interactions.
In the process of improving user experience, we will adapt more natural ways of interacting with our computers layer by layer. Our nature is fundamental to the way we improve the interface.
Each human sense and way of interfacing will be engaged. Spatial computing now unlocking look and pinch. In a conversation where someone is looking and how, what gestures accompany the speech may change the meaning dramatically. Each modality is low bandwidth on its own, but together, look, touch, voice, gesture, and fascial expression are very rich.
Context matters a lot. With many early voice enabled conversational devices, we had to start from the top every time: Alexa and Siri did not remember what we said previously. They did not sense a situation. They did not follow our context.
Now this is changing, and this is why voice interactions are becoming dramatically more viable. It is compounded effect of multi-modality.
When I’m arguing for natural agents, I’m not suggesting a world where every interaction is a voice one. Voice will play a very important role in two ways: 1. as a quick, convenient way to express a command (kind of like a keyboard shortcut) 2. as the most immersive way to engage in a deep conversation with an agent.
We will be speaking in short bursts with commands like reply, dismiss, open, etc. On the other hand, when the environment and context are appropriate, we will be having prolonged, in-depth conversations with the AI.
Both of these will be accompanied by dynamic visual information, the option to touch or pinch, or use a pointer, switch to a keyboard for input when silent interaction is preferable. We will see layers of natural interactions all coming together and reinforcing each other.
Everything we learned about UX building websites and apps is still applicable. It is just another layer and glazes on top.
Want to get my next post in your inbox? Signup.
