Ambient devices; ambient apps

AirPods have become the most successful computing device since the mobile phone. Their revenue now surpasses that of many major enterprise and consumer software companies—a remarkable achievement for what is essentially an accessory for your phone.

There's a competing vision for the next generation of computing devices that is obsessed with replacing your phone: perhaps with a holographic display pin or a new handheld device. 

Both visions are worth exploring, but phone replacement is unnecessarily restrictive and leads to screen-centric designs. AirPods' success points to a different future — one where multiple form factors coexist, and we have more degrees of freedom to build new user experiences.

A compelling new device must accomplish three things:

1. Make it easier to capture and understand your surroundings
2. Connect smoothly with your existing devices
3. Excel at delivering specific types of information

In other words, successful new devices must be ambient devices.

AirPods check all three requirements of our ambient device framework:

1. They make it easier to capture and consume audio
2. They play well with other devices via Bluetooth
3. They are optimized for long-form audio content (especially companionship content)

Now, another technology is following this same playbook: smart glasses. Despite several early flops, they represent the most promising new computing device since AirPods. Let's examine how they align with our framework.

Smart glasses excel at capturing surroundings:

• Built-in cameras let you capture photos and videos from your perspective without reaching for a phone
• This convenience could easily 10x video capture as smart glasses become common
• Natural inputs like eye tracking, head movements, and voice commands capture context that phones miss

Smart glasses work well with other devices:

• Like AirPods, they pair with your phone or laptop as an auxiliary device
• The Meta View app demonstrates this well, making Meta's Ray-Ban glasses extend your phone's capabilities. Users can listen to music, take calls, or share their real-time POV with others.

Smart glasses optimize information delivery:

• They excel at augmented reality overlays and timely notifications.
• Consider using Google Maps: instead of looking down at your phone, navigation appears in your field of view rather than obstructing it. Smart glasses are optimized for this type of low-density but high-value status information.

Ambient Apps

Ambient devices imply a new class of ambient apps — apps that take in your physical and digital context to give you what you need, when you need, where you need. 

New ambient devices alone are not enough to usher in a new wave of ambient apps. You need a computing engine that can understand any data modality, adapt information for different contexts, and send information to the right device. This sounds a lot like transformer-based multimodal language models. They can process any type of input, excel at remixing content, and understand user preferences for consuming information.

Everything is a computer

"We are getting to the point where everything is a computer in a different form factor."
- Steve Jobs at D5 Conference (2007)

This Jobs quote from 2007 is more prescient than ever. As everything becomes a computer in a different form factor. Every form factor works in concert to achieve unique user experiences. This would also mean, optimistically, that we are less reliant on-screen displays and have a more natural interaction with computers.