Back to notes

Mixed reality · · 16 min read

The headset lost, the glasses are winning, and AI gave them a job

Sales, comfort studies, attention research and what the assistant actually gets right: reading notes on a decade of mixed reality, written while it turned into eyewear.

On this page

Where this started

The question I wrote down in 2024 was whether AI would finally give mixed reality a job. The headset could see the room, the model could reason about it, and it seemed obvious that the two together would produce the use the hardware had been waiting a decade for.

Short answer: yes, but not for the headset. The thing that got a job was a pair of sunglasses with a camera and no screen, and the reason it got the job is that it asks almost nothing of the person wearing it. That took me most of a year to accept, so the notes are in the order I accepted it.

The headset decade

What the numbers did

Apple's Vision Pro is the cleanest test the category has had: the best hardware anyone has shipped, sold by the company best at selling hardware, with no cost ceiling. It shipped about 390,000 units in its 2024 launch year. IDC expected 45,000 in the last quarter of 2025. Sensor Tower counted digital advertising for it down more than 95% over a year in the US and the UK, its assembler stopped production early in 2025, and Apple did little to widen the rollout beyond the countries it launched in.

Morgan Stanley's reading was that cost, form factor and the lack of native apps kept it from selling broadly. I would put those in the other order. The apps did not come because nobody would wear it long enough to need them.

It was not an Apple problem. Counterpoint put VR headset shipments down 14% year on year, and IDC expected the mixed and virtual reality market as a whole to finish 2025 down 42.8%. Microsoft confirmed in October 2024 that HoloLens 2 production had ended, with no successor, and asked the Army for approval to hand its headset programme to Anduril. Meta's Reality Labs has lost $83.6 billion since 2020, $19.2 billion of it in 2025, and now puts 70% of that spending into glasses rather than headsets.

The headset decade in four rows. Company and analyst figures, not mine.
What happenedSource
Vision Pro~390,000 units in 2024; ~45,000 expected in Q4 2025; digital ad spend down more than 95%IDC and Sensor Tower, via MacRumors
CategoryVR headset shipments down 14% year on year; MR and VR market expected down 42.8% in 2025Counterpoint via MacRumors; IDC via The Register
HoloLens 2End of production confirmed October 2024, no successor; support to end of 2027UploadVR
Reality Labs$83.6bn cumulative operating loss to end of 2025; 70% of spend now on glassesOutlook

Why: the body says no

The study I keep coming back to is the one where sixteen people worked in a headset for a full week, eight hours a day, against a week at a normal desk (Biener et al., 2022). Two of the sixteen dropped out on the first day with migraine, nausea and anxiety. The rest rated the headset significantly worse on most measures, with simulator sickness the authors called concerning and usability below average. There was some sign people adapted over the week. Nobody preferred it.

That is the hardware of 2022, and the argument was that better hardware would fix it. Vision Pro is the better hardware. It weighs about 600 grams, or around 725 without the battery pack, and reviewers of the 2025 band describe what it is for: one can now wear it for two hours, another that it no longer sits on his cheeks. Two hours is a film, not a workday.

The glasses turn

While the headsets fell, Meta and EssilorLuxottica sold seven million Ray-Ban and Oakley glasses in 2025, against about two million in the whole of 2023 and 2024 before it, and the maker raised its capacity plan from ten million a year to twenty or thirty. Forrester's consumer survey has 17% of US adults saying they have used smart glasses in 2025, up from 4% in 2024. That is a survey of use, not ownership, and I would treat the level loosely and the direction as real.

What they are is the point. Most of the seven million have no display at all. They are sunglasses with a camera, open-ear speakers, microphones and an assistant. You can take a photo, take a call, play music, and ask what you are looking at. That is the whole product, and my reading of why it sells is that none of it asks the wearer to change anything about how they look, move or see.

The version with a screen tests the edge of that. The Ray-Ban Display puts a small display in one lens and a wristband that reads finger gestures off the muscles of the forearm. A reviewer who wore it daily for ten days found that the band almost never missed a gesture and never fired by mistake, that it made her look at her phone less, and that she felt self-conscious wearing the frames in public. Meta rates the battery at about six hours, and she found that depended on how much the display was used. The display in one eye is the compromise, and it is the thing the review keeps returning to.

Google and Samsung have announced Android XR glasses for the autumn of 2026 through Warby Parker and Gentle Monster, which tells you what they learned. The partners are eyewear brands. The pitch is Gemini in frames people would wear anyway. The word headset does not appear. Google said at I/O that the versions with a display are held for 2027, and a Korean trade report in September 2026 put Samsung's at late 2027. The same report has the first pair launching with no display at all, a camera and audio like the Ray-Bans, and the company still choosing between one eye and two. As I write this, on 19 September 2026, I have not found a price or an on-sale date for the first pair, so this is an announcement and not yet a product.

Three devices, three asks. What each one requires of the wearer, from the reviews and studies above.
Weight on the faceHow long people wear itWhat it asks
Vision ProAbout 600 g, 725 without the batteryAbout two hours with the 2025 bandA room, a seat, and your whole field of view
Ray-Ban DisplayHeavier frames, one-eye display, a wristbandA day, on a battery rated at six hoursThat you accept how it looks
Ray-Ban MetaOrdinary sunglasses plus a cameraAll dayNothing you were not already doing
When you have your cool, new wearable device idea, really fully bake it before you have a cool stunt involving skydiving and airships.
Sergey Brin, on Google Glass, which stopped selling in January 2015

What AI actually adds

The job, finally

The reason the glasses have a job and the headset did not is the assistant. A camera on your face was a party trick in 2013; a camera on your face that can answer what is this, read this to me, translate that, remind me where I parked, is a use. The model did not make mixed reality work. It made a much smaller thing worth wearing, and that smaller thing is what the market bought.

I should be plain about how far that goes. The sales figures say people bought the glasses. They do not say why. The nearest thing to a usage figure is Mark Zuckerberg on Meta's first-quarter 2026 earnings call: daily users of the glasses tripled year over year. That says people keep wearing them. It gives no absolute number, and it does not separate the assistant from the camera or the music; on the same call he described the glasses as able to answer questions now and a personal agent later, which is a plan and not a measurement.

I have looked twice for a published figure on how many owners use the assistant, and how often, and have not found one; a pair of these is also a camera, a pair of headphones and a pair of Ray-Bans. So the claim I can defend is the narrow one: the assistant is what turns a camera on a face from a trick into a use. Whether that is what sells them is a bet, and a usage figure from Meta would settle it either way.

The research is early and mostly demos

The CHI 2025 review of language models in extended reality (Tang et al.) is a map of prototypes. The most complete one I found is Guided Reality at UIST 2025: a headset that takes a question, generates steps, works out which kind of visual cue each step needs, finds the object in the room and draws the cue on it. Sixteen participants completed real tasks with it, rated it usable, and were let loose with it in the building afterwards. A second line of work (Teaching LLMs to See and Guide) feeds hand actions, task step and dialogue history into the model together and shows the answers get more accurate when it knows what your hands are doing.

Both are the right idea. One is sixteen people in a lab on a headset; the other is an offline run against a recorded dataset, with no participants at all. Neither has been run on the device seven million people bought, and the device seven million people bought has no display to draw a cue on. The best research and the best-selling product are not the same product yet.

The people who need it most, and what they found

The most honest evaluation of the assistant in the wild is not from a lab. It is the American Foundation for the Blind's review of the Meta glasses for low vision, by people for whom describe what you see is not a novelty. They found it useful and they found it unreliable in the same paragraph: it insisted on summarising when asked to read in full, it made things up, and asked whether there was a lamp in the room it said yes, there is one on the table by the bed, then put it several feet away, then slightly to the left, when it was to the right of centre. The review does not use the word, but that is hallucination, and it is the thing I would put on every product page.

The fix they use is the one the category should learn from. Say “Hey Meta, Be My Eyes” and the camera goes to a sighted volunteer instead of the model. The assistant for the easy questions, a person for the ones that matter. That is a design pattern, not a workaround, and it is the same shape as the finding in the note on AI chat: conversation for intent, something more reliable for commitment.

What the eyes cost

This is the part the demos never show. Anything drawn over the world takes attention from the world, and the effect has been measured for a decade in the places where missing something is expensive.

Dixon and colleagues gave surgeons a navigation task with the guidance either on a head-up display over the field or on a side monitor, and planted an unexpected finding in the field (2014). On the side monitor, 15 of 25 noticed it. On the head-up display, 8 of 25. Same task, same accuracy, same time; the only thing that changed was where the picture was, and it halved the chance of seeing the thing nobody had told them to look for.

An on-road study in 2025 found the same shape in cars: drivers' ability to detect things in the environment fell as the visual demand of the head-up display task rose, and the blindness was worst in the centre of the field of view, which is exactly where a display glasses maker wants to put the display.

  • For the headset, this is the second reason the workday never came. The room is the thing you are meant to be attending to, and the interface sits in front of it.
  • For the display glasses, it is the argument for one small display, off-centre, that shows one line at a time. The reviews read that as a compromise. The attention research reads it as the only safe design.
  • For the glasses with no display, it is why they work. Audio takes a different channel. The eyes stay on the world.

The bystander

Google Glass did not fail on the wearer. It failed on everyone else in the room. The camera that could not be seen recording is what produced the word Glasshole, the bans in bars, and the end of the product, which stopped being made in January 2015. Ten years on the camera is back, on seven million faces, with a small light to say it is on.

In October 2024 two Harvard students pointed the Meta glasses at strangers, ran the faces through a face search engine, and walked up to them knowing their names, addresses and relatives. They built nothing new. They chained things that already existed, and the glasses were the cheapest part. Meta's own assistant will not identify anyone; the point of the demonstration is that the refusal lives in Meta's software, and the camera does not.

I do not have a design answer to this and I do not think one exists at the level of the product. A recording light is a courtesy, not a control. The thing that will decide whether a camera on a face is acceptable is what the people not wearing it decide, venue by venue, and the model makes that decision more urgent, because it turns a photo of you into a query about you.

Where this leaves me

The question was whether AI gives mixed reality something to do. The answer I have is that it gave eyewear something to do, and that mixed reality, meaning a display over the world, is still waiting, and now has a measured reason to wait.

  1. Audio first. The assistant that answers in your ear, from a camera you forget is there, is the product that shipped. Everything I would design for glasses starts there.
  2. A display earns its place one line at a time. Off-centre, glanceable, dismissable. The moment it competes with the world it costs what the surgeons lost.
  3. A person behind the model. Be My Eyes is the pattern: the model for what is this, a human for is it safe.
  4. The headset is for sitting down. Two hours, a seat, a film, a design review of a room that does not exist yet. That is a real use and it is not a platform.
  5. Design for the person not wearing it. The wearer's experience is close to solved, at least for glasses with no display. The bystander's is the open problem, and no amount of model quality closes it.

What I have not settled is whether the display glasses are the next step or the next Glass. The reviews say the wristband works and the frames embarrass. The attention research says the small display is right. The sales say nobody has needed one yet, and Google and Samsung holding their display models back to 2027 is at least consistent with that. I would want a year of the Android XR frames in the wild before I bet either way, and I have written that down so I cannot pretend later that I knew.

What I would expect to outlast the scoreboard is the reason underneath it. A device gets worn by how little it asks of the person wearing it, and the attention research puts a ceiling on how much a display can ever ask. If a headset gets light enough, or a display learns to stay out of the way, either could clear that bar, and the winners in this note would change. The bar would not.

Sources

Sales and the market

Comfort, attention and the body

The assistant

The bystander

Last updated

The other questions
Back to the notesDiscuss this note