Product1 publisher3 min readPublished
Apple's Audio Intelligence puts the consent switch on one wrist
Apple built the wearer a careful consent flow for the Watch's new listening features and built nothing for the person across the table. That is the part it will have to answer for when it moves to glasses.
The Product Desk · Product desk
What happened
- Apple's Watch Ultra 4 and Series 12 add Live Rewind and Siri Recap, which Fast Company says reposition a health device into an always-listening AI assistant strapped to the wrist.
- Live Rewind keeps a rolling 15-second audio buffer on the watch that is continually overwritten, transcribing the last buffer onto the display when the wearer activates it.
- All four Audio Intelligence features, including Shazam song identification and alerts for doorbells and crying babies, remain completely nonfunctional until the user enables them.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- constraint A flat "it all runs on device" answer to a security review is wrong for half the pair, since Siri Recap's text makes a round trip to Apple's cloud even though its audio never leaves the wrist.
- exposure Anyone in earshot of an enabled watch is inside the buffer with no control of their own, so the job of telling them lands on whoever wrote the meeting policy.
- decision Allowing a wrist that manufactures meeting synopses into rooms where notes are retained is now a call someone has to make without a documented lifetime for the summary text.
- precedent If the wearer-only opt-in is the pattern Apple carries into smart glasses, as Fast Company expects, it will meet the bystander question on a device worn on the face rather than the wrist.
Start with the person across the table. Their voice goes into the same rolling 15-second buffer as the wearer's, and when Live Rewind is activated, their last sentence surfaces as text on a screen they cannot see [4]. The consent Apple built here is genuine and carefully engineered, and it belongs entirely to the one person who enabled the setting [8].
Opt-in settles what happens inside the wearer's own settings screen, nothing more. Per Fast Company's account of the design, the wearer's enablement and the exclave's access restrictions are the whole of the control surface; no notice to the other party appears anywhere in it [14].
The engineering does something specific here rather than something vague. Audio goes into a new Secure Exclave on the S11 chip, gets processed there, and gets deleted, out of reach of watchOS, other apps, Apple, and the wearer [9]. Run an hour-long meeting past Live Rewind and that buffer is written and overwritten roughly 240 times, with no audio left behind at the end [12].
The two novel features part company after that. Live Rewind's transcription runs on the watch and the audio never travels to Apple [5]. Siri Recap deletes the raw audio after on-device speech recognition and then sends the text to Private Cloud Compute to be summarised before the synopsis comes back, which Apple says it cannot read [7]. Counting off-device hops, that is zero for one feature and one for the other [13].
The artifact changes the calculation. A synopsis is not a transcript, by Apple's own description [6], but it is still a durable piece of text with the substance of a conversation in it, and text gets pasted into tickets, forwarded, and retained. Apple's account tracks the audio all the way to deletion and stops following the summary at the moment it lands on the wearer's screen [15].
Each listening feature sorts along two axes. Axis one: what durable artifact does the feature leave, and has that artifact been off the device? Axis two: who other than the wearer can switch the feature off? Live Rewind lands in the easy quadrant, leaving no artifact and never leaving the watch. Siri Recap lands in the quadrant where the artifact is text, the text has made a round trip to Apple's cloud, and the only off switch sits on somebody's wrist.
Fast Company reads all of this as a preview of how Apple will handle public unease about the smart glasses market it plans to enter [11]. The exclave travels well to that device. The wearer-only opt-in does not, because a sealed chip answers where the data went, not whether anyone else in the room agreed to its collection.
How long a Siri Recap summary lives once it is text is checkable and currently unanswered [15], and so is whether any off switch exists that is not on the wearer's wrist [14].
What to watch
- A published retention window for Siri Recap summaries, or an administrator control that lets someone other than the wearer disable Audio Intelligence.
- Whether Apple's smart glasses reuse the Secure Exclave design and add any signal to the people being listened to.
- Whether third-party developers ever get access to the exclave's buffers, which are currently sealed off from other apps on the watch.