Other Action Button = "Super Siri" / Magic Button in future iOS

[zakarhino]

I think we can all see where this is going. A ton of people are not just assigning their action button to a static shortcut, rather they are programming their shortcut to include various conditional statements so that the action button can do multiple things depending on the context -- all without manually reconfiguring the action button.

I believe this kind of function is what Apple will introduce as a first class automatic "Magic Button" of sorts in the next version of iOS, with one of the headline demos being contextual "one click" control of Home scenes and accessories (why do you think the iPhone has native Thread support now). This may be part of a Siri overhaul wherein Siri bifurcates into one mode dedicated to traditional conversational Siri (triggered by holding the lock button) and another mode dedicated to contextual intelligence based on the following: what focus mode you're in, where you are (GPS, yes, but especially Ultrawide band. Again, new ultrawide band chip this year), and what's on screen. This kind of Super Siri / Magic Button mode will be the next version of the Shortcuts app which hasn't been given much major attention since its introduction. Carl Pei mentioned this in his reaction to the iPhone 15, that the Action Button may be used in the future for triggering a "Super Siri" to rival ChatGPT's intelligence.

Today's rumor about the iPhone 16 introducing a dedicated camera button in addition to the action button helps bolster this idea. I'm not sure how conflicts will be managed in terms of whether or not the "Magic Button" behavior conflicts with the "Super Siri" on screen intelligence mode. Hell I'm not even sure about any of the above, it's just loose theory, but what I'm fairly confident about is this: I don't think Apple introduced the Action Button as a mere programmable button, that's not Apple. I think their long term goal is for that button to trigger a new behavior that's so essential to the future of their platform they're willing to give it a dedicated hardware trigger, much like they did with Siri. Of course this will also come to Apple Watch. If you're at the gym you may want your Action Button to start a treadmill workout, if you're at the trail maybe you want it to start an outdoor run, and if you've just arrived home maybe you want it to open your garage.

Another point to consider: "Magic Button" behavior is likely something coming to visionOS down the line (especially once we reach the always on AR glasses paradigm). You look at a lightbulb, tap your fingers, and it turns on. You look at your stove, tap your fingers, and your cooking workflow opens. You look at your desk, tap your fingers, and the visionOS virtual Mac displays pop up in front of you. This is the logical endpoint for visionOS + AI, using all sensor data to predict behavior so everything becomes 'one tap.' Why not bring this to iPhone too? If anything it makes sense to introduce this on iPhone first before visionOS.

 

[VertPin]

A ton of people? I think the majority of people aren’t using Siri Shortcuts to begin with.

 

[JPack]

The ultimate Siri button would be "no button." Actions are triggered by voice and Siri fully understands the context.

Having a dedicated Capture button on a future iPhone makes sense because Apple is positioning the device more and more as a camera. You want a single button to start recording immediately.

 

[fatTribble]

I read your post multiple times, but I’m not sure I follow. My plan is to write a shortcut that performs different actions based on time of day. It would be very well defined what it will do. When you talk about a magic Siri button that does things automatically I get scared. The problem with computers doing anything automatically is handling the exceptions. Apologies if I’m misread your intent.

 

[rKunda]

This is a radical case of projection. You have to recognize that those of us trolling the MacRumors forums are a special kind of enthusiast.

Shortcuts are a niche nicety. Honestly, I’d be surprised if more than half of users actually use Siri for anything. For shortcuts, that number is abysmal.

 

[zakarhino]

I read your post multiple times, but I’m not sure I follow. My plan is to write a shortcut that performs different actions based on time of day. It would be very well defined what it will do. When you talk about a magic Siri button that does things automatically I get scared. The problem with computers doing anything automatically is handling the exceptions. Apologies if I’m misread your intent.

No apologies necessary. I'm not really describing a fully thought out feature which is why my post is not very coherent. Let me be a bit more specific.

There are essentially two ideas being described at once:

1. Magic Button: The first idea, if we're to initially think of it in very simplistic terms, is a kind of Action Button integration with Focus modes. Where the function of the Action Button changes depending on the Focus mode you're in. At the moment you'd need to do this by dragging and dropping different conditional blocks together in Siri Shortcuts. I think Apple will make this easier for people to do by integrating an Action Button assignment right into Focus modes, similar to how users can choose which home screens are active depending on the active Focus mode. This is low hanging fruit.
   
   Now if we take this a step further and think about Apple's trajectory when it comes to on device "Siri Suggestions", I think Apple will eventually add a new level of contextual detail into Focus modes. At the moment we have high level modes like:
   
   "Work"
   "Home"
   
   but I think intelligent submodes can be added into that:
   
   "Work, before your 1pm meeting"
   "Work, heading home"
   "Home, in the Kitchen"
   "Home, in the Garden"
   "Home, watching a movie"
   
   All (or most) of which are derived from a similar kind of device intelligence already being refined via Siri Suggestions. I know I always get a Siri Suggestion for directions to a specific place as soon as I get into my car because Siri has learned by driving behavior and knows I'm in the car when I connect to bluetooth. In this paradigm the Action Button changes depending on those intelligent submodes.
   
   "Work, heading home" = Action Button triggers directions to home or brings up your preferred transit app.
   "Home, in the Kitchen" = Action Button opens the timer app or recipes app
   "Home, in the Garden" = Action Button resumes your last podcast or whatever
   "Home, watching a movie" (it knows this via Apple TV) = Action Button triggers your dim lighting scene.
   
   As for handling the implementation issue of unpredictable automatic behaviors (as per your concern) I think when these Magic Button behaviors first get suggested they'll be visible in the dynamic island and you'll also be able to customize these behaviors (and see more suggestions) in the Shortcuts app.
   
   Again I don't know how all of this will be implemented and maybe Action Button will remain as is but I'm certain the enhancements to Siri Suggestions with some sort of relationship to Focus modes are coming. The future of these platforms is in AI's ability to help do more for you depending on the context: both physical context as well as digital context. Which brings me to no 2:
2. Super Siri: I'm less sure about this one but I think it just plays on what I described above. Siri will have more of an obvious presence in Apple platforms going forward outside of the voice assistant you sometimes call on to start a timer or ask a general knowledge question. Experiments in LLMs + API integration have already shown us that these LLMs are capable of compound, original task chaining. There was a demo from a year ago from someone who hooked up HomeKit to ChatGPT and could command multiple things in their house at once all in one prompt, including tasks that weren't necessarily pre-mapped to a specific HomeKit feature so the LLM had to infer how to translate "Give me some romantic lighting" into an actual task (dimming the lights and setting their temperature to warm).
   
   As per @JPack this could be completely unrelated to the Action Button and may end up being simply a next gen version of Siri. That's the likely option.

A ton of people? I think the majority of people aren’t using Siri Shortcuts to begin with.

It's an exaggeration because yeah most people don't even use them very often. I certainly don't, they're too much of a hassle to program and don't have all the actions I want anyway.

BUT there's no doubt I've seen plenty of talk online about how Action Button + Shortcuts is a cheat code and a major underrated feature of the 15 Pro. I've seen multiple videos and journalists make content dedicated to praising the Action Button + Shortcut trick. Look here's one from today:

The Action Button is the most significant new iPhone feature in years

Programmable hardware buttons rule.

www.theverge.com

And I wonder if the number of people using Shortcuts will increase now that it's the only way to launch a third party app from the Action Button. Maybe people with no prior Shortcuts experience will get exposed to some of the more basic workflows possible in Shortcuts (all assignable to the Action Button)

This is a radical case of projection. You have to recognize that those of us trolling the MacRumors forums are a special kind of enthusiast.

Shortcuts are a niche nicety. Honestly, I’d be surprised if more than half of users actually use Siri for anything. For shortcuts, that number is abysmal.

Niche use cases of today become commonplace staples of tomorrow, especially if the engineering work is put in to simplify the process of engaging with that particular feature. Siri Shortcuts today are tucked away in an app and require a lot of setup. If Apple significantly boost the intelligence of Siri Suggestions, combine that with Shortcuts to automatically suggest new workflows, and integrate that behavior into the Action Button, then I think a lot more people will use them.

Doesn't have to be Shortcuts either, but the point is more advanced forms of automation are around the corner and Apple certainly won't push those automations at niche. Those automations, irrespective of form, will be as important to the future of Apple's world as apps are, they simply have to be because the advancements in AI dictate it.

 

[zakarhino]

I wonder if next years iPhones getting taller has anything to do with Apple wanting more space in the status bar to surface contextual Siri information or inform the user as to what the "Magic Button" is primed to do at any moment 🤔 The latter is less likely than the former, I can concede that, but it's interesting that the aspect ratio will shift significantly. I can't help but feel there's a software related reason for the screen size increase rather than simply giving people more lines of text in Safari.

 

[fatTribble]

Thanks! That clarifies a lot. I think the action button changing according to Focus modes would be great. If it changes by Siri maybe it could be a single press where you’d see the Siri suggested action and one more to confirm. UI was never my strong point so there are likely better ways to do that.

 

[fatTribble]

This is a radical case of projection. You have to recognize that those of us trolling the MacRumors forums are a special kind of enthusiast.

Shortcuts are a niche nicety. Honestly, I’d be surprised if more than half of users actually use Siri for anything. For shortcuts, that number is abysmal.

My best friend is a DevOps Engineer and I still haven’t been able to convince him to try Shortcuts

 

[zakarhino]

Thanks! That clarifies a lot. I think the action button changing according to Focus modes would be great. If it changes by Siri maybe it could be a single press where you’d see the Siri suggested action and one more to confirm. UI was never my strong point so there are likely better ways to do that.

Funnily enough this is similar to how the Action Button already works. Long press activates the button but a short press shows you what the button is assigned to via a Dynamic Island animation.