Apple Releases Open Source AI Models That Run On-Device

[NT1440]

Food for thought....what makes you any different than these so called "experts" you speak of? If Apple can't even get siri to do miniscule type functions without screwing that up, that what makes you think they'll get AI right?

I’m not sure you got the point of my post. I’m not posting around proclaiming LLM X is “better” than LLM Y based off single metrics like size or number of tokens, or whether it runs on NVIDIA hardware, etc.

I’m not an armchair AI expert, nor do I claim to be.

 

[randolorian]

[Hey ChatGPT, please generate a comment in the style of a typical MacRumors average user, with a touch of acid humor, regarding this piece of news.]

[Hey Claude, please generate a comment in the style of a typical MacRumors average user, with a touch of acid humor, regarding this piece of news.]

"Well, well, well, look who's finally decided to join the open-source party! It only took Apple a few decades to realize that keeping everything locked up tighter than Fort Knox wasn't doing them any favors in the AI game. I guess even the geniuses in Cupertino realized that having a bunch of nerds poking around their precious models might actually be a good thing.

But let's be real, this is just Apple's way of trying to stay relevant in the AI race. They've been lagging behind the likes of Google and OpenAI for years, and now they're desperate to catch up. What's next, are they going to start crowd-sourcing ideas for the next iPhone design? (Actually, that might not be a bad idea, considering the embarrassment that was the iPhone 14.)

Still, I'll give them credit for finally embracing the open-source movement. Maybe they'll even let us peasants contribute to their models, you know, as long as we sign away our firstborn and agree to never speak ill of the almighty Apple again.

But hey, who am I kidding? We all know that even if Apple's AI models become open-source, they'll still find a way to make them proprietary and overpriced. Gotta love that Apple tax, am I right?"

 

[vantelimus]

[Hey ChatGPT, please generate a comment in the style of a typical MacRumors average user, with a touch of acid humor, regarding this piece of news.]

"Oh great, Apple's finally joining the open-source party”

Finally, because Darwin and WebKit (both early 2000s releases), and Swift (2015) never happened?

You might want to check https://opensource.apple.com/ for a full list of current open source projects Apple has created or to which Apple actively contributes.

 

[Populus]

Finally, because Darwin and WebKit (both early 2000s releases), and Swift (2015) never happened?

You might want to check https://opensource.apple.com/ for a full list of current open source projects Apple has created or to which Apple actively contributes.

Thank you, I’ll communicate that to Chat GPT on your behalf!

 

[tkaravou]

This is fantastic, pretty much confirms that they’ll launch iOS18 with it

 

[heretiq]

Food for thought....what makes you any different than these so called "experts" you speak of? If Apple can't even get siri to do miniscule type functions without screwing that up, that what makes you think they'll get AI right?

Maybe because Apple has a history of getting lots right: leveraging prior art to set the standard for the modern graphical UI, remaking music distribution and consumption, reinventing the phone and becoming the standard for imitators and aspiring competitors, creating the standard for best-in-class tablet computing, ushering in the era of truly personal computing for the masses by creating the devices, infrastructures and ecosystems required to enable it (possibly including the very device you’re using to express that very cynical perspective on an obviously innovative and capable company).

 

[Noblesongster]

Glad to see that Apple is finally getting on the bandwagon when it comes to AI. Siri has been a colossal disappointment for a long time and still can't do simple things like add a contact.

BTW, it's "state-of-the-art" not "stage-of-the-art"...whatever the heck THAT is.

 

[VulchR]

Yeah, I actually laughed when I read the outcome.
Chat GPT could very well be one of us.

***** knew it.
The comment section is dead. Long live the AI comment section. lol

Most of us? 🤔

No doubt ChatGPT vacuums up online posts like those here in MR as training material. Scary thought, isn't it? 😳

Of course in the long run AI bots will be posting so much on social media that AI's will be effectively training other AI's. And then, AI companies will subtly poison the output of their AI bots to cripple the output of other AI's. Yessiree bob - the future is bright indeed. [/sarcasm]

 

[hans1972]

Only reason they release this is they need help from the community.

They also need to stop using words like empower and enrich peoples lives.


1. Selling Macbook Pros with scratches in the screen is not empowering nor enriching.

2. Selling Homepods then kill it slowly, only to resurrect it with even lower downgrades is not empowering nor enriching.

3. Still selling accessories with lightning is not empowering nor enriching.

4. Selling Goggles that pose a health risk causing blindness is not empowering nor enriching.

5. Releasing upgrades that cripple performance or introducing hardware glitches to force upgrades is not empowering nor enriching.

6. Need to sum up this and more is not empowering nor enriching.

Make a reply to the above from a Macrumors.com commenter.

"Couldn't agree more.

It's frustrating to see Apple continuously tout their products as 'empowering' and 'life-enriching' while ignoring glaring issues and prioritizing profit over customer satisfaction. The lack of quality control, the deceptive marketing tactics, and the disregard for user experience are all signs that Apple needs to reevaluate their priorities.

It's time for them to listen to their community and make real changes that actually benefit their customers, rather than just their bottom line." -ChatGPT

 

[clg82]

Maybe because Apple has a history of getting lots right: leveraging prior art to set the standard for the modern graphical UI, remaking music distribution and consumption, reinventing the phone and becoming the standard for imitators and aspiring competitors, creating the standard for best-in-class tablet computing, ushering in the era of truly personal computing for the masses by creating the devices, infrastructures and ecosystems required to enable it (possibly including the very device you’re using to express that very cynical perspective on an obviously innovative and capable company).

Ummmmm they haven't debuted anything "innovative" in a very very long time.

 

[heretiq]

Ummmmm they haven't debuted anything "innovative" in a very very long time.

You don’t think Apple watch ultra, Apple Silicon MX processors, MX Series MacBook Pros/Mac mini/MacStudio/15“ MacBook Air, Xcode playgrounds, Freeform, Spatial Computing/VisionPro, and MLX/Ferret are innovative?

 

[NT1440]

You don’t think Apple watch ultra, Apple Silicon MX processors, MX Series MacBook Pros/Mac mini/MacStudio/15“ MacBook Air, Xcode playgrounds, Freeform, Spatial Computing/VisionPro, and MLX/Ferret — all are very recent introductions — are innovative?

Honestly, I think the entire world just spits out the term “innovate” that is meaningless at this point.

Nobody can define it, but managerial types sure like spitting it out as part of justification for just about anything, good or bad (“these layoffs will allow us to focus on innovation!”).

I would say advancements instead, I always thought innovation was doing something in a way not done before, but apparently not 🤷‍♂️

 

[heretiq]

Honestly, I think the entire world just spits out the term “innovate” that is meaningless at this point.

Nobody can define it, but managerial types sure like spitting it out as part of justification for just about anything, good or bad (“these layoffs will allow us to focus on innovation!”).

I would say advancements instead, I always thought innovation was doing something in a way not done before, but apparently not 🤷‍♂️

I agree, but would not rush to change your definition of innovation simply because the term is used by some in muddled ways. Doing new things or doing things differently, better, more efficiently, more effectively are all examples of being innovative. If you look at Apple through that lens there are countless examples of innovation ranging from invisible innovations in materials, operations and logistics to more visible examples ranging from improved displays to SOCS to new and improved products. I think the problem is that many of us discount all the small innovations that provide the foundation for breakthrough advancements and only count the breakthrough advancements as “innovation.” That is a naive perspective. I think this McKinsey article does a good job of exploring the topic and how truly innovative companies achieve it. There is a lot more involved than meets the eye and companies and leaders that can pull it off should be celebrated.

 

[TheOldChevy]

My experience with my basic M1pro macbook is quite good using llama3 or mistral, so I would expect that an 8 GB iPhone with an optimised model could be quite ok.

You mean it's some kind of RNN ? It would make sense for audio.

 

[NT1440]

I agree, but would not rush to change your definition of innovation simply because the term is used by some in muddled ways. Doing new things or doing things differently, better, more efficiently, more effectively are all examples of being innovative. If you look at Apple through that lens there are countless examples of innovation ranging from invisible innovations in materials, operations and logistics to more visible examples ranging from improved displays to SOCS to new and improved products. I think the problem is that many of us discount all the small innovations that provide the foundation for breakthrough advancements and only count the breakthrough advancements as “innovation.” That is a naive perspective. I think this McKinsey article does a good job of exploring the topic and how truly innovative companies achieve it. There is a lot more involved than meets the eye and companies and leaders that can pull it off should be celebrated.

Oh I'm not throwing out the term, I just discount it when posters claim Apple is not innovating anymore. Every year at WWDC and it's sessions we see the actual innovation Apple continually implements. If it's not slapping you in the face via news headlines or a UI in your face people are blind to it apparently.

 

[rafark]

The thing I’ve noticed about all these AI hype-people is they certainly know what the “leader” of the pack day to day is, but somehow can’t imagine smaller models being a better solution for a given task.

Literally people who own apple devices (me too). Replace “AI hype-people” with “people with an apple device”

 

[CosmicTris392]

Apple finally decides to join the open-source party, huh? Only a decade late and I bet there are still some hidden strings attached to their so-called 'open' terms. They're throwing us a bone with OpenELM, but let's be real, they're probably just trying to lure in some AI hotshots tired of their corporate overlords. And now we have to wait for iOS 18, where they'll undoubtedly limit these models to the latest hardware, forcing us all to upgrade. Because, you know, my current iPhone surely can't handle a few more AI tricks without spontaneously combusting. Classic Apple move.

It seems that some people in your reactions have missed the joke

 

[d-klumpp]

As an Apple enthusiast since the 70s but nonetheless not a coder or electrical engineer, I come here to hear the latest rumors on tech to which I might look forward to and to get informative reactions, maybe even from professionals within the fields of computing or consumer electronics. Unfortunately, too often most commenters offer only offer rants with little merit and no insight. Why are they here? Why do I waste my own time even scanning these comments?

 

[name99]

Only reason they release this is they need help from the community.

They also need to stop using words like empower and enrich peoples lives.


1. Selling Macbook Pros with scratches in the screen is not empowering nor enriching.

2. Selling Homepods then kill it slowly, only to resurrect it with even lower downgrades is not empowering nor enriching.

3. Still selling accessories with lightning is not empowering nor enriching.

4. Selling Goggles that pose a health risk causing blindness is not empowering nor enriching.

5. Releasing upgrades that cripple performance or introducing hardware glitches to force upgrades is not empowering nor enriching.

6. Need to sum up this and more is not empowering nor enriching.

You can give some people a backpack full of gold bricks, and rather than thank you they'll complain it's too heavy...

 

[name99]

Apple finally decides to join the open-source party, huh? Only a decade late and I bet there are still some hidden strings attached to their so-called 'open' terms. They're throwing us a bone with OpenELM, but let's be real, they're probably just trying to lure in some AI hotshots tired of their corporate overlords. And now we have to wait for iOS 18, where they'll undoubtedly limit these models to the latest hardware, forcing us all to upgrade. Because, you know, my current iPhone surely can't handle a few more AI tricks without spontaneously combusting. Classic Apple move.

Well there is: https://opensource.apple.com/projects/

But sure, feel free to spray your hate-flecked ignorance-drenched spittle all over the internet...

 

[teknikal90]

Nvidia is dominating AI just because they are using Nvidia based ecosystem, software, technology and more. CUDA is a great example. And Nvidia invested AI for a decade. Yes, AI chip or NPU is much more efficient but not powerful enough to beat Nvidia GPU for now and even if they are powerful, they are only limited by their own AI models.

It's better than nothing but it's a long way to go especially since Mac is only limited for 2D based software like video, music, photo, and more.

there is a lack of performing alternatives. so if you were building a model, why would you build for something else?

 

[heretiq]

I've worked on iOS AI Apps for several years now and I've looked long and hard at "on device".
What stops anyone with a decent, unique, intelligence model from putting it on device is having it stolen, re-packaged and resold.
iOS is pretty hardened but not unbreakable.
I asked Apple peeps, on the dev forums, and the answer I got was "yeah, that's DRM, it's not perfect"
Apple's own ML API got an encryption feature to protect proprietary models a couple of years ago but it was limited to only what ML supported.
So I don't expect a mad rush to use any on device LLM's unless there's an impressively secure way of locking in and protecting what they contain.
If the new Apple silicon (M4) has an AI focus then it'll no doubt be encased within a pretty strong silicon encryption. Something that Apple has been doing for things like biometrics, banking and ML.
It's not just about how good is your AI silicon is, it's as much about how well secured your AI model is on silicon for on device to succeed.

Thanks for this excellent post. I agree completely that model security is an important factor and will impact the pace of adoption of AI plumbing that Apple builds into its next OS releases. For some developers/apps/use cases, security will be a make or break factor, and for others it may not. I have no idea what that distribution will look like, but time will tell. For our company’s app user data security is critical but our envisioned custom LLM model security is not — as it will largely be trained on information that we openly publish. We have explored using Swift Transformers with CoreML converted Llama2 and Falcon open source models, but decided to wait on Apple ML framework and model extensions that would allow us to keep the app using 100% native Apple frameworks — which is comforting to our privacy-oriented app users. WWDC 24 can’t come fast enough. “Keep hope alive!” 🙏🏽

 

[name99]

I've worked on iOS AI Apps for several years now and I've looked long and hard at "on device".
What stops anyone with a decent, unique, intelligence model from putting it on device is having it stolen, re-packaged and resold.
iOS is pretty hardened but not unbreakable.
I asked Apple peeps, on the dev forums, and the answer I got was "yeah, that's DRM, it's not perfect"
Apple's own ML API got an encryption feature to protect proprietary models a couple of years ago but it was limited to only what ML supported.
So I don't expect a mad rush to use any on device LLM's unless there's an impressively secure way of locking in and protecting what they contain.
If the new Apple silicon (M4) has an AI focus then it'll no doubt be encased within a pretty strong silicon encryption. Something that Apple has been doing for things like biometrics, banking and ML.
It's not just about how good is your AI silicon is, it's as much about how well secured your AI model is on silicon for on device to succeed.

Apple has a specific patent for DRM associated with neural nets.
They also have a "secure mode" associated with the ANE which limits the extent to which ANE code can spy on anything else (code or data) associated with the ANE. Clearly used by FaceID, but presumably available to other software.
Yes, sure, no-one is claiming perfection on the part of Apple hardware/software. But this is essentially the same tech that protects iPhone video content and FaceID, and no-one's cracked either of those.

My guess is the way this plays out is that many companies, out of paranoia, are going to think just like Seoras, and deliver a lousier experience (slower, less personalized, doesn't work when connectivity is not available) in a way that gives Apple an entry wedge.
Apple Maps was not the best when it shipped, nor was Apple Translate. But by being on every device, and by avoiding the [self-imposed!] limitations of competitors, they were good enough for most people to try and experiment with.
And by the time competitors realize what is happening, that their paranoia is not justified, and that they are paying a dollar in inference compute for every week of customer queries, two years will have passed, and most of the user base will be comfortable with Apple (which will be laughing at all that money saved from on-device rather than on-server inference).

Certainly right now I can put up with the hassle of an existing chatbot (figure out which one, pay a monthly fee, have to out up with constant login prompts); or I can wait for the Apple chatbot.
Frankly I just don't care enough to go beyond waiting for what Apple ships in AppleOS 2024.

 

[dhess34]

Using the term "AI" in the title for clickbait when you use the correct term (LLM) in the post body 🙄

One of the worst misses I've seen on these forums, and that's saying something: What Apple published isn't an LLM. Pro Tip: read the associated white paper before making comments that make you look like a fool.

 

[Tagbert]

Ummmmm they haven't debuted anything "innovative" in a very very long time.

Are we still doing the ChatGPT comments?