And you don't need that for everything. LLM's with 7 billion parameters are about 4GB and perform fine when tuned.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Apple Develops Breakthrough Method for Running LLMs on iPhones
- Thread starter MacRumors
- Start date
- Sort by reaction score
From my short experience AI is now mainly trained by curators to give "curated" answers and keep us in mental prison we are in now. Twice I managed after long fiddling to get unbiased answer but then was logged of immediately. It has never happened in other any occasion.
There’s no way Apple is giving this “Apple GPT” to devices older than the latest and greatest (whenever AI gets implemented in iOS).
And at that time, I don’t think any but the lowest end Apple devices will have less than 6GB RAM.
Holding out on RAM is one of the best ways to ensure we’ll keep upgrading even though most other specs on iPhones are more than capable enough to run the latest apps and OS.
Planned obsolescence but without having to tinker with the OS to cripple performance on older devices.
Nothing to do with cost… More RAM means more power usage. Mobile devices are optimized for efficiency. In Apple’s case, they take those optimizations to the next level in all areas of the device to maximize efficiency, while still taking performance in consideration when and where needed. Their SoC designs are a testament to their dedication to this pursuit.
Simply adding more RAM may be the easiest and most obvious method of handling the issue, but doing so limits the targets of where they may want to add this ability. What if their end goal is to add these abilities to say a watch? You can’t just cram X amount of RAM into every device and call it a day. Working through perceived limitations is what enables these things to move into areas where it was thought they never could. Most advancements are made not by working around limitations by throwing “more” into the equation, but by thinking of ways to overcome them using the finite resources given.
Unlike most other companies Apple takes the difficult route of limiting their resources when designing and developing systems. This forces them to think of novel ways to approach and overcome issues. This has enabled them to create a platform and architecture that scales from a watch all the way to a desktop computer.
This is a sound design decision, until it isn't anymore. If llms and similar technologies go mainstream, there will be no way around increasing RAM. The other way to solve this is to offload processing to the cloud. But I doubt this would be desireble long term. Creating the capability in the cloud will be very expensive and someone will have to pay for it. Most SOC on Apple devices are more than capable to process large models (watches and smart speakers are probably an excpetion). RAM size is the biggest bottleneck right now in my opinion on iPhones, iPads and all regular Macs.
True that on the price mark up, but in any case, it is my understanding that RAM as of today eats battery.
Finding a way to use storage to simulate additional ram sounds way more scalable than simply adding more ram.
Siri says I need to add that information to my contact, but I have. In years past it worked flawlessly. About 2-3 years it started acting wonky, then it stopped completely. I've even asked who my wife is, it shows the contact, then later when I go to use it in the, car it forgets. Might be some weird CarPlay thing.
😆
🤣
I've tried that multiple times. I've tried deleting and re-adding his contact information. I've tried manually adding the "husband"/spouse tag. I've told siri he is my husband, I told his name when siri said she doesn't know, I've tried everything. I'd be happy is there was an easy fix, but as indicated here in the thread: there's a bug preventing it from working.
And hopefully not just on the newest iPhones and devices. We have all waited long enough.
Sorry, didn’t know you had tried that. This might help, it suggests that problem might occur due to exchange account - and has a solution where you manually add relationships to your own contact card instead,
Siri won't add relationships? Here's a tip.
In a recent iMore article, we were told how to let Siri know about your relationships to your contacts. That way, you could tell Siri to call your mom or wife without having to say their names*. Unfortunately, some of us have found that Siri will not remember our contacts' relationships to us...
Thanks for the suggestion. I don't have any exchange contacts on my phone though, everything is iCloud-only in my contacts.
I wish it were that easy. 😄
Yup, same here, many times as a matter of fact.
That’s a great idea, never thought of that. However it suddenly started working for me again! I haven’t tried “a fix” in a while, resorting to “Call XXXX on mobile.” Of course every once in a while a “Call my wife…” would slip out. One time it finally worked again, about 3 weeks ago, and has continued ever since!Sorry, didn’t know you had tried that. This might help, it suggests that problem might occur due to exchange account - and has a solution where you manually add relationships to your own contact card instead,
Siri won't add relationships? Here's a tip.
In a recent iMore article, we were told how to let Siri know about your relationships to your contacts. That way, you could tell Siri to call your mom or wife without having to say their names*. Unfortunately, some of us have found that Siri will not remember our contacts' relationships to us...forums.imore.com
Has anybody else had trouble when sending text messages with Siri getting her to read it back to you before sending? I know that there is a way to do that if that is what you want to do, but Siri has always been untrustworthy enough that I would never send a message without confirming it first. But now, when I tell her to read it back to me she stops in mid message. She is also stopping halfway through reading my new notifications. I wonder what gives and how to fix it
Siri is terrible with CarPlay right now. She’ll ask if I want to reply to a text and will send “No” when I don’t. If i don't say anything then she’ll want to send a text with just the persons name.
Assuming 8-bit quantization, for 200b parameters you would need over 100gb of ram. Will the next iPhone have 128gb ram??
You can fill the RAM but your processor and battery is still being inundated by an LLM. An LLM has no place on a phone. It’s bad enough in a MacBook Pro.
As an example I have ran all the open source LLMs on M3 Max. They chew through the battery with the worst offender being Command-R which can deplete the whole battery in less than an hour. Even a Blender render is less stressful.
On a phone you need specialized language models that don’t need to ‘know it all’. A small size model can do many things without needing to ‘know it all’.
And you don’t need an LLM to switch to dark mode or set an alarm. That’s just voice commands which have been around since OS 9.1 classic.
Not that I think we really need massive LLMs on our phones, but that's what beefing up the neural processor, reportedly a major focus of the M4 should be good for. Running such things on a part of the silicon engineered for it is a lot more power efficient than on the general part of the CPU