Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Apple Develops Breakthrough Method for Running LLMs on iPhones
- Thread starter MacRumors
- Start date
- Sort by reaction score
Yes, though I would say that model needs fine tuning. Here are some statistics on the A series with a couple small models: https://github.com/ggerganov/llama.cpp/discussions/4508
With my husband it stopped working a couple of years ago as well. She just claims not to know who my husband is, even though it’s indicated in the contacts app.
I tend to agree with you. It seems the people most thrilled by ChatGPT are people who live in a world of words, and who find the ability of ChatGPT to remix words to be valuable. I'm not one of those people, and while I'm impressed that ChatGPT can make up a song, I'm unimpressed with its ability to do something I care about, like perform actual research on something unknown (as opposed to looking up reference material).
BUT
(a) many people apparently do find many aspects of this useful. It's absurd to deny them a tool because I don't find it useful.
(b) it's possible (early days...) that Apple, by embedding the tool in the OS and even providing APIs, will generate value in a way that doesn't yet exist. For example
- something I find a huge hassle is when I want to refer to something I read a year ago. I can sometimes establish a hook that can find it ("I think I mailed it to my brother?" "I think it was on AstralCodexTen?") but even that's a slog, and if all I can remember is the gist of the argument, forget it.
If Apple could track everything I read (on all devices) [and hell, why stop at reading, also track the lectures and audiobooks I listen to] and I can refer to something by whatever vague memories I have to perform the lookup, that's real progress...
- we can be less ambitious. If can simply perform local searches by semantics (ie considering word context, considering synonyms, etc) rather than exact word match, that would be a huge boost to Spotlight
- consider everything you'd want from a smart secretary. Summarize large repetitive documents. Tell me what's different between this file and that file. Read my email (or posting or comment) and tell me if it's too emotional, too predictable, whatever.
Much of this stuff is not available today, and will not be available tomorrow. My personal secretary needs to be personalized to me, to understand the features I WANT in my writing and the features I want removed, the things I find interesting and those I find a boring waste of time, etc. I think much of this is not yet done well, let alone in a personalized fashion.
But everything starts small. NIST Special Database 19 used to be state of the art for huge machines not long ago, now we just expect that we can cut and paste text in any random image. You get to where we want to be one OS release at a time.
This sort of thing is irritating, but it's dumb to blame it on the "AI" part of Siri. These things tend to be run of the mill coding bugs, especially, in Apple's current OS's, some stupidity related to security and permissions.
We all hope Apple will fix them, but it's not going to the AI team that fixes them, and blaming the AI team all you like won't change that.
What do you mean? I can still do that with mine. I call your wife all the time.
I really hope Apple gives us an option to disable and completely remove it. I don't want AI on any of my devices.. i've survived for 60 years without AI, pretty sure I can do without it now.
ChatGPT is only proof of concept that llms can be used to complete human tasks quicker. You can already personalise ChatGPT to yourself. You do this by either:
-Creating your own GPT using ChatGPT premium or use a chatbot service. Either option requires no coding and cost $20-40 dollars per month.
-Give it base prompt instructions e.g I am a female, financial analyst, these are my priorities, hobbies etc.
-In the base prompt, include 2-3 examples of your preferred writing. This is crucial.
-Feed it documents that you consider important sources.
Now, anytime that you ask it a question, the responses will be personalised to you. At first I was unimpressed by ChatGPT but once I understood and did the above, I use gpt4 chatbots everyday to expedite my work. I use ChatGPT plus to:
-Proof read and improve emails and documents for clarity.
-Give a 2nd perspective on my thinking e.g pros and cons.
-Find facts quicker than what it takes to use Google.
-Write and debug python scripts.
This is no different to when Search engines launched. We didn’t know how to use them but learned how to scroll, click and find instead of going to a library to find a book with the answers that we need.
Right... every day I say "Hey Siri, stop in 15 minutes" to set the music sleep timer. On most days she does set the sleep timer, but on some days she only replies "You have no timers set" or she deletes a non-existing timer. So she indirectly tells me that I have to pull the power cord of the HomePod to clear her memory.
Faceid and touchid are ml/ai based and run on device.
Spotify and apple music‘s recommendation engines are ml/ai based and have solved the problem of music discovery.
When you take photo you are using ml/ai. 10-15 years ago you had to set white balance, iso, shutter speed etc manually or use Aweful auto settings that phones and cameras had back then. Now ml/ai does the settings and post production for you. No need to post process the majority of your photos.
These are example of purposeful ai that’s embedded in to apps that we know and love. Bad ai is
-Facebook and TikTok’s content algorithms.
-Siri
I don't rely on any of those things on my iPhone. people need to learn to go back to using their head for something besides a hat rack.
I agree with your vision of a personal digital butler…that would be truly game changing ..but it kinda was the promise of Siri and we know how this turned out.
I have little hope that Apple will create such an all knowing , all encompassing system…
Amazing. Thanks for sharing the link. This is a promising. I wonder whether the ability to leverage flash memory where VRAM low, will make it possible to store Llm for an iPhone 15 on an external ssd.
Granted, the 1gbps transfer speed of ssds is just theoretical and not as fast internal ssd storage.
I tend to be a sceptic too and also think that there is a lot of hype currently around llms. But it's early days and the field is constantly progressing and evolving. Also, the underlying technology has many more applications than just general purpose chatbots. Think about the Whisper model by OpenAI for transcription and translation, which you can already use locally on beefier desktop computers. I tried it myself and it's amazing how accurate it is. Also llm chatbots are quite popular with programmers already. They won't write the whole program for you, but can help with producing a lot of boilerplate code just by asking the right questions in written.