Apple has new Mac app coming to make running local AI models easy (9to5mac.com)

41 pointsby linolevanan hour ago7 comments

pen2l 16 minutes ago

Apple, to me, is the winner in the end of this AI chapter.

Distillation works: we can get models leaner, have better performance with lesser and lesser parameters, more efficient, running locally. Apple gets this and we'll see the result with M7 Max/ultra chips and I reckon this will be one of the titular apps when M7 line of chips are introduced.

I guess it explains the haste Anthropic and OpenAI are in to IPO, cash out and leave the the retail investor holding the bag.

bigyabai 15 minutes ago

Man, if all that makes Apple a "winner" then I'm dying to hear your assessment of Nvidia.

Factoring in the increased cost of memory, Apple's threatened service revenue and the stagnation of ecosystem products like the Mac and iPad, I think they have a really challenging future to navigate. The whole "sell people a B2C ecosystem" business model is more threatened than ever.

pen2l 13 minutes ago

The contrast I'm drawing is running these models locally vs on the cloud and how that would be a selling point well received.

bigyabai 12 minutes ago

I don't think that's materially different from what you can do with Nvidia and AMD GPUs, besides the fact that you're limited to Thunderbolt interconnect.

n_u 4 minutes ago

I think the question to me is whether Apple can ship affordable consumer devices with enough memory to run sufficiently useful models sufficiently fast.

Right now 128GB MacBook is the top of the line and you could do Qwen 3.8 Flash or similar.

There is a rumor that the M7 chip can use 1.5TB of memory which would imply Apple is aiming for more memory on devices.[1]

Would that imply the entry-level MacBook will have 128GB and top of the line models could run larger reasoning models? Then the question is price and if any of this will translate to iPhone for on-phone models.

[1] https://www.tomshardware.com/tech-industry/semiconductors/ap...

nomel 3 minutes ago

Cloud compute has been completely commoditized. Yes, local models will get better and better, but I think for the vast majority, it will continue to be both cheaper and more productive to use cloud based models, far into the future, where bulk purchasing, oversubscription, and discounted industrial electric rates can be taken advantage of.

Where I am, it's cheaper to purchase multiple claude subscriptions than it is for the power bill from running a local model. Factor in the hardware costs, and you're looking at a break even a decade in the future. Yes, there's privacy, etc, but now you're down into fractions of a percent of users.

mgiampapa 10 minutes ago

I love that they are bringing Claris back for this. Moof!