Apple Caught Off Guard by AI Demand for Mac Mini and Mac Studio

There is a lot of "AI demand" that isn't just running inference on an LLM whose weights you downloaded.

I'm training a model using reinforcement learning with self-play. I can and do use vast.ai when scaling but for experiments it's far faster, and cheaper, to run it locally until the bugs are all figured out. Just provisioning a new instance and copying the relevant checkpoints and things can take 25 minutes. It's zero locally.

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have.

I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me.

Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardware to get there.

It's fun to see that even an extremely large company can find unexpected product market fit [0]. Per this article, "The company reportedly did not possess an engineering team dedicated to business customers or staff focused on developer relations, and lacked an enterprise AI strategy." That sounds insane in retrospect, but I think there's just inherent uncertainty in what people actually need and will use things for.

[0]https://pmarchive.com/guide_to_startups_part4.html: "In a great market—a market with lots of real potential customers—the market pulls product out of the startup... The product doesn’t need to be great; it just has to basically work."

Mac Mini's were really nice HTPC candidates, too, before the AI boom. Like all things genuinely useful and affordable, they were snatched from the hands of normal consumers by a bunch of schmucks chasing the latest gold rush.

What’s going on isn’t Apple behind in AI model building I thought I read that somewhere on the MacRumors site in the last two years, that Apple is behind its tech peers and Apple might as well just close the doors.

I always thought Apple was in a good position because unlike their peers they didn’t burn billions of dollars trying to build an AI model that has no financial moat around it.

I still think they’re in a good position in comparison to their tech peers and we will know even more when some of the new computers get into the hands of some of the tech reviewers.

I believe the new computer’s will be pretty good hardware wise what I’m interested in, is the Apple software support for connecting several Mac computers together, and some of the other (new?) software that Apple may have written in house to support those who want to run AI software locally that is just as important as the new hardware.

I really hope with Ternus taking the helm Apple starts to remember that it has products outside of iPhone.

Not just the high end stuff. The Neo is sold out until late September on the budget end, it seems like it is a smash for HS and college kids.

I hope Apple can take all this cash and do some stability releases like they used to do, bugs around things like Family Sharing, the painful "update" to Settings App, etc could all use a lot of love.

Most of those people will be dissapointed when they experience Q4 variants of those models getting stuck in loops.

I would wait till the ram crisis is over to fetch a future 64gb ram gpu to run Q8 models. Cloud inference until than.

I get the impression they want AI marketing points but don’t actually want people to use local AI on their products.

I have no idea why. They could be so successful if they leaned into local AI.