I don’t even know what I’d do as CEO.
Pat’s direction was spot on. Cut “side” divisions to focus on their core, stick with long term bets most CEOs wouldn’t (like the GPU division) and try to cut the delays.
…But the delays keep coming!
If they just can’t launch products on time with whatever rot is in the company, I’m not sure what Intel is supposed to do.
…So whoever they hire as CEO now is probably there to just distribute golden parachutes and eat the company as it dies :/
No, but it’s not unimportant.
As dystopian as it is, we are stuck with AI, and there are two futures:
Stagnant, earth burning, censored corporate APIs.
Open weight, power efficient, extensively community modified, locally or “swarm” run models.
And it really is urgent, because the later has a great fighting shot now, but funding/regulation dries up for it, it will never catch up again. Altman an the “AI bros” know this which is why they are pushing the AI danger angle so hard… so you can only by it from them.
If there’s any good that comes of this blatant corruption, I hope it’s Trump leaving open source ML alone instead of giving into whatever monopoly Altman is pushing.
There’s basically a fediverse vs Facebook war going on within AI land, and the tremendous irony is that Zuck is a huge open weights proponent.
Their reputation and past reporting is supposed to back up things they state as facts (like assuming that reviews they cite are real) for practicality and brevity. Imagine having to document every bit of background research in a presentable way.
They could have included screenshots though.
And the skepticism is healthy. I do personally ‘trust’ Axios (which I read almost daily but regularly double check).
The American way is to bail them out financially, then re-elect them.
And a small cluster like Alibaba used to train Qwen 2.5 is basically a drop in the bucket.
The hoard of GPUs Meta, Microsoft/OpenAI, and especially X have are apparently being used extremely inefficiently, or perhaps mostly not used to train AI at all, but do regular ad/engagement optimization stuff.
Ah yeah those were the good old days when vendors were free to do that, before AMD/Nvidia restricted them. It wasn’t even that long ago, I remember some AMD 7970s being double VRAM.
And, again, I’d like to point out how insane this restriction is for AMD given their market struggles…
Strix Halo is your unicorn, idle power should be very low (assuming AMD VCE is OK over quicksync)
Just that bursts of inference for a small model on a phone or even a desktop is less power hungry than a huge model on A100s/H100s servers. The hardware is already spun up anyway, and (even with the efficiency advantage of batching) Nvidia runs their cloud GPUs in crazy inefficient voltages/power bands just to get more raw performance per chip and squeak out more interactive gains, while phones and such run at extremely efficient voltages.
There are also lots of tricks that can help “local” models like speculative decoding or (theoretically) bitnet models that aren’t great for cloud usage.
Also… GPT-4 is very inefficient. Open 32B models are almost matching it at a fraction of the power usage and cost, even in servers. OpenAI kind of sucks now, but the larger public hasn’t caught on yet.
Oh also you might look at Strix Halo from AMD in 2025?
Its IGP is beefy enough for LLMs, and it will be WAY lower power than any dGPU setup, with enough vram to be “sloppy” and run stuff in parallel with a good LLM.
You could get that with 2x B580s in a single server I guess, though yoi could have already done that with the A770s.
Also “ridiculously” is relative lol.
The Llm/workstation crowd would buy a 48GB 4060 without even blinking, if that were possible. These workloads are basically completely vram constrained.
Like the 3060? And 4060 TI?
Its ostensibly because they’re “too powerful” for their vram to be cut in half (so 6GB on the 3060 and 8GB on the 4060 TI), but yes, more generally speaking these are sweetspot for vram heavy workstation/compute workloads. Local LLMs are just the most recent one.
Nvidia cuts vram at the high end to protect their server/workstation cards, AMD does it… Just because?
deleted by creator
There’s no way. I think he’s legit too old, and he only needs 1 term to pardon himself and everyone he wants.
Are the double capacity GDDR6X ICs being sold yet? I thought they weren’t and “double” cards like the 4060 TI were using 2 chips per channel.
I mean, if you even have to go into the bios or dip into the mechanics of drive letters and formatting, you have already lost most people.
I feel like I would reach for the used market over a 7600 based on what I read. Even if there were power constraints, one could power cap a 6000 series card.
…Hence I got a 3090 when the 4000/7000 series was long out.
Well AMD won’t do it ostensibly because they have a high mem workstation card market to protect, but the running joke is they only sell like a dozen of those a month, lol.
Intel literally had nothing to lose though… I don’t get it. And yes, this would be a very cheap thing for them to try, just a new PCB (and firmware?) which they can absolutely afford.
That’s, unfortunately, the position of everyone “big” in the open weights ecosystem now, just in different flavors.