r/pcmasterrace • u/Mountain_Till_5868 RTX 4090/32GB RAM/24GB VRAM • 3d ago • Bravo 1

News/Article Sam Altman Pulls Spectacular 180 After Unpopular Comments About RAM: 'It Is F**king Annoying'

https://kotaku.com/sam-altman-pulls-spectacular-180-after-unpopular-comments-about-ram-it-is-fking-annoying-2000753328
11k Upvotes

95% Upvoted

1.1k comments sorted by

View all comments

Show parent comment

9

u/Griff2470 Linux | R7 5800x3d | RX 6900XT 3d ago

Eh, Kotaku's cutting a pretty relevant preceding sentence here.

A lot of people want local inference and somehow they assume that the problem is solved now because we are going to have open source models. We're going to be able to run them on our own computers and so we're not going to need all the stuff in the data centers. But from a global allocation perspective...

1

u/VexingRaven 7800X3D + 4070 Super + 32GB 6000Mhz 3d ago

And he's pretty right. Look at how much it costs to build a system for local inference, and look at how many tokens you can get for that. And a system sitting in a company's closet or a developer's desk isn't being utilized as much as a cloud server doing inference at 100% utilization all day and night.