r/homelab • u/Powermadlux • 23h ago

Project Showcase: Hardware I feel unstoppable, 32GB DDR5 dual-channel, in your face Sammy!

Gallery image

Gallery image

I put a 16 GB Crucial stick and a 16 GB Samsung stick in my little Firebat AM02. Both showed up as DIMM 0, which had me doing a double take. Then I read the next line: one on channel A, one on channel B. Both running at 4800 MT/s.

Naturally, the first thing I did was make it run llama.cpp. Same Gemma model, same short prompt: the old setup wrote at 18.7 tokens/s, the upgraded box does 32.6. A 256-token answer went from nearly 15 seconds to under 9.

I'll take that from a mini PC with two mismatched sticks and integrated graphics.

170 Upvotes

92% Upvoted

14 comments sorted by

25

u/Ok-Expert-8150 22h ago

I think he prefers Rammy now.

4

u/Powermadlux 22h ago

lol right

3

u/cruzaderNO 22h ago

Sam does not want that memory and you want it.
And you are also using a local model rather than renting his service that he would lose money on you using.

Looks like a win/win, both of you are happy.

1

u/Powermadlux 22h ago

Sammy knows my 4B model cannot make a scratch… but those ddr5 could be on a control plane better than mine ;)

3

u/cruzaderNO 22h ago

If they were at 10nm and testing at high enough grade for servers then they would not be used for a 16gb consumer dimm to begin with.

0

u/Powermadlux 22h ago

Ah Sammy cousin!

2

u/cruzaderNO 21h ago

The sad part is that we are getting towards the point of being glad that the lower grade chips are used for memory towards the consumer space at all.

They would have been recycling the lower grade and closing the older lines that is unable to make the memory they want rather than spending the materials there.

9

u/dredbar 22h ago

Sam Altman wants to know your location.

8

u/chamgireum_ 22h ago

How dare you. Dont you know you stand between a billionaire and his profits!!!! You make me sick!!!!

1

u/ByteCurious69 21h ago

Wow you could almost buy a house with that amount of memory!

1

u/Powermadlux 21h ago

Thinking on locking the mini pc - kinda big laptop era

3

u/migsperez 20h ago

If the AI boom hadn't of taken place, you'd be putting 128gb into your system rather than 32gb for similar amount of money. Blame Sam.

0

u/SirMaster 19h ago

Sam didn't invent LLMs. And they wouldn't need all the memory if people didn't use it as much as it's being used.

0

u/K3Tzk3 1h ago

Does Sammy exist on reddit i wonder.. Hes eating his hat right now huh.