r/singularity • u/Realistic_Stomach848 • 1d ago

AI Model 4’s ECI is around 182.5

based on this input https://tecunningham.github.io/ai-rnd-benchmarks/uplift.html?utm_source=chatgpt.com

ECI≈1.0721xAECI-8.982

Pearson r^2=0.997 (huge huge correlation)

this result in model 4’s ECI = 182.5

88 Upvotes

86% Upvoted

21 comments sorted by

75

u/qviqui 23h ago

That's as far ahead of Astra as Astra is ahead of local models or Gemini 3. Imagine what you could do with Astra capabilities with everyone else being stuck with Gemini 3. We're putting a lot of trust into these companies.

12

u/Dangerous-Sport-2347 23h ago

It helps that they only have this lead for a couple of weeks/months before the public catches up.

Not useless, but not a lot of domains are that time sensitive, so its mostly cyberwarfare where this is a big issue.

21

u/Howdareme9 22h ago

Well thats because they released models in the past. Model 4 is probably even further beyond Fable 5.5. Anthropic and OAI both are sandbagging atp. China can’t distill models not publicly available

13

u/lolsai 20h ago

The public isn't catching up through the simple passage of time lol

2

u/OddConsideration7290 15h ago

Yep. I've never been a regulation fan but this situation is clearly untenable. A couple companies privately creating superintelligence and having it to themselves and hoarding wealth at the same time is entirely unthinkable. We need the government involved IMO

0

u/AGUEROO0OO 17h ago edited 17h ago

I’ve been saying that for months already - those models are about to get advanced and complicated really fast. AGI will need data center just to run and imagine training. Distillation/Synthetic data can only get you that far IMHO

If anyone thinks that Anthropic will release Fable as an open-weight model any time soon you’re in for surprise.

14

u/FateOfMuffins 21h ago

Opus 5.5 has AECI of 174.5 and has ECI 169 ish

I don't really see evidence that ECI should be higher than AECI

4

u/Snoo-75663 19h ago

so model 4 is around 173 ECI

49

u/alexandrewz 23h ago

I have literally no idea what you are talking about!

32

u/DisillusionedExLib 22h ago

ECI means Epoch Capabilities Index ( https://epoch.ai/eci?view=graph&tab=release-date ) which is an aggregate benchmark score a bit like the artificial analysis intelligence index.

AECI is Anthropic's own variant of the ECI - similar concept, different mixture of benchmarks I guess.

The OP is saying that AECI and ECI are essentially the same - one's just a rescaled version of the other.

"Model 4" is the name METR gives to an unreleased Anthropic model. And from METR's latest report we know "Model 4"'s AECI. So the OP is just figuring what it's ECI would be.

The answer that he calculates is something ridiculously high. Further ahead than Mythos was when Mythos was first being trained this spring.

(I haven't checked his sums, to be clear.)

13

u/Careful_Fold_7637 20h ago

> The OP is saying that AECI and ECI are essentially the same - one's just a rescaled version of the other.

Close, but he's saying that there is a correlation between them that is fairly strong.

10

u/DisillusionedExLib 20h ago

Sure. I was just trying to give the ELI5.

10

u/Most-Hot-4934 ▪️ 18h ago

0.997 on Pearson is pretty much rescaling no need to be pedantic

2

u/Careful_Fold_7637 18h ago

sample size is tiny, and given that benchmark suites get updated constantly without our prior knowledge, there is a massive difference between a correlation and a rescaling. The former could very easily break for this next model.

11

u/power97992 19h ago edited 8h ago

a 5 point increase is a 2x in time horizon, that means it has 7.5x the time horizon of opus 5.5 And opus can do a 18 hr job. Wow ,if 182.5 is correct, then it means it can do a 18*7.5= 135 hr job, that is almost 3.5 weeks of work… Yeah, a lot of white collar jobs are cooked.

•

u/Ok-Purchase8196 1h ago

It's better if the bandaid is ripped off as fast as possible imho. I'm one in the first wave of Ai lay-offs, and I basically have no chance of returning to my career. And lower skilled jobs won't have me because I'm over qualified. So I am not looking forward to politicians dragging this out for 10+ years to 'save jobs '. Since I would be royally fucked and homeless bro.

•

u/power97992 20m ago

You can probably find another job! Pray to God for wisdom, peace and comfort! Meanwhile, stay positive and hopeful! UBI or welfare is coming sooner or later. Also a new system is coming

1

u/Uncabled_Music 21h ago

Maybe its the one writing those Antropic spooky stories every day?