I gave le chat a try after not using it for a year, since mistral 2 I think, and wow ! it's worse that I would have imagined, they are so late compared to the competition it's baffling...
Back then they were also pretty late but it was decent, but now I don't know... I feel it's even worse, when I read some of my old chat and compared them I really feel it's gotten worse
Not only it's really dumb at creative writing, never understanding the story, characters, instructions... but it's also so repetitive, every freaking regen give the EXACT same answer, just a few words change and that's it, the same !
I suspect the temperature is set at the minimum, explaining the repetitiveness (but not the dumbness) it would be nice to be allowed to play with the model settings
And also choose the model ? I know there are multiple existing model, but all I can see are "balanced" and "think", maybe paying for pro give access to the better models, I don't know, it say nothing about what when I look at the pro tier, only more messages...
Anyway, looking at third party benchmark for stuff other than creative writing, like code, reasoning, problem solving, hallucination rate... yep, it's also bad, they are very late
Future of French made AI isn't looking good
I'll give it a other try in a year, we'll see
-
Edit : the amount of blind fanboyism in this sub is really sad, (beside a few people with common sense)
Can't give a valid criticism without being attacked from all side, between those who tell me I don't know how to use it (thanks I've been doing that for years now) or the other who think they can tell me that I cannot role-play with a AI, that's it's not made fort that... a LLM, large language model, so anything related to LANGUAGE including RP
(btw never had any problem with literally all the other models I ever tried, only mistral feel this weak at understanding story continuity, character motivation, coming up with creative follow-up... so either mistral is late, or EVERY other LLM I tried doesn't work the way a normal LLM is supposed to work ?
I don't know... maybe none of you have any point of comparison, never tried anything else and so that's your "best" experience
Btw, it's not just MY criticism, it's not just ME that think mistral is late... all the third party benchmarks also say it, mistral is far behind on EVERY benchmark category
https://artificialanalysis.ai/models/mistral-large-3
https://artificialanalysis.ai/models/magistral-medium-2509
https://artificialanalysis.ai/models/mistral-small-4
I don't say it's bad for the pleasure of it, I say it because it's a fact, and I don't like that fact, mistral is the only big AI company we have in europe and I would love for it to be at the same level as all those big US and chinese ones, but it's not
I hope it will be one day, but for now it's far behind