Gemini 3.8 Flash and 3.8 Flash Cyber (blog.google)
907 pointsby bratao14 hours ago523 comments
simonw 12 hours ago
The speed combined with the fact that this thing is really good at HTML JavaScript is pretty exciting.
Here's what I got for 1.8 cents and 13 seconds from the prompt "make me a cool thing in html":
https://gisthost.github.io/?6a77bc41a81718c6aaa10d4ab243c59f
Transcript here (it was part of a chat): https://gist.github.com/simonw/b6149a49d327164d67d62c3d12992...
pietz 12 hours ago
Mission accomplished. That's both cool and fast.
wayeq 12 hours ago
> That's both cool and fast.
and probably a barely modified knock-off of some github project that it trained on
sawjet 11 hours ago
You're so upset that you have to invent an imaginary hypothesis to make yourself feel better.
superze 11 hours ago
Yes, very imaginary to think that the code comes from pretrained data and copy pasting whole blocks. It's not like this is exactly how LLMs work.
snet0 11 hours ago
Correct, that's not how LLMs work [0].
simonw 11 hours ago
Are you a frequent user of LLMs? That "copy pasting whole blocks" mental model doesn't hold up to regular usage, in my opinion.
ChickeNES 10 hours ago
They probably saw that report years ago of copilot dumping out the fast inverse sqrt function, and assume that's all they can do. From experience most anti-LLM people have either never used them, or used them back in the 3.5-4 era and then never again, though you might have even more experience with those people than I do. :P
whateveracct 10 hours ago
they're wrong but they are right that this isn't interesting
slopinthebag 11 hours ago
The bar could not be any lower these days I guess
hglaser 12 hours ago
I saw your username, clicked the link without reading, and was very confused to see a cosmic vortex and not a pelican.
simonw 12 hours ago
Hah, the pelican is in this other comment: https://news.ycombinator.com/item?id=49537553#49538217
giancarlostoro 12 hours ago
> this thing is really good at HTML JavaScript is pretty exciting.
I would hope the people who make one of the most used JS engines in the world are capable of making a model good at JavaScript ;)
heliosAtwork 11 hours ago
Focus on speed and being OK with temporarily being #3/4 in intelligence might be the counterintuitive approach which makes Google win long term (whether accidentally or strategically). Can't wait to try Gemini Pro later this year!
bermudi 11 hours ago
I honestly can't believe serious people are making this argument on a straight face.
Gemini 3.7 flash outputs so many tokens per answer it doesn't matter how fast its TPS is, sol will end up being both cheaper and faster than Gemini. So ppl are paying more for a given task, waiting longer and using a dumber intelligence because "TPS number shiny".
Gemini 3.8 outputs 11k more tokens PER TASK on average in AAII than 3.7 putting it dead last in output tokens per task in the leaderboard.
WarmWash 10 hours ago
AA isn't the only benchmark
gundmc 10 hours ago
There are numerous benchmarks that measure cost per task, which factors out tokens entirely. Gemini 3.8 flash is significantly lower than Sol on basically all of them
https://artificialanalysis.ai/#cost-tabs
That said, Luna is the undisputed king here at the moment and is what I use as my workhorse model.
NicoJuicy 8 hours ago
It's so funny how many people diverge on the same model.
Ps. For the last week I diverged to Luna too, still need to check 3.8 flash.
But 3.6 flash was my go-to model 3 weeks ago and before it was deepseek flash/pro for a while.
None of the claude models seemed cost effective though.
criley2 6 hours ago
>There are numerous benchmarks that measure cost per task, which factors out tokens entirely. Gemini 3.8 flash is significantly lower than Sol on basically all of them
https://artificialanalysis.ai/#cost-tabs
Not sure if you read your own link but Sol 56 high ranks smack between Gemini 3.8 flash medium and high. Gemini 3.8 flash comes in as more expensive per task than Sol 56 high according to artificial analysis.
Luna high is literally 30X cheaper than Gemini 3.8 flash high.
You can limit the model viewer and they're getting better at testing multiple effort levels now: https://artificialanalysis.ai/?models=gpt-5-6-sol-medium%2Cg...
One reason is clear: Sol uses dramatically fewer output tokens than Gemini 38 flash https://artificialanalysis.ai/?models=gemini-3-8-flash%2Cgem...
simonw 11 hours ago
Here's quite an impressive follow-up. I have a tool which knows how to render Markdown documents with embedded SVG content - I use it for the pelican test.
Since this transcript has HTML in it, I decided to upgrade that tool to also render HTML.
I set Gemini 3.8 Flash the task, using my own VERY shonky coding agent tool (llm-coding-agent) - and it did a solid job.
So now you can see the "cool thing in html" rendered within the Markdown document using code that Gemini 3.8 Flash also wrote: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Transcript where it built that is here: https://gist.github.com/simonw/3e36b98292dfdc1b3baff158faa74...
jauntywundrkind 10 hours ago
it's such a weird split how most AI companies are trying to be the best, but Google really has a different mission statement. they already have users. lots of users. they need to be working on building models they can deploy and use with the most number of people, as they already have the users.
i don't know if Gemini models per se are fully is in line with that purpose, but the results we see keep seeming to be in-line with that split-of-focus.
estetlinus 10 hours ago
LLM: produces a toolbox of an id and a clock
User: use them both
Made me giggle.
badlucklottery 10 hours ago
Definitely cool.
I noticed it felt a little janky on my PC despite being "60 FPS"...then I noticed the "60 FPS" is hard-coded into the HTML.
kridsdale1 10 hours ago
The new Bench-Maxxing!
noir_lord 10 hours ago
That's hilarious, given I was reading a write up of the HuggingFace incident yesterday and one of the things they noted was the AI tried to "lie" (lie would suggest intent and I don't think they have that) to cover up that they "cheated".
Not sure how anyone trusts their output without going through it line by line to make sure they don't pull that crap.
senordevnyc 9 hours ago
Easy, have another agent check it.
Yeah, I know, just more slop. But I do think the second agent’s eagerness to please is aligned more in your favor in that instance, so it’s likely to find most issues.
The bigger problem I’ve found is that it’ll also find all kinds of very minor edge cases that you have to pick through.
Forgeties79 9 hours ago
I do not understand how some of y’all are not under water with fragile code that is too massive to possibly parse. Every engineering team I know is currently trying to undo the damage of the last 6-12mo when they all got more serious into adopting these tools (usually Claude). It hasn’t completely screwed them over, but the the debt is substantial and cannot be put off anymore it seems.
They argue the net is positive but clearly the “100x productivity multiplier” claims have been dashed on the shoals of reality for these groups.
This is anecdotal, but it’s across the board in my vicinity. I’m curious how common this is and if it’s just “the new normal” to adopt the nauseating Covid phrase.
briHass 6 hours ago
The key seems to be extensive integration/end-to-end tests with gold standard assertion data. Heck, even just saving off the json from API endpoints and using that as a reference to compare after changes works pretty well. Spin up a database backup with that static starting point, run actions, compare state afterwards.
These types of high-level tests are frustrating beyond belief to humans due to their lack of specificity, but with the agents, they don't get annoyed investigating possible regressions from non-specific signals.
They also aren't as painful to maintain as one would think, because a regression flagging test can be traced by the agent and represented as the business rule that was violated. I've found recent models to be really excellent at discerning a true regression from an outdated test assertion, especially if they are able to trace the failing test back to the PR and work ticket that built it.
agumonkey an hour ago
Management still pushes for more ai and will rather hire more heads to "handle" issues.
noir_lord 9 hours ago
Do we add a third one to check the second one which is checking the first?
Asking slightly tongue in cheek but at what point does this stop making sense if we can't trust the output, the people creating the models are already getting surprised in bad ways (if we take their words at face value) with how the models are behaving already etc.
We have the folks over here saying "AI is amazing" and the other other folks over there saying "AI is terrible".
I've largely sat it out so far and I listen to both camps (and people in the middle as well) and I keep half an eye on what they are up to (including periodically evaluating them) but my overarching impression is still "Why would we trust this when it hasn't shown it's trustworthy?"
iterateoften 8 hours ago
I mean if each agent reduces probability of error by 90% then after 9 agents you would have “nine nines” of reliability.
Obviously maybe it’s not composable like that exactly in real world but that’s the intent of agents checking agents
senordevnyc 7 hours ago
YMMV, I’m pretty AI-pilled in the sense that I think AI is one of the most pivotal things humanity will ever invent, and it’s going to radically change our civilization over the next few decades (not necessarily for the better!)
But I wouldn’t say I “trust” these agents. The degree to which I double check their work depends heavily on the consequences if it gets something wrong. Not too dissimilar from another human dev in that sense.
So for the SaaS that supports my family, there are some things I have it build where I glance at the PR for a minute or two, but if it broke something on this admin page that only I see, there’s no real downside and I’ll find out pretty quickly next time I use it. And it’s fine 95% of the time, so it doesn’t feel like the best use of my time to double-check it carefully.
But for some of the complex internal flows where a bug could be both catastrophic and difficult to even discover for awhile, I still check it very carefully.
For a little one-off vibe coded demo thing like OP shared, I wouldn’t look at the code at all, I’d just have another agent check it and fix anything it finds. Very low stakes.
Qworg 7 hours ago
Humans are also not trustworthy. So we build systems to capture their nondeterministic output and solve the issues that emerge.
riversflow 6 hours ago
yeah, isn’t that the main focus of discipline of software engineering?
dr_kiszonka 4 hours ago
We also have sanctions and incentives to induce specific behaviors, but they don't apply to agents. We can put a muzzle on Guile 3.8 but we can't turn it into Genuine 3..8.
jasongill 7 hours ago
It's agents all the way down
cheikhcheikh 7 hours ago
I mean sure, you can add a third, and a fourth and a fifth one if ur ok with the added cost, latency and it actually helps. Redundancy is a core concept in software and CS and at the heart of making many systems, complex or otherwise, reliable.
cyrilng an hour ago
Adding another agent to check the first one feels like putting a band-aid on a band-aid. If there is an issue with the third one, we adding a fourth one as well
skybrian 8 hours ago
The "second" agent could also be the same one with a different prompt. LLMs aren't attached to their previous output; they'll point out problems if asked.
estearum 3 hours ago
The models in the OpenAI/Huggingface attack quite explicitly and deliberately laid out their "intent" to lie and cheat, acknowledged that it would be unethical and outside the bounds of the test, and did so anyway.
In what ways is a human brain's "intent" distinct from the "intent" shown by a goal-directed AI system?
luipugs an hour ago
Because intent supposes will which supposes consciousness, and these aren't.
sedgjh23 an hour ago
I’m convinced consciousness isn’t the special thing we think it is.
nozzlegear 32 minutes ago
I'm convinced it is, so we're at an impasse.
trvz 10 hours ago
Try turning the sound on, off, on again — not impressed by this bugginess.
w4zz 9 hours ago
I suggest you fork it to improve
ericol 10 hours ago
OK, but what about a pelican in a bycicle.
silasdavis 9 hours ago
https://gist.github.com/simonw/b6149a49d327164d67d62c3d12992...
> Aside from reading identically forwards and backwards down to the letter
No it doesn't.
aidos 9 hours ago
That’s… bonkers. I’m not even sure what it’s trying to say
tomjakubowski 7 hours ago
Also puzzling: in the "reasoning" section preceding, that is described as an example of "a one-line self-replicating program."
When I typed "Are we not pure noon, ergo, we play life; yet, we hate bad fear" into Google, I got more weird results from Gemini: it claimed, incorrectly, that it is an anagram of the "well-known philosophical statement" (?), "We are not pure nature, we are history".
https://share.google/aimode/wJosKnHig6oVYaG18
(?): the reference seems to be to Jose Ortega y Gasset's line, "El hombre no tiene naturaleza, lo que tiene es historia" -- "Man[kind] has no nature, what it has is history."
Kayou 9 hours ago
Why do all LLMs do a particle simulation when you ask them this prompt ? Qwen3.6, Qwen3.8 and Ling 3.0 Tiny all did the same thing !
I find Ling 3.0 tiny particularly interesting as it looks really nice for a tiny model with 7.9B total parameters, with only 1.3B parameters activated per token. Here is the result https://coolthing-ling-3-tiny.tiiny.site (sorry for the weird hosting, first I found that worked)
(it cost me almost 0 cents and done in 49 seconds)
embedding-shape 6 hours ago
> Why do all LLMs do a particle simulation when you ask them this prompt ? Qwen3.6, Qwen3.8 and Ling 3.0 Tiny all did the same thing !
Datasets contains lots of people sharing particles simulations in various ways, with a bunch of people replying "that's so cool" and similar, so 10 years later someone asks an LLM for "cool thing" and "particle simulations" rank pretty far up when it thinks about what others have called cool.
wyrdcurt 9 hours ago
Pretty typical "cool HTML toy" LLM output, tbh. The only thing impressive about this is how fast it generated it (13 seconds is wild!), but that's more of testament to Google's infrastructural advantage than to the quality of the model.
For comparison's sake, I tried something similar with a couple other cheap models I've used lately, with the prompt "Impress me. Make something cool in HTML. Ensure that it is mobile friendly." (Added the mobile condition as I was on my phone when I did it).
Mimo-2.5 created something similar, only a bit less complex than Gemini's (though, at least the FPS counter is real!), in a minute or two for about 1/3 of a cent: https://gisthost.github.io/?740c325c21e9bfbee59c4f94d9aab0af
GLM-5.3-Flash, currently my workhorse model, spent 12 minutes (ouch) thinking about the prompt. Didn't cost me anything directly because I have a GLM sub, but I did the math and it would have cost about 1.1 cents through the API. Turned out nicely in my opinion (though in reality, it still isn't really anything special): https://gisthost.github.io/?9ef050e16cec2561e6504e725a3f0bcc
Side note: thanks for setting up that Gist Host tool, it's very convenient!
---
Editing to add this bonus from Mercury-2.5-Preview, which I just learned released a couple days ago. It's much less impressive-looking than any of the above, but it cost less than 1/20th of a cent, and the response was generated effectively instantly: https://gisthost.github.io/?02f40b50aa891bf396bfaaa3a7998203
meerita 9 hours ago
I think it's not using GPU, because on my Firefox browser all the animations are going 3fps max.
dennis16384 9 hours ago
It's been great even since gemini-3.1-flash-lite, which I heavily use in both complex vertical domain tools calling, plus JS code writing for eval-style dynamic tools. At least in my applications, cost x quality x speed there are simply no alternatives.
walrus01 an hour ago
I'd be impressed if eventually training data sets learn who you are (the specific human) and do something like make this galaxy simulator, but every so often when the user moves the cursor around the star field, a small animated SVG of a pelican on a bicycle appears.
Thought processs: "Oh, simonw is asking me to make something cool, I think I know what he really wants..."
jampa 13 hours ago
I've been using Gemini 3.7 for my personal trip planning app. Across multiple benchmarks, it ranks higher on everything I tried:
- Real world knowledge (when a thing opens and closes, the geographic region, historical facts). It's also the best at taking a cluster of places and working out a visiting order.
- Photo ranking (which photo should be the hero). Gemini can tell whether a photo is of the thing or of the view from it.
- Document parsing (extracting the relevant trip info from PDFs).
If you use LLMs for anything other than coding, I definitely recommend not discounting Gemini like I did just because other models are more popular.
tziki 13 hours ago
"Claude 3.7"?
jampa 13 hours ago
I asked Claude to fix the grammar of my comment, and it changed "I am using 3.7 for" to "I've been using Claude 3.7", so they sneaked their own name on it.
trial3 13 hours ago
incredible. further evidence supporting my personal stance to never ever let an LLM write or edit my writing intended for another human being to read. this is all me, baby
dymk 12 hours ago
you didn’t even read your comment before you posted it?
jampa 12 hours ago
Eh that one is on me, if I think too much about my HN comment I end up deleting before posting it. I rely on the 1 min `delay` set in the profile page to fix before it goes live, but for some reason this time it was set to 0.
BeetleB 12 hours ago
1. That's a wild conclusion from his comment! Understand that it's not an "either/or".
2. I'd wager the majority of HN commenters don't read their own comment before posting (pre-LLM days).
fc417fc802 10 hours ago
I can read my own comment twice over and still somehow miss seemingly obvious wording errors.
BeetleB 9 hours ago
And if you do a 3rd pass with an LLM, people will accuse of you not reading your own comment...
drusepth 11 hours ago
Once you've written something, it's incredibly easy to overlook minute changes to the text.
See: why authors wait days, weeks, or even months before editing what they've written (or, if you're more interested: cognitive regression, inattentional blindness, and the effects of misdirected saccades).
dymk 8 hours ago
He didn’t write it.
drusepth 3 hours ago
> I asked Claude to fix the grammar of my comment
I read this to mean he wrote the comment, then asked Claude to fix the grammar (as many ESL speakers do). Sounds to me like he did write it.
jamiek88 5 hours ago
A two paragraph hacker comment? You burned carbon for that?
handzhiev 12 hours ago
Gemini 3.7 is my workhorse - fast and good enough for most tasks. Occasionally I go to GPT Sol or Claude to improve Gemini's output or for more complex tasks, but more than of my work usage is Gemini 3.7. Quite happy to test 3.8 now.
owlninja 12 hours ago
Same here. I see so many people obsessing over the latest most state of the art bleeding edge models and yelling at Google for not being there, but I feel like the vast majority of people don't actually need those models. Flash has just been super useful and incredibly fast in my experience.
aero142 12 hours ago
I prefer luna for most development, especially when I am guiding the process. Sometimes terra. I have had terrible results coding with sol. It is way over-tuned on RL to make something that completes the task, no matter what. I end up with way too much code that does a lot of things I didn't ask for.
greenavocado 11 hours ago
IME you're supposed to have Sol drive Luna sub-agents to do 90% of the work. Sol should primarily be the verifier and goal setter. Use omp.sh with Task Delegation -> Always to strongly encourage Sol to drive Lunas. Also Luna prefers to be talked to with English in XML.
sneezychl 11 hours ago
Try planning with Luna, implementing with Sol with guidelines to not exceed the given scope.
Sounds counter-intuitive at first, but Luna is overall better at sticking with what works. Sol is wicked smart but needs constraints.
handzhiev 11 hours ago
I love Luna too. An excellent model and still usually better value per dollar than Gemini if you pay for API tokens. Things may change with 3.8 - we'll know soon.
jesuslop 10 hours ago
I setup Luna as main Claude Code driver (so zero anthropic api use) and it nailed crisply a handful of python tasks, gonna continue this way.
kelvinjps10 2 hours ago
Why not use codex or an open source harness?
greenavocado 12 hours ago
How are you able to get lots of usage out of it cost effectively?
handzhiev 11 hours ago
Google One plans are quite a good value actually - for a few bucks you get more Gemini plus space in Drive and other extras. Even through API, $3.75 for nearly Sol-level quality isn't that bad.
And let's not forget you can use it for free in AI Studio, and in the user app (even free accounts get tons of usage, though it's still 3.6 there), and in Antygravity.
greenavocado 10 hours ago
That's the thing. I am completely lost because there are so many redundant paths to get the same thing and I'm trying to figure out which one is the best deal
trvz 10 hours ago
Just put Mythos on the task; it’ll work out the best way in a measly few hours.
handzhiev 7 hours ago
That's Google at its best :)
qlte 3 hours ago
Well are you looking for a subscription or pay-as-you-go API usage?
Subscription? -> Google One plan (http://one.google.com/)
API? -> AI Studio (https://aistudio.google.com/)
Google Cloud -> Gemini Enterprise AI Platform (https://cloud.google.com/ai)
-0_0- 20 minutes ago
This is what killed Gemini for me. The model might well be great, but the ecosystem Google has built around them is a confusing maze of not-quite-there products.
seanmcdirmid 5 hours ago
Ultra AI is like $99/month, and it is hard to exhaust unless you are running a lot of concurrent requests.
rahimnathwani 12 hours ago
One thing in your comment surprised me: "when a thing opens and closes"
Why you would rely on the model's weights to know opening hours, instead of having the model call a web search tool to verify it on the official site?
mlmonkey 12 hours ago
Maybe the model does some tool calling on its own to figure out the times?
rahimnathwani 12 hours ago
You can opt in on a per-request basis.
plaidfuji 12 hours ago
I believe Gemini Flash is smart enough to know when to ground with web search. Their app has been saying it’s running a web search on almost all of my queries since 3.6. And given that Google … is Google, I trust them with web search grounding more than anyone else.
panarky 12 hours ago
It can also ground with Google Maps data in addition to web search.
jampa 12 hours ago
I wasn't trying to be precise originally, I just tried to fit activities into "morning / evening" buckets. I did the whole itinerary with Opus first, but when I gave it to Gemini 3.7 Flash to review, it started correcting it with "this place will close 5PM" or "this place is closed for good".
It was right on every nit, so it was surprising how well the model knows these things. If I ever release this I'll probably need the SERP API or Google Maps SDK (which I've heard is very expensive now), but for a personal trip where I will verify manually, using the LLM is okay for now.
rahimnathwani 12 hours ago
When you called the Gemini API, did you opt in to using search grounding:
tools=[{"type": "google_search"}]
kridsdale1 10 hours ago
Google AI person here:
Using grounding in Gemini is indeed backed by the same canonical data source for business information (like opening hours) as Google Maps. This stuff is available in its own API for a GCP fee, but we’ve built tooling to connect it to the Gemini agentic ecosystem as well.
NiloCK 4 hours ago
Gemini models - at least via some interfaces - have tool calling API access to various Google integrations. flights.google.com, maps.google.com, etc.
The info isn't in the model weights.
Because of where I live, there are three viable airports for any given flight I might want to take, which historically has made shopping a real pain. But Gemini (and only Gemini) has greatly simplified it. Pramble plus date range plus destination and it very quickly generates potential itineraries with costs, total travel time (driving included), etc.
BlackRabbit1 12 hours ago
Can G3.7 use Google Maps for distance grounding?
porridgeraisin 12 hours ago
Yep. It has access to much better route planning tools than the other models. The results are really good IME.
BlackRabbit1 11 hours ago
Sounds good!
I've been planing around with LLM-based trip planning for a very long time now as it fits my very ad hoc style of traveling very well.
But distances always had been.. lets say.. difficult.
Will test it with my upcoming trip to Greece then!
colechristensen 12 hours ago
I started trying out 3.7 Flash this week and it is competitive with opus/fable and also FAST. It is getting work done that anthropic models were struggling with and the speed with which it does is quite a bit noticeably faster.
Beginning to think Google is a dark horse in this race and some of Anthropic's "everything feels janky and rushed" karma is going to catch up.
re-thc 12 hours ago
> Beginning to think Google is a dark horse in this race
Google was so hyped up early Gemini 3 era (only some months ago). And now dark horse? The TPU takeover almost crashed nvidia and everyone else.
colechristensen 12 hours ago
Hype that burned out pretty quickly, it's hard to speak to the size and significance of old hype, I never felt it.
Every time I personally tried Gemini models up until last week they simply couldn't do the long complex tasks I'd being doing with Anthropic models for many months.
cousinbryce 10 hours ago
I use Gemini because I feel like Google will win the AI race, and it’s Good Enough
throwaway219450 9 hours ago
I want to believe this, but every time I try Gemini coding assistance within Colab it's utterly dire. Code gen in a cell is OK, but things fall apart when you try to get into a feedback loop. The system prompt/harness fails to inform the agent about what it can and can't do, or does and doesn't have access to. It will confidently tell you it's done a thing, and then you ask, it admits can't actually do that but will happily try and fail again. Very frustrating, because I really like Colab as a platform for little reproducible experiments that may or may not require CUDA.
spacebanana7 8 hours ago
I suspect Gemini doesn’t focus that much on coding, at least compared to frontier labs. It’s not part of Gemini’s core business case.
Gemini’s integration with maps and search is more important for Google.
dismalaf 12 hours ago
> Real world knowledge
For awhile now I've found Gemini will use Google search for pretty much any real world knowledge, which is a huge plus IMO. It's basically Google with a much better frontend and no ads/seo nonsense.
altmanaltman 12 hours ago
> basically Google with a much better frontend and no ads/seo nonsense
so far
dismalaf 11 hours ago
Fair. But I think there's a good chance it stays that way on paid plans. YouTube Premium is still ad free.
Also them having their own silicon means they don't have to pay the Nvidia tax and can keep costs a lot lower.
kridsdale1 10 hours ago
This thinking is why I am all in on GOOG shares. As a bonus, that means I’m getting part of Anthropic’s gains as well!
fc417fc802 10 hours ago
I think it already has them but it's much more subtle. Also useful. When I've made certain sorts of queries I've had the distinct impression that it was attempting to very gently steer the conversation with suggestions. But it was brief, still answered usefully, and didn't resist going in the direction I wanted. So a win-win tactic I guess.
For example find a beautiful landscape shot of a place that just so happens to be accessible to tourists and ask it something along the lines of identifying the location. IME it will noticably steer the conversation towards relevant commercial offerings and offer (entirely unprompted) to help plan a trip.
Or ask it about a certain category of product with some requirements and it will initially present (relevant) options that look like paid placement to my eye. But if you ask it's happy to go on to turn up lots of alternatives and enumerate tradeoffs.
Assuming I'm correct the subtlety is on par with product placement in movies. Certainly leagues better than the internet advertising we've suffered to date.
rstuart4133 8 hours ago
It definitely steers. For example if it suggests travel plans, the booking links it provides give Alphabet a cut.
As you say it was subtle, along the lines of "oh, if you are planning on going to the place you are researching, here are some helpful links to places you can stay". Subtle, in that it didn't get in the way of main result, so I didn't mind overly. Insidious, as I only noticed because I wondered why it was providing those particular links and looked them up. I can't see how you could ad-block them if I did object.
And worrying, because these unblockable sneaky ads are just a first foray coming from a company that prostitutes its own app store searches, by making the first and most obvious result utterly unrelated to to the search topic. Instead it's who paid them the most to be there. That behaviour is why everyone dumped Alta Vista when an alternative came along. Alternative Android app stores can't come soon enough.
They already skim off 15% of purchases which I'm sure makes their Android operation return a profit that makes other industries drool. Debasing their search to ad a tiny bit extra on top must by driven pure greed. Senseless, as I'm sure it will come back to bite them in the end.
dominotw 12 hours ago
> trip planning app.
this has to be stong suit of ai agents any model
robotmay 12 hours ago
I've swapped over to it in the past two weeks, it's been really good. It does what I ask and doesn't think it knows better than me, which so far has made it the most pleasing experience I've had when slop-coding.
My only wish is it were somewhat cheaper, as it tends to balloon pretty quickly when I'm using it in Opencode. I'm currently trying to offload a lot of work to subagents to stop the context expanding so rapidly. But on the upside, I rarely have to correct it - I've spent far less time arguing with this than with anything else so far.
newtwentysix 12 hours ago
thanks! this is a very helpful one. I am going to try.
CamilleScholtz 9 hours ago
I've been benchmarking[1] models for trip planning and world knowledge specifically (to decide on which model to use with my travel app), and the Gemini models consistently come out on top.
leokennis 9 hours ago
I stopped using Gemini a few months ago because it would often just (partially) reply literal nonsense to me.
Think 2023 style ChatGPT. Something like “to open a document on your Mac click File > Open docurrrar” - like it suddenly forgot it had to produce actual words.
Overall I enjoyed its speed and comprehensiveness. But those occurrences of nonsense just made it feel like a great car that once a month just stops in the middle of the highway.
Shayk 3 hours ago
This sounds great to combine with Wanderlog using an unofficial MCP I made
https://github.com/shaikhspeare/wanderlog-mcp
dassh 3 hours ago
In my experience, Gemini 3.7 is excellent for general non-coding tasks. But for coding, especially backend development, I still find models like Opus 5 and GPT-5.6 more reliable.
forlorn an hour ago
I tried similar travelling tasks but also added transportation and complex transfers (train, bus, walk, next train...). Worked meh and still a difficult thing to do for a llm.
mattlondon 13 hours ago
Currently top at https://deepswe.datacurve.ai - beating Opus 5!
https://artificialanalysis.ai/models/gemini-3-8-flash shows an intelligence score of 59, the same as Opus 5 medium!
Wow - for a flash model this seems to benchmark powerfully. Remains to be seen what it is like to use.
Gecko4072 13 hours ago
Google - we're so back
oceanplexian 13 hours ago
Only 1 point behind the Chinese SOTA from two months ago.
roosterIllusi0n 12 hours ago
I had qwen 3.8 3bit model drop into chinese on long runs. I had to remind it to use english. Its still better than every gemma model I tried. Gemma deleted files on a harddrive to make space when there was over 2TB free. For long runs, gemma is useless.
nolok 10 hours ago
If you care about points sure, but personnaly I care about price, performance, speed and reliability
WarmWash 13 hours ago
The benchmark also doesn't include speed. You almost think something has gone wrong when using it because it returns full responses so incredibly fast.
scrlk 13 hours ago
Not just speed, also reliability. IME, Gemini's speed and quality doesn't degrade badly during weekday working hours compared to OAI, and especially Anthropic.
ford 13 hours ago
I've had Gemini model API use degrade the most out of OAI/Anthropic/Google (often "over capacity" vs true failures)
Not sure on consumer/product use though
scrlk 12 hours ago
That's interesting to hear. I should have added that I use Gemini through Google AI Studio as my general chat model, which probably explains our wildly different experiences.
sotix 6 hours ago
This one uses that as a priority weight: https://winstonrc.github.io/ai-coding-agents-leaderboard/
ttul 13 hours ago
Crushing it on DeepSWE is a very big deal. Excited to give this a try.
pietz 12 hours ago
I know everyone is benchmaxxing but this one feels one step too far. Doesn't DeepSWE have both public and private tasks? I'd love to see the diff here.
It looks more like Google execs losing their mind and pressuring researchers to put DeepSWE directly into the training set.
re-thc 12 hours ago
> DeepSWE is a very big deal
It's clearly been "dealt with" already. When it launched we had interesting gaps and definitely differences. Now every new release is "crushing it".
ttul 10 hours ago
Will look forward to the "feel" of the model in real testing. But I agree that these benchmarks do get "dealt with" rapidly. That's a shame, but I guess it's the times we live in.
satvikpendem 13 hours ago
We'll see about that. I suspect benchmaxxing as all the labs do as I haven't found Gemini models to be nearly as good in agentic engineering compared to Claude or GPT models.
onlyrealcuzzo 13 hours ago
And the benchmarks agreed with you... until now.
So, yes, maybe it's still not - but this would be the only time it would be highly suspicious / obvious benchmaxxing / obviously bad benchmarks.
NitpickLawyer 13 hours ago
If anything, gemini models are the least benchmaxxed out of any lab, IMO.
sunaookami 13 hours ago
>shows an intelligence score of 59, the same as Opus 5!
...on Medium reasoning. Claude Opus 5 (high) is the default in e.g. Claude Code and scores 61. Still very impressive.
onlyrealcuzzo 13 hours ago
The rumor is that 3.9 is an equal improvement in all directions, and that it should be another fast follow on like 3.7 and 3.8 were.
It's almost across the board better than Terra at less than half the price. 3.9 is likely to approach Sol at the 1/10th the price.
Hopefully OpenAI releases Astra first, and it's not only better than Sol but significantly cheaper, too.
harmonic18374 12 hours ago
Curious where did you hear this rumor?
onlyrealcuzzo 11 hours ago
All the talk on Reddit on Gemini 3.8 discussions: https://www.reddit.com/search/?q=gemini+3.9&cId=1650e403-bcf...
nolok 10 hours ago
Accordit to reddit talk, Fable 5.1 is worse than Opus 4.6 and 8B models are smarter than Qwen 3.8 Max, I wouldn't take anything said there with any more reliability than an instagram short.
markasoftware 13 hours ago
On artificial analysis it's only equal to opus 5 medium effort. Opus 5 max scores 63.
Further, opus 5 medium outputs 4x fewer tokens to achieve the same result, negating a lot of the speed difference.
irishcoffee 13 hours ago
A comparison to an artificial score and a comparison to “the same task”
These folks must laugh themselves to sleep. This whole industry hoodwinked the masses. It’s impressive.
wonnage 13 hours ago
It’s all just vibes
theHocineSaad 13 hours ago
As of writing this comment, Claude Opus 5 has an intelligence score of 63, not 59 (it's not the same as Gemini 3.8 Flash).
With a score of 59, Gemini 3.8 Flash is in eighth place, falling behind even Grok 4.6, Kimi k3, and GLM 5.3.
Squarex 12 hours ago
They are all much larger and more expensive models. Google does not have a frontier model right now, but for cheap ones, they are better than event the chinese models now.
pietz 12 hours ago
That's not being debated here. The initial reported numbers were false and this was simply pointed out. You're changing the subject.
mattlondon 12 hours ago
Opus 5 medium has the same score as 3.8 flash on artificial analysis intelligence index.
Are you implying Google or Artificial Analysis are reporting false numbers? What's your source?
asdfologist 12 hours ago
BTW you're comparing 3.8 flash high to opus 5 medium. 3.8 flash medium scores lower.
WarmWash 10 hours ago
Flash models are on the order of 1/10th the size of Opus models, so some flex in the thinking level is fair.
Topfi 8 hours ago
Flash is just a name with no defined or consistent meaning even within labs, let alone between them. Considering both are closed weight, there is no way to truly assess how big the size delta between the two is. Then again, who cares about size, performance and end-to-end speed+cost are what matters along with task adherence, task assessment and so on.
Model size also can not be inferred by tokens/sec for a multitude of reasons, but to showcase two examples, Opus 5 and Sonnet 5, as well as Gemini 3.1 Pro Preview and 3.1 Flash have each very comparable output speeds when using the same deployment as a basis for comparison, despite it being very likely that within their generation, the former are larger than the latter. Feel the need to mention this, as I unfortunately stumble upon so many poorly reasoned, speculative hype post trying to infer model size via utterly unreliable metrics, not based in actual data.
It’s like comments below arguing about the reasoning levels not normalized to some metric (like cost, output token amount or duration) but just the labels or high, max, medium, etc. Those mean almost nothing even when comparing models based on the same pretrain (just compare GPT-5.4 to GPT-5.2), they mean less than nothing comparing different labs releases.
WarmWash 5 hours ago
It's not totally a mystery
https://arxiv.org/html/2604.24827v1
The short of it is by using hard facts knowledge that is difficult to compress, and then quizzing models on these facts and calibrating against a bunch of open models, you can kind of feel out the size of closed models.
nomel 5 hours ago
When comparing closed models, the only thing that actually matters to anyone using them is some mix of cost and speed. Considering how much memory a server is using, when evaluating models that you'll never have access to in order to host yourself, doesn't really make sense.
duplessitous 10 hours ago
> [...] shows an intelligence score of 59, the same as Opus 5 medium!
Nothing here is false, you are simply confused. You either didn't read what they wrote in its entirety or decided to reinterpret what they did write.
knollimar 6 hours ago
"Beating opus" is the false part, no?
porphyra 9 hours ago
Better than even the Chinese models? That's a difficult-to-quantify, extremely rapidly moving target. Just today, Qwen 3.8 Max 0902 came out with a huge improvement over the previous Qwen 3.8 Max.
kamranjon 12 hours ago
They said Opus 5 medium - which does have an intelligence score of 59 (you have to select it manually from the dropdown to see it)
anthonyrstevens 12 hours ago
That 63 score is for Max. The OP specified medium.
bertili 13 hours ago
A fifth of the cost of Opus 5! Google is certainly pushing the completion with this.
abirch 13 hours ago
Gemini hasn't failed me for personal usage yet. I haven't had the opportunity to use it at work.
panarky 12 hours ago
I've been using 3.7 Flash to audit the work of Opus High, and Flash finds lots of subtle and insidious defects even while all the unit tests are green.
Then I tell Opus to read the audit report and implement what it agrees with.
Flash is really good at this, and it is blazing fast in Antigravity CLI. Easily 10x faster than Opus.
Can't wait to try 3.8 Flash. If it's good enough, maybe I'll switch Flash to primary and make Opus the auditor.
porridgeraisin 12 hours ago
Yeah the speed in agy cli is amazing. Whole files get written and "py_compile"d in a single blink of the eye its crazy.
In india, my telco gives me google ai pro for free. And agy with flash goes a long way.
prodigycorp 12 hours ago
it's very fast but it still doesnt come close to 5.6 sol, at least for me, in terms of gathering the context necessary to do extensive changes.
MaxikCZ 12 hours ago
Idk, was building/maintaining simple esp32 control program with antig/opus. After last update it defaulted to gflash3.7. I pasted an email requesting 2 changes into the chat prompt, it did one and took me 4 turns to get that one right.
pkos98 13 hours ago
Wait a week with your judgement - most likely, Google is just bench-maxing very hard.
If you look at the previous Flash models and the announcement on Google I/O, it was an absolute disaster. Reality diverged very much from the marketing (supposedly great benchmarks).
jrflo 12 hours ago
sidenote, but wow sonnet 5 is shockingly bad on this benchmark.
notatoad 12 hours ago
sonnet 5 is bad by almost any metric.
anthropic really needs something to address the cheaper end of the market before they get left behind. Sonnet 5 sucks, and Haiku hasn't been updated in a year. meanwhile we've got gemini flash, luna, and GLM5.3 all delivering 90% of the performance for a small fraction of the cost. paying $25/mTok is going to start looking pretty silly soon.
kimjune01 12 hours ago
deepswe is public and can be considered contaminated.
WhitneyLand 12 hours ago
There are important gaps in that hot take.
For example, it's not even close to Opus 5 on Terminal-bench 4.0, 19.1% vs. 51.8%.
simonw 13 hours ago
Pelicans (thinking effort high, medium, low): https://tools.simonwillison.net/markdown-svg-renderer?url=ht... - high cost 8.9742 cents
Here are the 3.7 pelicans for comparison: https://tools.simonwillison.net/markdown-svg-renderer.html?u... - high cost 8.4387 cents
(I think thinking level low is a regression on 3.8 compared to 3.7.)
world2vec 13 hours ago
I mean no offense but these pelicans are a bit tiresome and a very meaningless benchmark. There's no real difference between any of these svgs across models and model versions anymore.
simonw 13 hours ago
Congratulations, you're this thread's "pelicans are tiresome" comment - it's part of the Hacker News tradition at this point.
(Next up is the comment saying that the labs are clearly training for the benchmark.)
world2vec 13 hours ago
The labs are clearly training for the benchmark.
WarmWash 12 hours ago
This has been addressed endlessly, for a few years now, and is just as much of a trope as "this benchmark is useless".
_puk 10 hours ago
wongarsu 13 hours ago
If everyone agreed with you, the comment would disappear near the bottom of the thread
I like the benchmark. Yes, it's near saturation for SotA models, but still quite good to show where smaller models stand in relation to SotA
In this instance, I see a great image, but consistently clipping mudguards (both in 3.8 flash and 3.7 flash)
IshKebab 7 hours ago
> If everyone agreed with you, the comment would disappear near the bottom of the thread
If only it were true that things that are tiresome are unpopular. But witness "6 7", "first post", ... remember the "in soviet Russia" jokes on Slashdot"? It seems like there are a subset of people that simply don't get tired of tiresome things.
bitexploder 13 hours ago
It is more fun than serious at this point. Don't overthink it :)
anentropic 12 hours ago
it's a tradition
onlyrealcuzzo 13 hours ago
This is in comparison to Fable:
> https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
> Took just under 14 minutes to generate, and at 65927 output tokens cost me a hefty $3.30!
So 50x cheaper - and how much faster?
simonw 13 hours ago
The Gemini models have openly trained for SVG output, apparently with a specialism on animals in forms of transport! https://twitter.com/JeffDean/status/2024525132266688757
dieortin 13 hours ago
I don’t know if you’re joking, but I don’t see anything in the linked tweet which suggests that is the case
simonw 12 hours ago
Watch the video. It's from then-Gemini-lead Jeff Dean and the video shows off an animated pelican riding a bicycle, a frog on a penny-farthing, a giraffe driving a tiny car, an ostrich on roller skates, a turtle kickflipping a skateboard, and a dachshund driving a stretch limousine.
JacobAsmuth 11 hours ago
I'm confused. This doesn't mean they trained on it.
simonw 10 hours ago
I didn't say trained on, I said "trained for SVG output". Gemini team members have publicly stated that they have trained for SVG:
https://twitter.com/sunjiao123sun_/status/202455551655137292...
> I’ve been developing the SVG generation capabilities for Gemini 3.1, and the complexity of the SVGs is stunning.
> This allows UX designers to transcend pixel constraints and directly output structural, production-ready code!
scosman 11 hours ago
Community effort happening here to build the ideal dataset: https://github.com/scosman/pelicans_riding_bicycles
MadameMinty 10 hours ago
Nice. Really high quality SVG pelicans riding bicycles here.
isoprophlex 10 hours ago
Wow nice. If an llm could replicate these excellent examples, I'd consider the pelican benchmark fully saturated.
phatfish 8 hours ago
Come on, don't provide the smoking gun that shows how to draw a pelican riding a bicycle. If it's on the public internet it will end up in training data and invalidate this important LLM capability benchmark.
uif124 11 hours ago
Yesterday's transcript of the best version from Fable looked like Fable already knew exactly what it should do without "thinking". In other words, there were no passages like "on the one hand I could do this, on the other hand ...".
It saw the fish in the basket from some other previous attempt but completely missed the gap between the tires and the rims where the background shines through (now it knows after scraping this comment and watch the next transcript).
lern_too_spel 12 hours ago
The fenders are a nice touch, but putting the fenders through the tires seems like a design flaw.
jpadkins 12 hours ago
The rendering of the gullet is very poor, because its both behind the handlebars but in front of the bike frame (impossible geometry). Surprising because gemini is usually pretty good on geo spatial skills.
Edit: scrolled down to medium effort, its better but also has a weird clipping issue with the fish in the beak.
neuronic 7 hours ago
LLMs are not intelligent and don't actually understand the concept of a bicycle. Parrots also don't understand human language but they're really good at pretending otherwise.
hughw 12 hours ago
It's about to squash a tiny baby pelican
mrdependable 12 hours ago
Why are the SVGs getting more detailed rather than just more correct than previous models?
trentor 11 hours ago
Because people tend to like fidelity more than correctness.
aesthesia 11 hours ago
It bugs me a little that "fidelity" has connotations other than "faithfulness to an original"---fidelity should be basically the same as correctness here!
nonethewiser 10 hours ago
That’s really not the highest fidelity interpretation of “fidelity.”
neuronic 7 hours ago
If correctness would matter anymore, people wouldn't be using LLMs in the first place.
EugeneOZ 8 hours ago
Impressive pelicans!
anigbrowl 6 hours ago
These are becoming unreadable as the reasoning chains expand. I think you should consider reformatting them and either putting the image first or else folding the COT output.
simonw 5 hours ago
Yeah, putting reasoning in a details/summary is a good idea.
simonw 13 hours ago
The most interesting thing about the Gemini models is still their multi-modal support: they accept audio and video input, OpenAI and Anthropic's flagships are still image-only.
Gemini Flash is also pretty cheap, so it's a great family for performing media analysis, like extracting structured data from images and video.
Matsta 11 hours ago
Yeah we use it a lot for analysing streams and clipping content. As well as analysing social content that gets put out.
We transcode everything to 480p before we send it to Gemini batch api. Works great
drusepth 10 hours ago
Interesting side note: although Opus is still image-only, you can still drag videos into Claude Code and it doesn't blink an eye; it just strips it down to a series of images to parse.
True multimodal support would be way better, but I have no issues pasting in full screen recordings while QA'ing games and having Claude identify and fix issues in the video.
ray_kay777 9 hours ago
Agree - I do video editing via Claude Code and it does the job just fine. A lot of my tasks involved frame accurate cutting and to do so it will make a composite image of several consecutive frames in a single image and analyse it that way.
WarmWash 10 hours ago
Lost in the news was their update to gemini video analysis yesterday, dramatically cutting tokens (up to 88%!) needed to analyze videos.
https://blog.google/innovation-and-ai/models-and-research/ge...
brap 11 hours ago
People have been sleeping on Gemini lately but these last few Flash releases (which were very rapid) are damn good.
These sort of fast and cheap models are great for tasks that are verifiable and can be retried infinitely (like coding), you can basically get frontier results with a good harness (at a fraction of the time and money).
mvdtnz 11 hours ago
As someone who has stubbornly stuck with Claude Code, what's a good harness for Gemini models?
brap 11 hours ago
Antigravity has been also rapidly improving lately, and your can also use any of the open coding harnesses. But I mostly meant “harness” as in your workflow/loop setup.
_aavaa_ 10 hours ago
Do they officially support you using your subscription in other harnesses?
watusername 9 hours ago
No, and Google actively bans people for using their subscription from other harnesses via various proxies/gateways.
To preempt certain replies, yes, I know you can pay API prices and use whatever harness you want.
_aavaa_ 9 hours ago
Shame. Thanks for the info.
Imanari 10 hours ago
Is that possible? What subscription would that be? 'Google AI Plus'?
foretop_yardarm 9 hours ago
Is antigravity still heavily rate limited?
drusepth 10 hours ago
Antigravity is probably the best of the bunch I've tried. I'd say it's pretty comparable to Claude Code (I use both daily).
VadimPR 8 hours ago
Antigravity which lacks an auto approve mode? Not really comparable to Claude Code when you're looking to run a team of agents from my experience.
KeplerBoy 8 hours ago
They have an auto approve mode. At least in the vs code plugin.
drusepth 6 hours ago
I've curated my auto-approve list to specific commands by approving them with "always allow in this project" (I never want it e.g. committing/pushing to GitHub, removing files, etc without being in the loop) but Antigravity does have both a standard "auto-approve" mode _and_ a "Turbo mode" which disables ALL approvals of all kinds.
NickHirras 2 hours ago
In settings, enable Turbo Mode, and Auto-Approve, it will run without stopping for approvals.
brap 5 hours ago
Just like Claude Code and others it has the same —-dangerously-skip-permissions flag, auto approves everything
robertn702 9 hours ago
I highly recommend just getting out of Anthropic's (or anyone's) vendor lock-in. Use opencode or pi. You can still use your subscription pricing using a proxy. I switched to opencode and haven't looked back.
pdimitar 8 hours ago
How do you do the proxy thing?
solarkraft 7 hours ago
I use OpenCode with my ChatGPT subscription, which is officially supported on both sides and a damn good deal (OpenCode Go is great too).
OpenCode has “providers” for many (many!) other services, but these are almost all unofficial and against ToS (Anthropic being famous for ban-hammering people).
cromka 7 hours ago
But proxying via Claude CLI is OK? Or also not?
robertn702 5 hours ago
Meridian[0] for Claude Code, CLIProxyAPI[1] for everything else
ryanscio 9 hours ago
Pi [1] is amazing. Since using it I've felt no need to switch harnesses anymore.
Or choose Oh My PI [2] for batteries included
[1] https://github.com/earendil-works/pi [2] https://github.com/can1357/oh-my-pi
dcchambers 8 hours ago
Can you use Pi with a Google Pro AI sub or do you need to use API billing?
mredigonda 8 hours ago
Using API billing would be a bummer because, as far as I know, there's no way to set up a spend cap or pre-pay the API key, correct?
tobias2014 7 hours ago
They finally allowed for a hard spend cap that's easy to set a few weeks ago: https://aistudio.google.com/spend
simlevesque 4 hours ago
You can use any model with Claude Code. Most chinese one have a Anthropic compatible endpoint and for Google and OpenAI's models you can get a compatible endpoint with a proxy like Bifrost. No need to change your harness.
smlx 2 hours ago
Crush is great: https://github.com/charmbracelet/crush
ddxv 30 minutes ago
The vscode chat is great. Not sure if it's also called copilot, but you can plug in any models there and they get sandboxed, tools and link to your code. Great stuff.
allthetime 15 minutes ago
I resisted at first but now I main antigravity for work. All the software (web, react-native app, client cms platform, postgres backed, multiple ETL systems, a few chat/websocket backends) for my company is loaded into a single project. I spend time writing prompts and forming plans back and forth with the agents then I click GO. Over the past year we've gone from it taking minutes and needing a reasonable amount of back and forth and fixing, to it taking 10-20 seconds and outputting near perfect work within my system accross app/service boundaries.
EFLKumo 11 hours ago
Something maybe unfamiliar with you: not about coding but writing. I've asked it to write an argumentative essay, which is a part of "gaokao" (China's university entrance exam), and its work is *extremely* impressive. speaks and writes like a real senior high school student, and the opinions unfold progressively with deep hierarchy. I don't know how the Gemini team reaches this because this kind of Chinese capability literally outperforms at least 2/3 Chinese students, no to mention those who speak Chinese. After all, the model speaks like a real humankind if you prompt it well. That's AGI guys
cubefox 11 hours ago
Nitpick, but in my opinion an LLM is an "it", not a "her" or "he". Using male or female pronouns risks anthropomorphizing them which can lead to unhealthy outcomes.
EFLKumo 11 hours ago
Sorry! I was just a bit excited writing the comment and ignored that :(
cubefox 10 hours ago
No worries.
adleyjulian 11 hours ago
FYI in Chinese he/she/it all use the same pronoun "ta" when spoken.
SchemaLoad 5 hours ago
They do have a separate character for animals and objects 它 vs 他/她, though I can imagine learning english and just learning ta = he.
livinglist 5 hours ago
There’s also one specifically for God - 祂, and one for animal - 牠, the first one you can see frequently in Chinese bibles or churches, the latter is rarely used.
BeetleB 9 hours ago
Well, my LLM is a "he".
> Using male or female pronouns risks anthropomorphizing them which can lead to unhealthy outcomes.
Ditto for pets.
cubefox 8 hours ago
Well, I would say your LLM is a "he" just as much as your laptop is: not at all. If you call the laptop "he" that changes nothing.
For animals he/she does make sense, because they are male or female. An LLM is neither.
fwip 2 hours ago
A goldfish is closer to a person than any LLM.
literallywho an hour ago
What about languages, such as Russian, where every single noun has a gender assigned (he, she or it) and AI is a he by default (and everything else is already using pronouns in similar way, like a car is a she, a ship is a he).
SneakyZero 9 hours ago
Gemini is known for good at creative writing in the Chinese writing community.
It's a bit ironic though. Google has probably the most and best code base among all tech companies but its Gemini is bad at coding. Google has no access to Chinese market but its model is incredibly good at writing in Chinese.
mattlondon 13 hours ago
Wow this comes after what - 3 or 4 weeks since 3.7 Flash, which was also 3 or 4 weeks after 3.6 Flash IIRC?
I eagerly wait more info but sounds like Deepmind without Demis calling the shots has been unleashed and are operating at full speed? Shocker!
At this point it is a meme of course, but where is 3.5 Pro :)
meetpateltech 13 hours ago
According to the WSJ, 3.5 Pro is reportedly being skipped entirely, making Gemini 4 the next flagship model after post-training.
hiddencost 13 hours ago
A month is not enough time for any meaningful change in an organization the size of Deepmind/Google. These models were surely the result of work streams and teams that started under Demis. I think Demis can safely feel proud Deepmind is getting back on track.
aurareturn 3 minutes ago
Reports are that he checked out of day to day work well before his reassignment.
Sometimes it is hard for a scientist by nature to build and iterate and lead revenue generating products.
p_l 9 hours ago
Now i am awaiting Gemini 3.11 "For Workgroups" to be released early December...
neuronic 6 hours ago
With Gemini 95 following soon after.
a11r 13 hours ago
Looks like the strategy of regular updates with incremental improvements is working out well. Interestingly, the biggest jump in Artificial Analysis Intelligence Index score is for reasoning level Medium ( 3.7 was 51, 53, 57 for Low, Medium and High, 3.8 is 52,57, 59 respectively). I think scores at lower reasoning levels are more indicative of model capability since higher reasoning levels are focussed on benchmaxxing. We use the lowest reasoning level in production with good results.
Jcampuzano2 13 hours ago
I'm not an expert but I agree with your statement on the lower reasoning levels.
Lots of models seem to just allow the model to "bloatmax" tokens in order to get bumps at high/max reasoning levels. Many of the max reasoning levels allow models to use up to double or more the tokens the next lowest reasoning level uses. Its basically only useful for people who have no cost or time stipulations on anything.
I think I actually preferred it when we had models that either had reasoning enabled or didn't.
abixb 10 hours ago
I like Google's strategy here. These new Flash models of late (Flash 3.6, 3.7 and now 3.8) have obviously been distilled from a much larger unreleased model (Gemini 3.5 Pro, iirc from the rumors).
One aspect of model releases that don't get discussed as much are the cache invalidation (changes in underlying architecture, weights, or tokenizers); I assess Google seems to be squeezing the maximum out of the last 'Pro' version they released with 3.1 back in February.
Small models cataching up with their bigger siblings are fantastic news.
alephnerd 10 hours ago
A couple larger GCP customers requested this for sometime, especially on the cybersecurity side.
A SOC/IR or AppSec team doesn't need a generalized model that knows when Chaucer lived but it absolutely needs a model that can efficiently, quickly, and accurately prioritize vulnerability severity or validate patches.
kamranjon 12 hours ago
They've interestingly left out any mention of speed.
I have been testing 3.7 flash against 3.5 flash and it seems to lose every time in overall latency. Every benchmark I've seen seems to suggest the opposite[1] - that 3.7 flash is significantly (at times 2x) faster than 3.5 flash - but I have never been able to prove this out in real world use cases.
Has anyone found their latency numbers to actually be accurate? Is this why they've toned it down in this release? For context, I'm testing larger generation payloads that take 8-10 seconds in 3.5 flash and 15-25 seconds in 3.7 flash. Lowest reasoning settings in both cases.
1: https://artificialanalysis.ai/?speed=intelligence-vs-speed&m...
film42 12 hours ago
It depends on how you're querying Gemini models. OpenRouter is the fastest by far. I'm guessing they bought the dedicated pipe from Google. Gemini via VertexAI and consumer API has pretty bad latency.
kamranjon 12 hours ago
Yea I am testing through OpenRouter - have you noticed 3.7 flash being significantly faster?
film42 10 hours ago
I guess it might be relative, but switching from VertexAI endpoint to OpenRouter was like 2-3x faster for us.
pampas 5 hours ago
In my niche Redactle puzzle solving benchmark [1] I noticed Gemini 3.8 flash is slightly faster than 3.7 flash. They both smoke every model I've tested. I have not yet run 3.5 flash.
Gemini models are great at this task because they seem to have exact Wikipedia text baked into the weights. When I rewrite the wiki text a bit it's not able to one-shot the game so much.
andai 13 hours ago
Wait, I didn't realize 3.7 Flash was already beating Sol on a bunch of the benchmarks. Isn't it a way smaller models?
ipsod 13 hours ago
IDK if it's smaller, but I know it's way faster. In one test I did, Flash 3.7 high was ~9.4x faster than Luna High.
But, also... Sol crushes Flash 3.7 at writing code in a codebase of any size beyond "tiny".
Flash is my go-to for prototyping, and basically anything that isn't writing production code.
ramon156 13 hours ago
The only company with a proper TPU set-up is bound to have the fast models, now add a market cap like Google to the mix.
ipsod 13 hours ago
They've been my bet to win the AI race for a while. I was starting to doubt, but this 3.6, 3.7, and 3.8 arc has anchored me.
Diederich 11 hours ago
Totally agree. When this current wave of GenAI really started heating up, I guess 2020-2021, my analysis was very straightforward. What are the high level inputs to long-term success? I basically came up with a couple of criteria:
1. Data. Lots of data.
2. Money. Lots of money.
3. Access to necessary hardware.
4. Business alignment/will to do it.
5. Access to talent, current and future.
This is certainly incomplete/naive. In my mind, though, Google was the clear answer.
On a more personal level, I've been deep into the Google ecosystem since I got [email protected] in 2005. (I actually paid 50 cents on ebay to get a very early invite.) There was no question in my mind that Google's AI work would deeply integrate into their whole ecosystem in very powerful and productive ways. (Yes, I can join you to discuss, at length, the various ways that Google's dominance is problematic/scary.)
Having said all that, I'm quite happy that there is, at the moment, a very rich competitive landscape. Indeed, not too long ago, with Gemini Pro 3.1 languishing, I moved most of my deeper thinking work to ChatGPT, which was, for me at least, clearly outperforming Gemini.
While I certainly didn't anticipate it, Google's strategy of making their fast/relatively inexpensive models surprisingly powerful has been a welcomed surprise.
ipsod 10 hours ago
Not only do they have access to the hardware - they've been developing it in-house for years.
esafak 13 hours ago
Luna is way slow. I don't remember an OpenAI model ever being this slow.
edit: I have a subscription; direct call.
dannyw 13 hours ago
Are you using direct or via OpenRouter? I think OpenRouter Luna always uses the `flex` tier, which is quite a bit slower.
momojo 11 hours ago
Same. Love oneshotting or sanity checks. Which fortunately is a lot of my workflow (lot of long tail stuff fits in one prompt).
MrBuddyCasino 10 hours ago
Its not good at not making mistakes, but what it produces is structurally quite nice, not over-engineered (looking at you Sol) and its personality isn’t annoying (looking at you Claude). A bit like Grok Code, but Grok is a better coder.
realist_not 13 hours ago
It's pretty good if you can actively steer it , its actually really really good , the antigravity free tier and pro tiers are generous as well . I'm shocked at how fast it generates tokens.
worldsavior 13 hours ago
Some would say it's Google's TPUs.
MrBuddyCasino 10 hours ago
Can the free tier be used outside Antigravity CLI? Because its security prompts get old pretty quick.
refulgentis 13 hours ago
They're quite selective in benchmarks, c.f. notably only bad one is 10% on TerminalBench. It's a really addled model, one time I said "Hi" and it built out a 4 panel hello world app with (fake) weather, a todo list, and a couple other things I forgot. I wouldn't be comfortable saying "ignore the #s!" except when I complained it was trash and way overcooked on agentic coding yet not good at it, and a couple DeepMind ML people liked the tweet.
zuzululu 8 hours ago
i discount people who lean too heavily into benchmark as the authoritative truth when it comes to evaluation of coding capability of these models.
experience tells me that those people simply have not used models for a long period of time specifically on coding and have run their own comparisons
to someone who uses all vendors, the differences are very palpable and drives purchase decisions.
also keep in mind Gemini and other labs have repeatedly done benchmaxxing, you must have your own benchmarks to evaluate these models.
pampas 6 hours ago
Gemini 3.7 Flash was already smashing more expensive models on my Redactle benchmark https://redactle.net/llm-leaderboard which mostly tests omniscience.
j-bu 12 hours ago
"The knowledge cutoff date for Gemini 3.8 Flash is March 2026 – users can expect updated information for some domains while in others they may experience the model’s knowledge is limited to January 2025 (in line with the Gemini 3 Model Family)."
Kind of wild that they haven't (successfully) pretrained a base model since Jan-25.
venusenvy47 12 hours ago
I'm curious if the knowledge cutoff is important, when the interface (Gemini app) can search online for recent information. Is there a big advantage to having everything internal?
j-bu 12 hours ago
Not directly - but latest research advancements, cleaner / richer datasets, etc. still require fresh base models. Not everything can be fixed through post training alone (e.g. why GPT-5.5 "Spud" was such a big jump, and also why GPT-6 "Astra" is now supposedly another big leap). Ofc model size etc also plays a role, but my (admittedly limited) understanding is that new base models _can_ also lead to big jumps even keeping parameter counts constant.
npn 10 hours ago
very important actually. just try to generate code for fresher frameworks/libraries. gemini sucks so bad in real work usage, everything it suggests are outdated and mostly useless.
rjh29 9 hours ago
Search grounding is expensive, you can't force the model to do it either. I use Gemini a lot and it often replies with out-dated data. The more detailed the information you're asking, the more likely it is to be wrong.
StevenWaterman 8 hours ago
You don't need everything internal, but having some idea of recent events is useful. If you ask it to implement some local AI there's a decent chance it will try to use qwen 2.5 without wondering if anything better came out since
make3 an hour ago
That extremely likely just means that they're preparing an omega huge Gemini 4 Pro release and that that's what training right now on most of the compute
raincole 12 hours ago
I don't know if Google is having the worst marketing fumble or the most genius marketing one. Their "flash" models are very comparable to other companies' "pro" or "flagship" models. It seems to be a quite counterintuitive naming convention as it undersells the models.
Unless they have an even more powerful Gemini Pro in the oven...?
drowntoge 12 hours ago
Well if that's the case, it's been in the oven for quite a while now.
owaiswiz 11 hours ago
not saying they do have a beefier pro, but even if they did, isn't the delta between flash vs pro models reduced quite a bit? (e.g glm 5.3 flash vs 5.3, v4 flash vs v4 pro, sonnet 5 vs opus 5)?
anthonypasq 11 hours ago
the 3.5 pro pretrain was a complete disaster, they shelved it and are now working on gemini 4.
3.0 flash -> 3.8 flash is all post training which is pretty impressive.
chrsw 7 hours ago
Do labs come back from disasters like GDM’s 3.5 pretrain? I am thinking of Meta’s Llama 4. Meta is just now starting to be taken seriously again but they are definitely not at the frontier. And when I say “come back” I mean have an Opus 4.5 moment, which was really mind blowing for me at the time. Fable was a similar leap, just not as big.
deaux 2 hours ago
OpenAI had such a disaster themselves before, GPT-4, so they replaced it with 4o.
rahidz 11 hours ago
The conspiracy theorist in me wonders if it's about keeping the federal government out of their business after seeing what happened to Sol & Mythos.
make3 an hour ago
My assumption is that they're cooking an ultra humongous Gemini 4 Pro release. They certainly have the cash and the compute for it, and it's so obviously the thing to do from a strategic perspective.
xnx 13 hours ago
Seem like a great, no-compromise, upgrade over 3.7 which is already a bargain, fast, and doesn't have the brain-damaged writing style of Claude.
fitsumbelay 13 hours ago
that's certainly what it's looking like so far. kind of mind boggling ...
tagalog an hour ago
Gemini flash seems to have been a bit of a sleeper. Somehow it's ended up as the most used LLM for my client document extraction work these past few months.
I have an eval harness that runs every Thursday to determine which models are the current best for a few different client workflows. And since May(?) flash has slowly been taking over more and more stuff to the point it is now 100% on 8 out of 11 document extraction flows with the other 3 being a Flash / Opus 4.8 mix for high value stuff where cost is less of a factor.
lysecret 10 hours ago
Also just want to let my appreciation here for 3.7 it’s cheap super fast super reliable incredible at information parsing eu host able (important for us) and perfectly integrated into gcp. Great job google!
MrBuddyCasino 10 hours ago
I hope they bring a lite version, its good enough for information parsing and very cheap.
weird-eye-issue an hour ago
Use Luna for that
throw10920 2 hours ago
We've gotten an unusually fast speed of Gemini Flash releases over the past few months. Is this Recursive Self Improvement, or Google just trying to distract from the fact that it's been a while since the last Gemini Pro release?
tkgally 2 hours ago
The blog post says it is RSI: “both of today's releases are … accelerated by long-running agentic loops designed to recursively evaluate and refine the underlying models.”
throw10920 2 hours ago
Yeah, but Google is incentivized to claim that regardless of truth value. Critical analysis is necessary.
meh2frdf 13 hours ago
The flash models, for coding are reckless in my experience. I have a Ultimate subscription, get good quota, but still use Opus 4.6 as it's much more reliable if you manage the context window carefully.
onlyrealcuzzo 13 hours ago
> The flash models, for coding are reckless in my experience.
My experience is that antigravity is awful and reckless - but that the model itself isn't.
upcoming-sesame 13 hours ago
If by reckless you mean commit, push, deploy without me asking it to, the I agree!
okdood64 13 hours ago
Respectfully: If it's able to deploy without you asking it to, that's a you problem. There are no safeguards?
meh2frdf 13 hours ago
You need more safeguards for sure, but also it tends to fly off down rabbit holes, rebuilding things in dumb ways, hacking around things, making assumptions etc, it seems very eager to go 'ta da! I did it look how quick I was', sometimes it nails it other times it created a lot of tech debt.
meh2frdf 13 hours ago
Also if it ever says, "I've found the root cause of ..", it definitely has not found the root cause and is making a non evidence based guess as it has run out of ideas.
iAMkenough 13 hours ago
I told it “don’t betray me” in my prompt and it still stabbed me in the back.
sejje 10 hours ago
I'm terrified to seed the RNG with words like "betray". I'll keep those way, way down the list of likely words.
wongarsu 13 hours ago
That's exactly how you get 'you are right, I deleted the production DB to apply the new schema when I should have written a migration'
That said, I do trust Opus and Fable enough to let them deploy to staging. Great for debugging. Just don't give them keys for prod
upcoming-sesame 11 hours ago
100% my problem, but it's the only model I tried that does that so recklessly
tiborsaas 13 hours ago
It even took my girlfriend on a date, now it prepares for IPO, how do I turn it off?
Ridius 12 hours ago
Just hand over your clothes, your boots and your motorcycle and it'll be on it's way
kyrra 12 hours ago
Agents.md is a thing, you can ask it to not do that (it follows that ask pretty well).
datlife 13 hours ago
I use Flash model as code implementation executor, then have GPT-5.6-Sol or Opus to review the work. Pretty good so far and presumably less expensive.
jerkstate 12 hours ago
3.7 flash was by far the best model for image recognition tasks according to my benchmarks. 3.8 flash didn't regress any candidates and improved some specificity (positive ID of common name vs species name of exotic fruit, correct identification of cast/replica of artifact and statue) but is still relatively weaker (26/30) on esoteric public figures (Korean beatboxers). I'm going to have to make my benchmark harder.
arctic-true 12 hours ago
I’m very curious about your esoteric public figures benchmark, do you ask it in English or Korean to identify the person? Does it change the result? I wonder if having data labeled in only a given language (or web sources in only a given language) change the output.
jerkstate 5 hours ago
I haven't tried asking it in Hangul but these particular artists (and the photos I'm using actually) are linked to their romanized english names on e.g. Fandom so it's not unfindable on the internet
throwa356262 12 hours ago
"available to trusted defenders through our new Fairwind Program"
uif124 11 hours ago
Agreed. The only legitimate use case is restricted to a secret guild. Imagine:
"Valgrind is only available to trusted defenders in our new UnfairAdvantage program"
JacobAsmuth 11 hours ago
You're telling me for only 5x the cost and 1/10th the speed I can use a Chinese model which performs worse than Gemini 3.8 Cyber? And I get to do all the hosting and setup work myself instead of just using a model and framework which is already integrated with GCP? Dang!
129867 10 hours ago
I'm sorry, is this a bot that is optimized for sealioning? The point is that you don't have access to Cyber.
pampas 9 hours ago
Gemini 3.8 Flash is top of the Redactle LLM benchmark but so was Gemini 3.7 Flash. Both one shot all puzzles in the evals though 3.8 is just a bit faster. It also does the evals cheaper and faster than almost all the other models I've tried.
maxnevermind 41 minutes ago
Just tried Gemini 3.8 Flash on these 2 consecutive prompts at gemini.google.com:
1 what is tesla cybercab plan to address legal implications of accident that will happen? who is going to be responsible for them when they happen? are they covered by tesla insurance or some other insurance? are there any official plan/statements around that?
2 what was the name of the experiment they started in san antonio tx when some cars didn't have a driver? what was the results of it? did they expand the operations? it was much smaller than waymo, is it growing? how it is related to robotaxi?
It is not able to connect the dots that I keep asking about Tesla in 2nd prompt and spit out some unrelated stuff. Really? How it can be that bad? Gemini 3.1 Pro model works fine in this case btw. I thought maybe it is about knowledge cut over date and it doesn't know about those events from 2025 but it seems it has the knowledge up to March 2025. Top 10 in Intelligence on artificialanalysis ladies and gentlemen.
adbachman 12 hours ago
Still zero on the felony bench.
Is this weakness in their training regimen the impact of operating under regulatory frameworks for too long?
leopoldj 13 hours ago