I just posted a follow-up about how Google's new "AI mode" is also impressive (notably different from "AI overviews" which are terrible): https://simonwillison.net/2025/Sep/7/ai-mode/
I tried "Is the pelican in her pity unique to English churches or does it show up in France too?" (Easy) followed by "Find places I can see this near nontron in France" (which I anticipated to be extremely hard): https://chatgpt.com/share/68bd9ca1-f6d0-8006-a507-c817857957...
It gave me a result that looked convincing! But I don't trust it. I very much doubt the internet has reliable, comprehensive data about instances of that piece of religious imagery in small churches around this region of France.
Reposting my comment from simonw's original item that somehow didn't catch on like this item did.
Pretty wild! I wonder how much high school teachers and college professors are struggling with the inevitable usage though?
"Do deep internet research and thinking to present as much evidence in favor of the idea that JRR Tolkein's Lord of the Rings trilogy was inspired by Mervyn Peake's Gormenghast series."
I know it's well intentioned but please don't copy/paste comments across threads - it makes merging threads a pain. If you wanted, you could give us a heads-up about the split thread at hn@ycombinator.com - that way we can hopefully get around to merging them sooner.
Noted, and sorry about that! Since that post/item hadn't caught on I thought it was left in the dust. But makes sense to merge them, didn't think about that happening.
This is a link to my duplicated newsletter, but the better URL would be the post itself: https://simonwillison.net/2025/Sep/6/research-goblin/
I just posted a follow-up about how Google's new "AI mode" is also impressive (notably different from "AI overviews" which are terrible): https://simonwillison.net/2025/Sep/7/ai-mode/
We've merged the comments* to https://news.ycombinator.com/item?id=45152284 and re-upped that one.
(* except the ones that only make sense here)
You should have posted the instructive failures you mention, not just the successes.
This one was meant to be illustrative of a (thought provoking) failure https://simonwillison.net/2025/Sep/6/research-goblin/#histor...
I wondered what "something I’m certain it won’t be able to handle" would look like.
I tried "Is the pelican in her pity unique to English churches or does it show up in France too?" (Easy) followed by "Find places I can see this near nontron in France" (which I anticipated to be extremely hard): https://chatgpt.com/share/68bd9ca1-f6d0-8006-a507-c817857957...
It gave me a result that looked convincing! But I don't trust it. I very much doubt the internet has reliable, comprehensive data about instances of that piece of religious imagery in small churches around this region of France.
Reposting my comment from simonw's original item that somehow didn't catch on like this item did.
Pretty wild! I wonder how much high school teachers and college professors are struggling with the inevitable usage though?
"Do deep internet research and thinking to present as much evidence in favor of the idea that JRR Tolkein's Lord of the Rings trilogy was inspired by Mervyn Peake's Gormenghast series."
https://chatgpt.com/share/68bcd796-bf8c-800c-ad7a-51387b1e53...
I know it's well intentioned but please don't copy/paste comments across threads - it makes merging threads a pain. If you wanted, you could give us a heads-up about the split thread at hn@ycombinator.com - that way we can hopefully get around to merging them sooner.
Noted, and sorry about that! Since that post/item hadn't caught on I thought it was left in the dust. But makes sense to merge them, didn't think about that happening.
To me, there's nothing shocking in here. I do think that search has gotten slightly better in GPT-5, but these examples all feel like softballs to me.
Also as a note to the author - a lot of links are missing spaces after them.
Rats, that looks like a bug in my copy-and-paste to Substack workflow.