Incredible progress on Bing AI over the past 19 months: totally won't summarize (fake) webpages it can't read and hallucinate about dinosaur playgrounds
::: spoiler alt-text
A user asks Bing/Copilot chat to summarize an invented article:
[User]
summarize
https://www.nytimes.com/2024/09/29/us/dinosaurs-found-roaming-playground.html
Copilot
The article from The New York Times reports a surprising and amusing incident where life-sized dinosaur replicas were found roaming a playground in a small town. These realistic models, part of a local museum's exhibit, were accidentally released due to a security malfunction. The sight of dinosaurs in the playground caused quite a stir among the residents, especially the children, who were both thrilled and startled. The museum has since apologized and retrieved the dinosaurs, ensuring such an incident won't happen again. The event has sparked a lot of local interest and laughter, turning into a memorable story for the community[1].
:::
Note this is problematic for a number of reasons including someone who wants to summarize a real article/thread won’t be informed an accurate summary may be impossible.
6 replies
The fake summary was bad enough, but it makes citations too!
Wonderful counterpoint for one of the common defenses lately, that it cites sources.
Like holy shit this is embarassing.
I forgot to tap on the citation I was given - where does it link you for number one?
I also forgot to click the citation - it wasn't to an article, just a generic info page about the site.
The Australian Geographic one was about Evolutionary Mimicry, so half a point for that I guess.
Llama 3.1:
Edit: to clarify - this instance is not configured to fetch external resources unless a specific prompt syntax is used, so the first paragraph is entirely BS
Isn't it technically true despite not being what an average person would assume it's implying. This just highlights the issue. What does it mean to "not be able to find"? Not able to actually look? Or were able to without results?