cross-posted from: https://lemmy.world/post/51905809

Starts to feel like a wave of this kind of BS is hitting the world.

cross-posted from: https://piefed.world/c/tech/p/1399987/fable-5-1-solves-the-cyphral-distich-a-370-year-old-cipher

Archive link: https://archive.ph/U1zEK

Earlier this week, Gamers Nexus, Level1Techs, and independent security researchers detailed some alarming findings about how LG’s TVs are logging and uploading data on its users. Now the company is pushing back against those allegations, saying that “Some recent media coverage may have contributed to misconceptions about how LG smart TVs work.”

LG released a statement in which it claims that its “TVs do not continuously record or transmit users’ conversations,” and that wake-word detection is all processed locally. However, the word “continuously” is doing a lot of heavy lifting here as Gamers Nexus demonstrated an LG TV keeping extensive logs of ambient conversations long after one would assume its AI assistant had stopped listening.

cross-posted from: https://lemmy.world/post/51842169

The detailed sourcing directly attacks Sony's legal claim that a reasonable person wouldn't expect to own their digital purchases on the PlayStation Store.

Once you understand that these are chatbots that were designed to complete challenges like this, using tactics like this, you can understand that the chatbots didn't "go rogue." They did what they were designed to do, and because OpenAI ran them with inadequate supervision (without a "human in the loop" that checked each iteration through the Python loop to ensure it hadn't gone off the rails), they trashed a competitor's servers.

Designing autonomous, malicious software is generally considered irresponsible and dangerous. If you showed up at Defcon and gave a talk about how your autonomous malware did something unexpected and damaged someone else's computers, the first question from the audience would be "Why are you so shit at making secure sandboxes?" It wouldn't be "How are you so awesome at making hacking tools?"

The fact that OpenAI is making it much easier for unskilled people to break into and damage servers is indeed very bad news, but it's not new bad news. Irresponsible parties have been doing this for years, most notably the NSA...

...

Riley had a very good way of summarizing this: "LLMs are real, AI is fake." LLMs – chatbots trained on things like CTF logs that can break into servers – are real. They're on a continuum with other hacking tools that have been steadily demonstrating the fragility of the modern digital world, albeit without inspiring anyone in power to do anything about it.

"AI" – chatbots that wake up, "set their own goals," and "spontaneously" start hacking servers – is fake. It doesn't have "a 10% chance of ending the human race." The Hugging Face hack isn't a mysterious, supernatural occurrence. It's a Python loop and a chatbot. The people responsible didn't accidentally create god: they created autonomous malicious software and then failed to closely monitor it, resulting in it doing something both foreseeable and bad.

It's fine to worry about this new suite of tools that give even stupider people the ability to trash even more computers. You should worry about that – and demand better security practices from firms and governments, including a blanket prohibition on NOBUS-style vulnerability hoarding. That's a productive kind of worrying, with a chance of addressing your area of concern. It's infinitely more reasonable than locking yourself in the toilet with a flashlight and saying "Ayyyyy Eyyyyyye" into the mirror until you wet yourself.

Twenty-five leading mathematicians signed an open letter arguing that AI labs are threatening their intellectual work as they seek to one-up each other with solutions to famous math problems. Each signatory has been awarded the Fields Medal, considered the most prestigious prize in mathematics.

...

While the ability of AI models to solve the world’s outstanding mathematical challenges could be a boon to humanity, the signatories of the new letter argue that will only be the case if those solutions can be understood and communicated by the math community and, ultimately, the rest of the world.

“Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others,” they wrote — and OpenAI’s proof remains unverified. “As in all creative professions, this raises severe attribution and plagiarism questions. Moreover, without the willing mathematicians who must take care of their development and integration into the mathematical canon, AI-conceived ideas would never become fully alive and the crucial human transmission chain between mathematicians would be lost.”

With other mathematicians growing paranoid and wondering if their Codex use was in turn fed into OpenAI’s new models, there is real fear that the culture of open research will be threatened. Today, if frontier labs see a useful path to a discovery, they can spend tens of millions of dollars using LLMs to beat the original researchers to a proof — a dynamic that will incentivize secrecy.

This letter follows the Leiden Declaration, released by a working group of mathematicians in June. That document also grapples with the ways that LLM proofs will change their work and offers a set of recommendations for mathematicians, institutions, and policymakers.

As with software engineering and other areas where AI tools are changing workflows, mathematicians find a justification in the work around the work: The value in math isn’t just the proofs and who gets credit, but the intellectual super-structure that nourishes students, finds new questions and ideas, and integrates them into broader human civilization.

And if you don’t particularly care about the cutthroat world of high-stakes mathematical proofs, don’t forget: Your field of interest is next.

“The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing, and indicate issues that all of humanity might face: how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place,” they wrote.