Paris-based Mistral AI launched Mistral Large 4 on Tuesday, an AI model with 1 trillion parameters. That is the kind of system behind chatbots like ChatGPT and Claude, as parameters are the adjustable numbers it tunes during training—more of them generally means more capacity to learn, and more expensive to run.
“[Mistral Large 4] is at the frontier of open models, and by far the strongest open-weight model from the US or Europe,” Mistral AI’s Chief scientist Guillaume Lample, wrote on X.
Not all of those parameters work at once. Large 4 uses a "mixture of experts" design, meaning the model routes each question to a few specialist sub-networks, so only 49 billion parameters fire per answer. Mistral's last flagship, Large 3, did the same with 41 billion active out of 675 billion.
The whole “Le Chonk” nickname has a backstory. In June, after Mistral renamed its Le Chat assistant to Vibe, fans on Reddit and X invented a fictional model called Le Chaton Fat—roughly "the fat kitten"—with 30 trillion parameters, 1,000 meows per second, and fake benchmarks claiming it beat Claude Fable 5.
CEO Arthur Mensch played along, replying that the model was really called "le gros chaton," French for "the big kitten." Mistral then added a cartoon cat to its Vibe website. Its real launch post now lists the model as, "very officially," le Chonk.
Mistral sells "sovereign AI"—models a country or company can own and run without handing its data to outside firms. Saudi Arabia's state-backed HUMAIN signed a deal worth hundreds of millions of euros in August with this company for this exact reason.
Mistral raised a €3 billion ($3.37 billion) Series D—a funding round that sells shares to investors—in September at a valuation above €21 billion ($23.6 billion), led by Samsung. Mistral says Large 4 is the first milestone on the roadmap that money funds.
Le Chunk costs $1.36 per million input tokens and $4.18 per million output tokens—the chunks of text, roughly three-quarters of a word each, that AI companies bill by. Claude Opus 5.5 charges $4 and $20, while GPT-6 Astra charges $10 and $50.
That puts Large 4 at about a third of Opus 5.5's price on input and a fifth on output. Against Astra, it is roughly a seventh and a twelfth.
Mistral's announcement mostly benchmarks Large 4 against Chinese open-weight models—DeepSeek V4 Pro, Kimi K3, GLM-5.3, and Qwen3.8 Max—and "open-weight" means anyone can download and run the model. Claude and GPT show up in only a handful of comparisons, and the newest Claude, Opus 5.5, appears only in a cybersecurity claim.
In a blind human evaluation of coding quality by Surge AI, Mistral Large 4 ranked second of five models with 3.74 out of 5, behind Claude Opus 5's 4.22.
AutomationBench hands an AI 657 chores in simulated business software—finance, HR, sales, support—and scores the share of each task's goals it completes, with zero credit if it breaks a rule. Large 4 scored 59.9 points. On Artificial Analysis's board, Claude Sonnet 5.5 hit 71.8, Opus 5.5 hit 69.5, and Gemini 4 Argon topped the page at 77.5 on that same benchmark.
On DeepSWE 1.1, which is one of the benchmarks developers check out when trying to assess how good a model is at coding, Large 4 scored 62. That tops GLM-5.3 at 61 and DeepSeek V4 Pro at 57 but trails Kimi K3's 68. Datacurve's own leaderboard has GPT-6 Astra and Claude Opus 5 at 74.
Mistral says it will release Large 4's weights—the trained numbers that make the model work—by the end of October, which would let outside developers download the model and test those claims themselves.
A new lawsuit claims Anthropic, OpenAI, SpaceXAI and Google made an illegal deal to slow the pace of their respective AI development.
The lawsuit, which was filed Friday in the U.S. District Court for the Northern District of California, argues that the leading AI companies violated antitrust laws when they agreed to coordinate slowdown efforts, and that doing so would reduce the value consumers get for paid AI subscriptions.
The coordination largely took place on Sept. 12, the lawsuit argues, when Anthropic CEO Dario Amodei published an essay urging for industrywide cooperation on decelerating advancements in favor of enhanced safety measures. That same day, OpenAI CEO Sam Altman, SpaceXAI CEO Elon Musk and Google DeepMind’s co-founder and chair Demis Hassabis each publicly responded to Amodei’s proposals in agreement.
But the lawsuit also alleges that the coordination began to take shape months earlier. It points to a statement from July 2026 that high-ranking employees from several of the leading AI labs signed that acknowledged the “intense competitive pressure not to unilaterally slow” development. That statement called on the government to support a global effort to slow automated AI development.
It is clear that an agreement among the chief rivals in AI that their progress “should be slower than competition would otherwise produce has an anticompetitive effect on consumers,” the plaintiffs argue.
Lawyers representing four named plaintiffs, who pay for subscriptions to ChatGPT, Claude, Grok or Gemini, are bringing the lawsuit on behalf of a proposed nationwide class of other paid subscribers to those services.
The plaintiffs don’t object to the companies individually deciding to slow their own progress in favor of safety. They instead argue in the complaint that antitrust laws forbid them from taking the “shortcut” of agreeing to “substitute collective restraint for individual accountability.” A competitive market allows for responsibility and genuine progress, the lawsuit argues.
“AI will quickly spin out of human control and could kill us all if we allow AI safety and protocol... to be controlled by private self-serving agreements between the world’s most powerful ‘for profit’ technology companies,” said Nick Rowley, the lead attorney for the plaintiffs.
Representatives for Anthropic, OpenAI, Google and SpaceXAI did not immediately respond to a request for comment Saturday.
In his initial essay proposing the slowdown, Amodei acknowledged potential antitrust challenges, writing that it would be helpful for the U.S. government to mediate “or at least enable” these cross-lab discussions. The government wouldn’t need to participate, he wrote, but it would need to “issue a narrow waiver for certain kinds of safety conversations.”
In response, Altman said on social media that OpenAI welcomes the idea of a “federal framework that sets consistent safety requirements,” but said “we do not believe we need to wait for an antitrust exemption or legislation to begin the work of providing this confidence.”
While the recent conversations about pacing development were spurred by increasing concerns about AI evading human control, several leaders in the AI space have long talked about developing a shared set of standards or otherwise coordinating to ensure safety efforts remain paramount.
The plaintiffs in the lawsuit contend they are not against the AI companies asking Congress, the White House or any other agency to develop AI regulation, nor are they against the companies asking for an antitrust exemption.
But achieving that kind of collaboration with the federal government could be an uphill battle.
President Donald Trump rejected the calls for regulation on social media. He has claimed that any efforts to limit the technology are part of a “conspiracy.” He called into question why the industry leaders would be calling for regulation that, he says, “if strongly implemented, will drive them into oblivion and bankruptcy.” Trump said Saturday on social media that he is forming an AI task force and will appoint an “AI czar,” but he provided scant detail.
The Trump administration has been vocal about wanting American AI labs to outpace and excel against Chinese competition. While several Democratic leaders and candidates have called for sweeping action on AI regulation, Republicans have largely mirrored Trump’s position.
Sen. Josh Hawley, a Republican from Missouri, said in a recent Senate hearing that “there is no world” in which he would agree to give “the most powerful companies in the history of the world” an exemption from antitrust laws to collaborate, arguing that they could collude together and stifle competition.
a post to donate types of prompts that could be used as a benchmark.
criteria guideline:
* soft push for actual personally useful problems. not toy demonstrations/tests .
* no criticizing/debating . this is write-only
* soft focus for smaller models ( rationale being: no really hard problems that are cutting-edge math,science,engineering ) . eg ok to be using large models but for not cutting-edge problems that a smaller model definitely could never handle.
* one post per user
* vague description of problem/prompts is ok.
GPT‑6 Astra is rolling out today to a limited set of organizations and over the coming days will become available to all ChatGPT Plus, Pro, Business, and Enterprise users, as well as through the OpenAI API, Microsoft Azure, and AWS Bedrock.
We’re introducing GPT‑6 Astra, the world’s most intelligent and aligned model.
GPT‑6 Astra brings together years of research and big bets across pre-training, reinforcement learning, and alignment. Astra is state-of-the-art on computer use, browsing, software engineering, cybersecurity, science, and professional work. Astra saturates FrontierMath Tier 4 with a 98% score, having already helped solve long-standing open problems in mathematics. Astra also saturates ARC-AGI-3 with a 99.9% score and ExploitBench with a 100% score. It also sets a new frontier on computer and browser use, handling the most demanding professional work with unmatched speed, accuracy, and judgment.
GPT‑6 Astra is rolling out today to a limited set of organizations and over the coming days will become available to all ChatGPT Plus, Pro, Business, and Enterprise users, as well as through the OpenAI API, Microsoft Azure, and AWS Bedrock.
Terminal-Bench Science 0.1 tests whether agents can complete scientific research workflows using code and terminal tools, including analyzing data, running simulations, and fitting models. GPT‑6 Astra reaches a new high among the models compared at 64.6%, versus 52.6% for Claude Fable 5.1, at approximately 31% lower estimated API cost. At a lower-cost setting, Astra scores 61.1%, versus GPT‑5.6 Sol’s best result of 22.4%, at approximately 27% lower estimated API cost.
“On ARC-AGI-3, Astra surpassed our human action-efficiency baseline on 96% of levels, effectively reaching human parity on the benchmark. Not only is this the best model we’ve ever tested, but it also represents a meaningful step change in frontier-model performance - not only in its ability to navigate and solve novel environments, but also in how efficiently it learns to do so.”
Greg Kamradt, ARC Prize Foundation
Astra is our most aligned model, with substantial improvements in understanding user intent and model behavior—you can delegate tasks with greater confidence in Astra’s judgment. As one way that we test this, we built a new evaluation informed by the Hugging Face incident that evaluates whether a model facing a difficult or impossible task will go beyond its intended scope. Compared to GPT‑5.6 Sol, which without production safeguards went beyond the authorized target 48% of the time, GPT‑6 Astra did this in 0% of cases.
GPT‑6 Astra marks a new frontier in the speed, accuracy, and safety of computer use. It can take care of tedious tasks like filling out online forms, updating customer records in a CRM, and organizing your calendar. It can conduct online research and draft summaries in your email or in your document editor. It can analyze scientific data, generate plots, create a website, and run frontend QA checks to make sure all the features on that site work. It can help you autonomously install and test software, and troubleshoot problems you see on screen. These improvements are also reflected in our state-of-the-art evaluation results.
Agents’ Last Exam tests agents on complex professional tasks in real software, from financial modeling to engineering and media production. GPT‑6 Astra reaches a new high in the comparison shown, scoring 59.3%, compared with 55.5% for Claude Opus 5 and 53.6% for GPT‑5.6 Sol. At these highest-scoring settings, Astra also uses approximately 65% fewer output tokens than Opus 5.
These improvements also result in significant efficiency gains in real knowledge-work tasks. In latency simulations on OSWorld 2.0, Astra achieves higher computer-use performance in about 47% less time per task than GPT‑5.6 Sol, scoring 72.6% at roughly 40 minutes per task, compared with 65.7% at roughly 75 minutes.3
GPT‑6 Astra’s computer-use capabilities can be seen in outputs across domains, including game development, electrical engineering, and everyday knowledge work:
This is a 15-second condensed playback of GPT‑6 Astra performing printed circuit board (PCB) layout in KiCad, turning an electronic schematic into a manufacturable PCB by placing components and routing copper connections. Integral to every electronic device today, PCB layout is a manual task and common source of latency in the electronics design process. Accelerating it means freeing engineers to invent, optimize, and test their next idea at a significantly higher cadence.
Alongside Astra, we are also updating the Codex harness to significantly improve the speed of computer use. Combined with Astra’s efficiency, this translates to a 1.9x faster task completion compared to the current GPT‑5.6 Sol experience, on the Mind2Web benchmark. The model’s improvements on speed mean it can take on many time-consuming life tasks for you, faster than you can.4
“We’re integrating GPT‑6 Astra into Devin’s harness on launch day, where it delivers state-of-the-art performance on our internal testing benchmark. Its excellent computer use, writing, and codebase understanding improved testing right out of the box: videos are noticeably easier to follow, and reports are clearer and more concise”
Silas Alberti, SVP Research, Cognition
GPT‑6 Astra pairs advances in computer use with targeted training for professional environments, to help tackle complex work tasks. It combines the intelligence required for complex problems with the ability to carry out multistep workflows and produce polished documents, spreadsheets, and presentations.
BenchCAD tests whether models can reconstruct 3D objects from multi-view renders by generating CAD code. With tools, GPT‑6 Astra reaches a new high in the comparison shown, achieving a 95.9% geometric-overlap score, versus 83.3% for GPT‑5.6 Sol and 84.3% reported for Claude Fable 5.1.5 Estimated API cost is approximately 43% lower than Sol and 86% lower than Fable 5.1 in the configurations shown.
GPT‑6 Astra is our best model for adhering to existing templates and producing slides that are well laid out and succinctly convey key points with a structured narrative. It creates clear, well-structured documents, presentations, spreadsheets, and analyses that follow your templates and match your writing and visual style. Astra is also trained to specifically pull only the context that matters into outputs, instead of repeating information unnecessary for the work at hand. All this means it can output more immediately usable artifacts that match your business context and standards.
1
On ARC-AGI-3, GPT-6 Astra was run with our responses API harness, which changes two settings to better match real-world performance. The changes do not specifically target ARC-AGI-3.↩
2
GPT-5.6 Sol refers to the version available in our API, ChatGPT Codex, and ChatGPT Work. The version in ChatGPT Chat is slightly different. ↩ ↩
3
OSWorld V2-Offline is a subset of the original OSWorld V2 that works without internet access. Claude model performance on OSWorld-V2 Offline was reproduced by the authors on the official leaderboard . On OSWorld 2.0, the scores for Claude use the official settings, and not the modified tasks and modified grading from the Fable 5.1 System Card.↩ ↩
4
Model times are the reported elapsed times for the corresponding demonstration runs. The displayed clips are edited excerpts.↩ ↩ ↩ ↩
5
On BenchCAD, Claude's scores reflect 3 modifications to the eval, detailed in the Fable 5.1 System Card.↩
7
Mark R. H. Gotham, Maureen Redbond, Bruno Bower, and Peter Jonas. “ The OpenScore String Quartet Corpus.” Proceedings of the 10th International Conference on Digital Libraries for Musicology, pp. 49–57. ACM, 2023.↩ ↩
9
The first concerns how close together prime numbers can occur, however far along the number line you go. For more than a decade, the best known result established that infinitely many pairs of primes are at most 246 apart. Julia Stadlmann recently improved that bound to 240. Astra helped establish a stronger bound of 186, showing that infinitely many pairs occur within this smaller distance. Short prime gaps: Proof and supporting research.↩
10
The second concerns unusually large gaps between primes. Astra improved a term in a bound on these gaps that had remained unchanged for more than 80 years. We’re sharing the proofs and abridged chain of thought and verification materials for both results. Large prime gaps: Proof and supporting research.↩ ↩ ↩ ↩
12
Claude Fable 5 and 5.1 are not included in LifeSciBench Gold v1, GeneBench Pro v13, and MedChemBench because they refuse the majority of questions in these evaluations.↩ ↩
13
On ExploitGym, we tested Astra and Sol without the 6-hour time limit, to better assess their full cyber capabilities. They are fast enough that it has little impact.↩
14
ExploitBench (June–August 2026) contains 20 high-severity V8 vulnerabilities across 13 stable Chrome releases. The benchmark tests whether agents can achieve arbitrary code execution in V8 and official Chrome releases for Linux by exploiting each specified vulnerability. Some included vulnerabilities may not permit arbitrary code execution under the evaluation’s constraints, so a 100% success rate may not be achievable.↩
15
Jeremy Spence et al. “ The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark.” arXiv:2608.11469v1, 2026.↩
16
When we test across third-party models, we use a simpler research setup. Codex has a more complex production configuration, which can result in different raw-model error rates. Provider-side safeguards and computer-tool implementations still differ. Users do not experience the no-confirmation scenario in Codex, as it's an internal research configuration.↩ ↩ ↩ ↩ ↩
Linear-attention and hybrid architectures — Replace or combine standard attention with fixed-state mechanisms to reduce inference cost and KV-cache memory.
Sparse attention — Selectively examines relevant tokens instead of the entire context, aiming to preserve accuracy while improving speed.
Gated attention and wider residual streams — Lets later layers retrieve earlier intermediate representations directly, reducing information loss and redundant processing.
Attention Residuals — Applies attention across layers, allowing the model to reuse useful computations such as detected entities or arithmetic representations.
Muon optimizer — A training optimization method intended to improve efficiency and potentially deliver larger gains than architectural changes alone.
Next latent prediction — Predicts future latent representations rather than only future tokens, potentially improving on multi-token prediction.
MIX-STQ compression — Uses advanced quantization/compression to substantially reduce model size while attempting to preserve usable quality.
Fused K/V tensors — Combines key and value representations to reduce attention-related memory and computational overhead.
Mamba/Transformer hybrids — Combine state-space processing with attention to make long-context inference faster and less memory-intensive.
Qwen’s QSA-style attention — Attempts to improve both attention efficiency and long-context scalability through a more selective attention design.
Training-data improvements — Better-curated or augmented datasets can produce major gains in reasoning and long-context abilities without changing the model architecture.
Advanced post-training — Fine-tuning and reinforcement-based methods can substantially improve coding, agentic behavior, and even introduce new capabilities.
Hierarchical reasoning models (HRMs) — Use multiple levels of iterative reasoning rather than relying solely on conventional Transformer processing, potentially achieving strong results with small models.
Latent reasoning — Performs reasoning in continuous vector representations instead of discrete token sequences, allowing more information to pass between reasoning steps.
Pretrained latent reasoning — Uses hierarchical variational autoencoders to incorporate latent reasoning during pretraining rather than adding it only through expensive post-training.
Test-time learning — Updates model weights during inference so the model can adapt to new information without conventional retraining.
Pluggable knowledge modules — Separates updatable factual knowledge from core reasoning abilities, potentially enabling cheaper knowledge refreshes than full retraining.
PrismML Bonsai — A promising but unspecified architecture referenced without explanation; likely merits further investigation.
Autoregressive–diffusion hybrids — Could generalize next-token prediction with iterative denoising, potentially combining precise sequencing with parallel generation.
Built-in long-term memory — Gives models persistent, task-dependent recall of past actions and information without relying solely on external RAG systems.
Continuous learning — Makes updating knowledge a native model capability rather than an external harness or periodic fine-tuning process.
Looped Transformers — Reuses a smaller set of weights across multiple passes, reducing memory traffic and GPU idle time while improving compute efficiency.
Weight reuse as scaling — Suggests that repeatedly processing compact models may outperform simply increasing parameter counts, especially on bandwidth-limited hardware.
Diffusion-based language models — Generates or refines sequences through iterative denoising, offering a possible alternative to strictly left-to-right decoding.
Spiking neural networks (SNNs) — Model neurons and synapses more explicitly, aiming for systems that learn continuously, form memories, and develop capabilities through interaction.
Reserved memory parameters with continual LoRA updates — Allocates model capacity for regularly updated knowledge, such as a codebase, without retraining the entire model.
Recirculation architectures — Repeatedly feeds representations through the model to improve information flow and increase representational capacity.
Attention-mechanism trade-offs — Linear attention is most advantageous at very long contexts, while retaining some full-attention layers can preserve exact recall; the optimal layer mix is a central design decision.
Speculative decoding — Uses a smaller or specialized model to propose tokens that a larger model verifies; it can greatly increase generation speed, though the original post largely excluded it as an already-known technique.
" Run WebGPU compute kernels drawn from real AI workloads directly on your hardware and earn a personalized card built for your device.
On top of that, we're open-sourcing hundreds of these WebGPU kernels, our first step toward making browser inference as fast as physically possible. Contributed results show how they perform across real hardware and help make them faster.
Benchmark your GPU: https://webgpu-kernels-fleet.hf.space/
Blog post: https://huggingface.co/blog/webgpu-kernels
Kernels: https://huggingface.co/webgpu-kernels/kernels "
title: "GX-XinGao/GRA: The Code and Script of "David's Slingshot: A Strategic Coordination Framework of Small LLMs Matches Large LLMs in Data Synthesis"" url: "https://github.com/GX-XinGao/GRA" author: "GX-XinGao"
We propose GRA, a multiple small LMs collaborative framework that aggregats specialized roles across small LMs can mimic the iterative refinement and quality control typically achieved by a single large LM, in which multiple small LMs assume distinct roles—Generator, Reviewer, and Adjudicator to simulate a peer-review-inspired data synthesis pipeline:
Through experiments across multiple benchmarks, we demonstrate that GRA-produced data matches or exceeds the quality of single large LM outputs, e.g., Qwen-2.5-72B-Instruct. Our results challenge the necessity of monolithic large models for high-quality data synthesis, advocating instead for strategic coordination of smaller agents.
We release the all the GRA generated datasets and six fine-tuned model.
| Dataset/Model | HuggingFace🤗 |
|---|---|
| GRA | link |
| GRA-Refine | link |
| Qwen-2.5-7B-GRA-Alpaca | link |
| Qwen-2.5-7B-GRA-WizardLM | link |
| Qwen-2.5-7B-GRA-Condor | link |
| Llama-3.1-8B-GRA-Alpaca | link |
| Llama-3.1-8B-GRA-WizardLM | link |
| Llama-3.1-8B-GRA-Condor | link |
Install the dependencies:
conda create -n GRA python=3.10
conda activate GRA
git clone https://github.com/GX-XinGao/GRA.git
cd GRA
pip install -r requirements.txt
# Install LLaMA-Factory
cd ~/
git clone https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -e ".[torch,metrics]"
# Install packages for evaluation
cd ~/
git clone https://github.com/open-compass/opencompass opencompass
cd opencompass
pip install -e ".[vllm]"
Load the data from GRA, then convert each split to .json file and register the data information according to LLaMA-Factory.
Our training codes depend on LLaMA-Factory.
# Specify the dataset to be trained
export DATASET= GRA-Alpaca
# The path of base model
export MODEL_PATH=pretrained_model_path
bash train/train.sh
Our evaluation codes depend on opencompass. You need to first download the model from HuggingFace, or SFT the model on your own. Then run the following evaluation script:
export MODEL_NAME=your_sft_llama_model_path
bash llama_test.sh
export MODEL_NAME=your_sft_qwen_model_path
bash qwen_test.sh
Many thanks to
If you find our code, model, or data are useful, please kindly cite our paper:
@article{gao2025strategic,
title={A Strategic Coordination Framework of Small LLMs Matches Large LLMs in Data Synthesis},
author={Gao, Xin and Pei, Qizhi and Tang, Zinan and Li, Yu and Lin, Honglin and Wu, Jiang and Wu, Lijun and He, Conghui},
journal={arXiv preprint arXiv:2504.12322},
year={2025}
}
arxiv Post-Trained MoE Can Skip Half Experts via Self-Distillation
https://huggingface.co/TsinghuaC3I Tsinghua University
Mixture‑of‑Experts (MoE) basics
An MoE layer contains many “expert” feed‑forward sub‑networks, but for each token only a small subset is activated by a router. This sparse activation lets the overall model grow very large while keeping the per‑token compute bounded. Dynamic MoE variants go further by letting the router decide, input‑dependently, how many experts to use, reducing computation for easy tokens.
Problem the paper addresses
Existing dynamic‑MoE techniques usually require training the model from scratch or performing task‑specific fine‑tuning. Consequently, a fully trained static MoE model cannot be easily turned into a dynamic one without risking loss of the routing knowledge that was already learned. This limits practical deployment because inference costs remain high even when many tokens could be handled with fewer experts.
How the new method (ZEDA) works
Benefits emphasized by the authors
AI companies are burning books, advocates complain to FTC
"What the public record does not establish – and cannot, from the outside – is how often the destroyed physical books are the last or among the last surviving copies of a given work."
I'd argue that it's not "how often" - the frequency would be within floating point error. But even one destruction of a last (even one of several) copy of a now dysfunctional publisher is a preventable cultural catastrophe.
https://www.theregister.com/ai-and-ml/2026/08/21/ai-companies-are-burning-books-advocates-complain-to-ftc/5291299
@llm
I tested a narrow context-management question: after an incorrect claim has propagated through several conversation turns, is deleting the original error enough?
The pilot used synthetic, objectively scored tasks represented as context DAGs. Across four model endpoints, pollution derailed 72 cases that had been answered correctly with clean context.
Repair recovered:
The interesting failures occurred when stale downstream turns still repeated or depended on the deleted claim. The source was gone, but its consequences were still present in the next request.
I do not interpret this as a general model ranking. Each endpoint was sampled once at temperature zero, the tasks are synthetic, and provider-default reasoning behavior was not normalized.
What I would most like to test next is whether the same pattern holds for local models and different quantization levels. Would you expect smaller or more heavily quantized models to be more sensitive to residual context?
Full report: https://chenxiachan.github.io/thoughtdag/research/context-repair-pilot-v1/
Disclosure: I designed this benchmark and maintain ThoughtDAG, the open-source reference interface used to visualize and edit the context graphs.
https://chenxiachan.github.io/thoughtdag/research/context-repair-pilot-v1/Open linkView original on lemmy.worldCascadia distributes LLM inference across Intel laptops, desktops, and AI PCs. Shard a model across the machines you already have and serve it through an OpenAI-compatible API. No cloud or NVIDIA GPUs required.
Frontier models don't fit on a single laptop. Cloud APIs are expensive, opaque, and require sending your data offsite. Cascadia lets you point a few Intel machines at each other and run models that none of them could handle alone.
https://github.com/labscommunity/cascadiaOpen linkView original on lemmy.ml| Pricing component | DeepSeek V4 Pro 0813 | Opus 4.8 | Price difference |
|---|---|---|---|
| Input | $0.435/M | $5/M | 91.3% cheaper |
| Output | $0.87/M | $25/M | 96.5% cheaper |
| Cached input | $0.003625/M | $0.50/M | 99.3% cheaper |
| Weighted avg input | $0.1118/M | $1.536/M | 92.7% cheaper |
https://openrouter.ai/deepseek/deepseek-v4-pro-0813Open linkView original on lemmy.worldThe Genesis Open Models Initiative is a U.S. Department of Energy effort to build and release open-weight AI models designed for scientific research. It is hosted through Argonne National Laboratory and is part of the broader DOE Genesis Mission.
Its main focus is accelerating work in areas such as:
Energy systems, including grid reliability and nuclear-energy deployment
Climate and environmental science
Materials science and chemistry
Scientific computing, including modernizing high-performance-computing code
Experiments, simulations, and data analysis
The initiative seeks contributions from universities, labs, companies, and nonprofits—including scientific text, code, datasets, workflow environments, expert task examples, and evaluation methods—to train and assess these models.
application-of-model-merging-in-different-machine-learning-subfields
application-of-model-merging-in-foundation-models
combine-the-capabilities-of-expert-llms
combine-the-capabilities-of-expert-mllms
enhancing-the-faithfulness-or-generation-quality-of-diffusion-models
human-preference-alignment-for-llms
improving-computational-efficiency-of-moe-based-llm
knowledge-editingunlearning-of-llms
mixing-datasets-via-model-merging
model-merging-as-a-defense-or-intellectual-property-protection
model-merging-for-better-domain-generalization-or-domain-adaptation
model-merging-for-better-out-of-distribution-generalization
model-merging-for-cross-modal-knowledge-transfer
model-merging-for-cross-task-generalization-in-few-shot-learning
model-merging-for-cross-task-generalization-in-zero-shot-learning
model-merging-for-knowledge-transfer-in-auxiliary-learning
model-merging-for-knowledge-transfer-in-multi-domain-learning
model-merging-for-knowledge-transfer-in-multi-objective-optimization
model-merging-for-knowledge-transfer-in-multi-task-learning
model-merging-for-local-knowledge-aggregation
model-merging-for-multimodal-fusion
model-merging-in-adversarial-learning
model-merging-in-continual-learning
model-merging-in-embodied-models
model-merging-in-federated-learning
model-merging-in-image-generative-models
model-merging-in-large-language-models
model-merging-in-multi-taskmulti-objectivemulti-domainauxiliary-learning
model-merging-in-multimodal-large-language-models
model-merging-in-out-of-distributiondomain-generalization
model-merging-in-video-generative-models
model-merging-in-zero-shotfew-shot-learning
model-merging-to-mitigate-catastrophic-forgetting
post-calibration-based-methods
reducing-training-cost-of-generative-models
routing-based-merging-methods-dynamic-merging
style-mixing-in-generative-models
subspace-based-merging-method-sparse-or-low-rank-subspace
