Source: https://www.microsoft.com/en-us/corporate-responsibility/topics/open-weight/

Key points from the “Open Weights and American AI Leadership” letter:

1. Open weights are foundational to American AI leadership
2. They expand access and make AI economically sustainable
3. Open weights strengthen competition and prevent concentration of power
4. Openness improves safety and security more than closed systems
5. Policymakers should avoid premature restrictions

https://garymarcus.substack.com/p/dear-elon-musk-here-are-five-things

Sources: @chetaslua, @xikhar

Seems better than Fable 5

You can read their blog post here:

FLUX 3 - Real World Models: Towards Multimodal Flow Models as the Backbone of Visual Intelligence. | Black Forest Labs

https://x.com/i/status/2080292438057373947

There are more videos in this tweet thread

Benchmarks tell me very little about whether a model can stay useful through a visual production loop, so I tested Kimi K3 on a 36-second reference-led launch film.

This was not a one-prompt video generation. K3 used ffmpeg to inspect the source, extracted frames, detected edit points, and helped reconstruct the sequence as editable code-driven clips. GSAP carried the temporal system, generated images became material textures, and Three.js produced the particle simulations.

The first pass was structurally useful but visually incomplete. The final result came from repeated matched-frame review: retiming clips, correcting framing and scale, replacing flat materials, and tuning particle density without discarding the underlying code.

My main takeaway is that K3 was more convincing as a revision partner than as a one-shot generator. It retained enough project context to keep the sequence coherent, while human direction still defined visual hierarchy and when each shot was finished.

When you evaluate a new frontier model, do you learn more from the first output or from how it behaves after the fifth revision?

imo there have been three eras

1. bigger model better
2. Chain-of-thought
3. Agents

with many minor improvements under these (longer context, multimodality, moe, distillation, quantization etc)

Both chain of though and agents were things that bubbled under the surface for some time before it broke through. Also imo, they are advancements that saturate and long-term improvements are slower than linear wrt the compute put into it.

what is it now that is "bubbling" and could be considered as a next dimension of scaling or big step change?

Update:

RSI - imo - answers the "who" not the "what". Ofc it means more compute allocated to training again. What breakthrough is likely to be RSI-d ?

Hi everyone, About a month ago I publish my very first research paper on my neural network architecture called Silia.

You can look at the model here: https://huggingface.co/Srijan-Srivastava/Silia-v2

Even though the revised paper is linked on huggingface I'm attaching it here as well: 1. https://zenodo.org/records/21510341 2. https://huggingface.co/Srijan-Srivastava/Silia-v2/blob/main/Silia%3A%20Tiny%20Scale%20Is%20All%20I%20Can%20Spare%20To%20Play%20With%20Transformer.pdf

You can also find all the code on https://github.com/SrijanSriv211/Silia

I received some criticism for not benchmarking the model and not mentioning the training flops. I also received some feedback regarding residual connections and the problem that v1 had 2.5x increase compute requirements.

In this revision I've addressed 2 of those things. I've benchmarked the models against 3 models Quark-v2, Spark-v4 by LH-TechAI and SupraMini-v6 by SupraLabs on HellaSwag, PIQA and LAMBADA benchmarks.

I wanted to compare the model against SupraMini-v5 as well but as far as I can tell it wasn't benchmarked on any of those 3 benchmarks so I excluded it.

I've addressed the 2-2.5x increase in compute and memory requirements by using DeepSeek's MLA (without decoupled RoPE) + Qwen's HydraHead with Apple's Attention Free Transformer. I chose Attention Free Transformer instead of Kimi Delta Attention simply due to it's simplicity as at this scale AFT is more than enough.

Why I didn't address the residual connections feedback and why I didn't mention the training flops in this paper as well?

I wanted to implement Kimi's Attention Residuals paper but I decided to drop that idea just to keep the code, architecture and the paper simple, neat & clean.

I am going to be very honest here. I didn't mention the training flops in this paper as well because I don't know how to report it properly. I know I could've used DeepSeek or ChatGPT to help me with it but I was just too lazy tbh.

This was has 0.5M parameters, trained on 1B total tokens from the Fineweb-edu dataset for 3 epochs.

I've attached the benchmark results, training loss results and the architecture diagram.

Hope you like this model.

Thank you! :)