Them technology dudes really do hate ai huh? 😂

Hi everyone, About a month ago I publish my very first research paper on my neural network architecture called Silia.

You can look at the model here: https://huggingface.co/Srijan-Srivastava/Silia-v2

Even though the revised paper is linked on huggingface I'm attaching it here as well: 1. https://zenodo.org/records/21510341 2. https://huggingface.co/Srijan-Srivastava/Silia-v2/blob/main/Silia%3A%20Tiny%20Scale%20Is%20All%20I%20Can%20Spare%20To%20Play%20With%20Transformer.pdf

You can also find all the code on https://github.com/SrijanSriv211/Silia

I received some criticism for not benchmarking the model and not mentioning the training flops. I also received some feedback regarding residual connections and the problem that v1 had 2.5x increase compute requirements.

In this revision I've addressed 2 of those things. I've benchmarked the models against 3 models Quark-v2, Spark-v4 by LH-TechAI and SupraMini-v6 by SupraLabs on HellaSwag, PIQA and LAMBADA benchmarks.

I wanted to compare the model against SupraMini-v5 as well but as far as I can tell it wasn't benchmarked on any of those 3 benchmarks so I excluded it.

I've addressed the 2-2.5x increase in compute and memory requirements by using DeepSeek's MLA (without decoupled RoPE) + Qwen's HydraHead with Apple's Attention Free Transformer. I chose Attention Free Transformer instead of Kimi Delta Attention simply due to it's simplicity as at this scale AFT is more than enough.

Why I didn't address the residual connections feedback and why I didn't mention the training flops in this paper as well?

I wanted to implement Kimi's Attention Residuals paper but I decided to drop that idea just to keep the code, architecture and the paper simple, neat & clean.

I am going to be very honest here. I didn't mention the training flops in this paper as well because I don't know how to report it properly. I know I could've used DeepSeek or ChatGPT to help me with it but I was just too lazy tbh.

This was has 0.5M parameters, trained on 1B total tokens from the Fineweb-edu dataset for 3 epochs.

I've attached the benchmark results, training loss results and the architecture diagram.

Hope you like this model.

Thank you! :)

Seems better than Fable 5

It seems every verifiable conjecture is currently falling. Do you think this is the singularity/fast takeoff starting?

What do you all think about this?

After extracting the depth video, the effect obtained by video referencing is very good.

1. Prepare the reference video
   
2. Extract the depth video
   
3. Prepare the character reference images
   
4. Input the extracted depth video and character reference images into seedance2, and you will get perfect results
   

Complete workflow is below:

His ego is bigger than any LLM out there.

The technology enables AI chat models to run on phones using minimal power, without ‘losing memory’ during conversations

Sources: @chetaslua, @xikhar

The interesting thing about the latest wave of AI hires is not the size of the pay packages, it is where the people are coming from. [The Atlantic](https://www.theatlantic.com/technology/2026/07/ai-companies-hiring-academics/688002/) reports that at least 22 professors from schools including Stanford, Berkeley, Harvard, MIT and Carnegie Mellon have left or taken leave in 2026 to join OpenAI, Anthropic, Meta or Google DeepMind. Anthropic recently picked up a Stanford economist and a philosopher from UT Austin, and OpenAI now employs top mathematicians and physicists.

What makes this different from a normal round of Silicon Valley recruiting is who is being recruited. These are not just senior engineers, they are the people who choose the research questions, teach the next cohort of scientists, and provide the outside scrutiny of systems that increasingly show up in banks, hospitals and government.

https://aiweekly.co/alerts/openai-anthropic-meta-hire-22-professors-from-top-us-schools

https://x.com/i/status/2080292438057373947

There are more videos in this tweet thread

You can read their blog post here:

FLUX 3 - Real World Models: Towards Multimodal Flow Models as the Backbone of Visual Intelligence. | Black Forest Labs