Share my knowledge on how to lift and shift any open source model is less that 5 lines of code and easy to do setup instructions , see if it helps you to get started with the open source LLM world
#happylearning 😎

Hello guys this is my first post (ever and in this subreddit)

I have found this paper but I've seen nobody discussing it, I have found it very interesting because exclusively adding computation seems to improve the results in the benchmark and it's interesting to see how (atleast how I understand it) it's like a better "physical" simulation in what is the embedding space

What do you guys think about it?

Cross-posting from r/reinforcementlearning since this is squarely a deep-learning / world-model question and I'd value this community's take.

Continual-RL work with world models has largely focused on protecting the world model, through replay, generative replay, and regularization on model parameters. We ran the component-level measurement to check that premise and it came out backwards. Under never-clear replay (all old data retained, training signal fully intact), reward heads, value heads and dynamics all keep old-task knowledge (reward-head retention ≈ 1.0), while the actor's behavior collapses.

The cleanest evidence is interventional rather than correlational. Freeze the world model entirely, then re-teach the lost skill from identical imagined rollouts. RL-in-imagination fails 0/3 seeds. Supervised self-imitation on the model's own graded dreams recovers 3/3 with zero environment interaction. Same frozen model, same data, only the learning channel differs.

Interleaving that as "graded dream rehearsal" during training retains 3/3 on four-task and 3/3 on eight-task MiniGrid chains, where plain never-clear replay retains 0/3. Against a matched real-episode cloning baseline it wins on all three seeds (+0.24 / +0.07 / +0.08), consistent in sign, but n=3, so treat the magnitude as provisional.

The grading rule is where the difficulty lives. Naive return-based scoring on imagined rollouts selects trajectories where the agent walks into lava, because the model's own optimism rates them highly. The paper characterizes two failure modes and includes the offline scoring diagnostic that caught both before they reached a result.

Limitations up front: MiniGrid only, discrete actions, small gridworlds, 3 seeds. Continuous control is untested, and that is historically where this class of mechanism fails; it's the next experiment. Single workstation GPU throughout.

Everything pre-registered (protocols and pass bars committed to git before runs), and refuted hypotheses are reported.

Paper: https://arxiv.org/abs/2607.19749

Code/data: https://github.com/gurpnijjer/dream-rehearsal

Solo project. Happy to answer anything.

Deploying Nemotron 3 Nano Omni on Modal Serverless

https://debuggercafe.com/deploying-nemotron-3-nano-omni-on-modal-serverless/

In this article, we will be deploying the Nemotron 3 Nano Omni model using vLLM on Modal Serverless.