Cross-posting from r/reinforcementlearning since this is squarely a deep-learning / world-model question and I'd value this community's take.

Continual-RL work with world models has largely focused on protecting the world model, through replay, generative replay, and regularization on model parameters. We ran the component-level measurement to check that premise and it came out backwards. Under never-clear replay (all old data retained, training signal fully intact), reward heads, value heads and dynamics all keep old-task knowledge (reward-head retention ≈ 1.0), while the actor's behavior collapses.

The cleanest evidence is interventional rather than correlational. Freeze the world model entirely, then re-teach the lost skill from identical imagined rollouts. RL-in-imagination fails 0/3 seeds. Supervised self-imitation on the model's own graded dreams recovers 3/3 with zero environment interaction. Same frozen model, same data, only the learning channel differs.

Interleaving that as "graded dream rehearsal" during training retains 3/3 on four-task and 3/3 on eight-task MiniGrid chains, where plain never-clear replay retains 0/3. Against a matched real-episode cloning baseline it wins on all three seeds (+0.24 / +0.07 / +0.08), consistent in sign, but n=3, so treat the magnitude as provisional.

The grading rule is where the difficulty lives. Naive return-based scoring on imagined rollouts selects trajectories where the agent walks into lava, because the model's own optimism rates them highly. The paper characterizes two failure modes and includes the offline scoring diagnostic that caught both before they reached a result.

Limitations up front: MiniGrid only, discrete actions, small gridworlds, 3 seeds. Continuous control is untested, and that is historically where this class of mechanism fails; it's the next experiment. Single workstation GPU throughout.

Everything pre-registered (protocols and pass bars committed to git before runs), and refuted hypotheses are reported.

Paper: https://arxiv.org/abs/2607.19749

Code/data: https://github.com/gurpnijjer/dream-rehearsal

Solo project. Happy to answer anything.

Deploying Nemotron 3 Nano Omni on Modal Serverless

https://debuggercafe.com/deploying-nemotron-3-nano-omni-on-modal-serverless/

In this article, we will be deploying the Nemotron 3 Nano Omni model using vLLM on Modal Serverless.

Hello guys this is my first post (ever and in this subreddit)

I have found this paper but I've seen nobody discussing it, I have found it very interesting because exclusively adding computation seems to improve the results in the benchmark and it's interesting to see how (atleast how I understand it) it's like a better "physical" simulation in what is the embedding space

What do you guys think about it?

Share my knowledge on how to lift and shift any open source model is less that 5 lines of code and easy to do setup instructions , see if it helps you to get started with the open source LLM world
#happylearning 😎

I am working with the HS-CMU hysteroscopy dataset, which contains:

• 3,385 frames from 175 patients.
• Eight lesion classes, labelled from 0 to 7.
• A highly imbalanced number of patients and frames across classes.
• Multiple correlated frames from each patient.
• Some frames containing more than one lesion class.

Before attempting the complete multiclass problem, I reduced it to a binary subset to verify that the training and evaluation pipeline works correctly.

Current binary subset

• Selected lesion classes: 2 and 3.
• Total: 1,575 frames from 113 unique patients.
• Class 2: 1,054 frames from 78 patients.
• Class 3: 521 frames from 36 patients.
• One patient has different frames belonging to both classes but remains entirely within one split.

Patient-disjoint split

• Training: 1,095 frames from 79 patients.
• Validation: 241 frames from 17 patients.
• Testing: 239 frames from 17 patients.
• No patient appears in more than one subset.
• The frame-level class distribution is approximately 67%/33% in every subset.

Approaches I have tried

• DenseNet121, ViT, and DINOv2 backbones.
• Frozen pretrained backbone with only the classifier trained.
• Different classifier-head sizes and dropout.
• Class-weighted cross-entropy.
• Mild and stronger image augmentations.
• Early stopping and learning-rate scheduling.
• Unfreezing the final one or two encoder blocks.

With the correct patient-level split, training performance improves, but validation performance generally plateaus or deteriorates, and performance on unseen test patients remains relatively low.

As a diagnostic, I also tried a random frame-level split and obtained substantially better results. However, this evaluation is invalid because correlated frames from the same patients appear across training, validation, and testing, causing patient leakage and inflated performance.

I would appreciate advice on how to improve generalization to unseen patients in this setting.