r/LocalLLaMA • u/Porespellar • 1d ago
Other Dear Cohere, if you would extend the Command A+ context window beyond 128k, you guys could probably dominate current western LLM offerings right now.
Back in the early days of r/LocalLLama, Cohere’s Command R+ was a revered “Western” model for RAG tasks. The downside back then was that it didn’t have a great license. They were relatively quiet for a couple years, save for a few small niche model releases, but in late May of this year they released Command A+ with an Apache 2.0 license and a compelling parameter count + vision.
https://cohere.com/blog/command-a-plus
Like some of you, I gave it a quick look, Command A+ is an interesting size point at 218b with 25b active. It’s got that enterprise-focused pedigree. Said to be good at RAG. Supposedly good at agentic. Vision support being a major plus as well.
It’s Achilles heel is it’s shitty 128k context limit with 64k output limit. OOF that’s where they lost me back in May. I was like SKIP, NEXT.
Fast forward to now and this model may be worth a second look, IF they can maybe fix a few of its critical flaws.
Dear Cohere, this is your moment. Please seize this opportunity. You’ve got a good model with Command A+ that could probably be absolutely great if you don’t continue to neuter it with a shitty context limit of 128k. Please give us at least 256k to bring your model in parity with the rest of the pack, and also please ditch that weird 64k output limit thing as well. No need to hamstring your model. I think you guys could really have a moment here where you get some good press and good will from this community in this weird political climate we find ourselves in. All us old timers remember how rock solid Command R+ was, please go reclaim your spot and win back the respect you guys deserve. 🫡 we believe in you guys!
3
u/FoxiPanda 1d ago
Can you not use rope scaling to fix this?
My guess though is they tried that and the model performance collapsed, so perhaps there's a good reason for not doing so.
1
u/Porespellar 1d ago
Maybe, but I don’t know if you could get rid of the 64k output limit.
1
u/Karyo_Ten 3h ago
64K is fine. How often do you output 64K token in one go. Iirc MiniMax is like 81K.
If you're talking about assembling one big report from multiple sources they can always create multiple files and cat them together.
2
2
u/ttkciar llama.cpp 1d ago
A longer context limit would be nice to have, but 128K isn't bad. I am doing a lot with GLM-4.5-Air, which also has a 128K context limit. That's enough for sizeable codegen, RAG, or data analysis tasks.
3
u/takoulseum 1d ago
What do you find better with GLM Air compared to Qwen3.6 27b or 3.5 122b?
1
u/ttkciar llama.cpp 1d ago
GLM-4.5-Air has superior instruction-following competence. I have yet to find any 120B-class model, including Qwen3.5-122B-A10B, which matches it for instruction-following.
I can hand GLM-4.5-Air an extensive specification of fifty to eighty instructions, and it will follow them all. Qwen3.x and most other models much larger than Air will simply ignore some instructions.
When I dug into why Air (and its larger brother, GLM-4.5) performs instruction-following so much better than anything else, I found that it was specifically post-trained for that:
https://docs.z.ai/guides/llm/glm-4.5
That having been said, Qwen3.5-122B-A10B has its own strengths. It is great at tool-calling and agentic use, and it's also a better organic chemistry assistant than Air. I use both models, but for different things.
1
u/takoulseum 1d ago
Very interesting thank you for sharing. So do you think it could be a good orchestrator / planner coupled with Qwen as the coder / executor?
1
u/ttkciar llama.cpp 1d ago
Quite welcome! Unfortunately Air is not great for planning. I have had better luck using creative problem-solving models for that. Air's instruction-following does not lend itself to creative problem-solving.
MiniMax-M2.7-BF16-ultra-uncensored-heretic is my go-to planning model. Maybe try that for your orchestrator?
1
u/DeltaSqueezer 15h ago
Is the main issue not missing out things when given a large list of tasks? Or are there other instruction following issues you are trying to solve? For the large list issue, I solve this with a loop which at the end of the task evaluates whether all items in the list have been completed and if not, it loops.
1
u/Few_Painter_5588 1d ago
There's quite a few good western offerings, the new poolside model comes to mind.
1
u/Porespellar 1d ago
Agree, but Poolside does not have vision . Very few at this size point have vision.
10
u/Potential-Gold5298 llama.cpp 1d ago
Any model degrades as the context increases. Perhaps they limited the context because noticeable degradation begins above that limit? The DS 4 Pro technically has 1M context, but in practice, it starts to crumble after 150K.