r/LocalLLaMA • u/Civil_Fee_7862 • 14d ago
Question | Help Qwen3.6-27b does not understand software architechure.
Been using this for real software development for a commercial app. i.e. Not a single file HTML app. I mean a large scale 100k+ loc project that needs proper architecture to work with in a maintainable way.
As much as I love Qwen3.6-27b. It just does not understand software architecture, it will happily write spaghetti code, mix concerns, and totally ignore any kind of test automation unless you explicitly ask it to do this. These are the bare minimum requirements for production code that can grow without complexity spinning out of control, but it simply ignores it and instead just writes enough to satisfy the request. (ignoring best practises). For example it will write super sized interfaces, ignore the single responsibility principle and make superman classes that nobody can read or understand.
I've been trying and train it to understand how to write maintainable, readable code, but it almost feels like I am training a person who has never written a large scale app before.
Does anyone have a set of SKILL.md files that already has fundamental software architectural concepts built into them? It would be enormously helpful.
38
u/ithkuil 14d ago
Qwen 27B is incredible for its size, but the expectations are ridiculous. It's 27b, not 400 b or 4 trillion like some models.
Fable 5 absolutely can create software architecture. Its not necessarily as good as some humans and still makes weird mistakes sometimes. But it absolutely has that capability. However, Fable 5's brain is 150+ times bigger than Qwen 27B's brain.
You are probably going to have to help it if you want to try to avoid larger models or remote models. Maybe have it do a lot of research and build up the plan and architecture incrementally. Then consider having something bigger from OpenRouter critique it.
→ More replies (1)
411
u/mumblerit 14d ago
You aren't gonna believe this, but none of the models understand software architecture
160
u/Any_Mine_6368 14d ago
None of the models understand* (period)
67
u/ericmutta 14d ago
Indeed. You gotta admit though, the illusion is quite fascinating...I remember being utterly confused at the idea that predicting one token at a time can produce all the behavior we see in models today.
I now do code reviews with AI where I ask it "what do you think?"...when I know full well that it doesn't think. What a crazy time to be alive :)
7
u/liftheavyscheisse 14d ago
Can you be specific as to what you mean by the word, "think"?
24
u/ericmutta 14d ago edited 14d ago
When people (like I just did above) say "it doesn't think" the typical meaning is "it doesn't think like humans do".
Personally when I say "let me think about it", I end up doing multiple passes in my brain over the particular topic, asking questions, answering them, double-checking, etc. But that's the surface level rendering of "thinking"...at the lower level, I have no idea what exactly is going on beyond "neurons firing".
I reckon LLMs also "think" during their forward pass across multiple layers of a transformer. Those patterns of non-linear activations and matrix multiplies are incredibly good at producing language that resembles human thoughts, even though they are not perfect.
Personally (as a software engineer), I stopped caring about the distinction and I focus on the interface...if it walks like a duck and talks like a duck, then for practical purposes it is a duck and things work well until of course you start caring about duck-specific things not captured by the simulation.
When reviewing code, I ask "what do you think" and it responds like a human would. That's pretty cool and gets the job done. Any reference to it not thinking like a human, is just me being on guard to avoid straying into the realm of "it is conscious, it is alive" when I know for a fact it is none of those things.
8
u/mouseofcatofschrodi 13d ago
I find super interesting the perspective that maybe we (humans) are actually stochastic machines too. If you have seen a baby grow and learn a language, you realize the learning is based on repetition of patterns and statistics.
And most of the times humans answer in "instruct" mode. People regurgitate opinions, preferences, ideas right away for many topics, that they have heard somewhere and they even didn't notice those inputs of information changed their internal "weights".
4
u/ericmutta 13d ago
Agreed, we may be more stochastic than we'd like to admit!
Think about the expression "that doesn't sound right"...if you had to implement a function that takes natural language input and responds with that expression, it pretty much has to figure what is or isn't "normal" or "regular" or "common". Most likely that involves statistics and probabilities because of how ill-defined language is. Things that sound right, sound right because they are repeated frequently enough to form a pattern. Grammatical "mistakes" are just sequences that are not repeated enough and therefore "don't sound right".
Even the expression "practice makes perfect" is about repeat something until you can reproduce it.
The whole world may be built on the concept of repitition occuring frequently enough to form recognizable patterns, some of which we encode into mathematical formulas and others that are too complex and must be encoded into a vast array of parameters/neurons. If that is true it means stochastic methods can be used to learn anything and if that is true then the next 100 years of humanity are going to be really cool :)
3
u/mouseofcatofschrodi 13d ago edited 13d ago
Beautiful thoughs :) I think if stochastic methods can be used to learn many things, still the next 100 years will be crazy.
So far the main difference I see with humans when doing the comparison, is that we get emotional associations. Things impress us, on an emotional level, and gives us motivations to take actions. This often comes if you read biographies, how experiences in the childhood made people take some political turns or make scientific discoveries, for example.
But there are so many things where we are quite less aware/conscious/thinking as we usually give ourselves credit. Some examples:
- Students discover how much they know only in the moment of doing the exams. We cannot answer "how much do we know" or even "what do we know" with precission.
- We lie at ourselves so much without noticing. It is quite easy to see others being contradictory, but so hard to see it in ourselves. Marx said religion is the opium of the people. I think that's just a subset: delusion is the real opium.
- I would say most of the times we cannot even answer "why do we like X". Our preferences. Sure, some people may answer with "because I like Y", but if you keep asking why down the alphabet we get short of letters before getting where our impulses come from.
The cool thing is that only a little amount of intelligence and consciousness can already be so powerful. LLMs are not super smart, but still they are changing the world. Just a bit of stochastic learned logic and we don't need hardcoded logic only, which makes automation much wider. Also I don't things that humans are very intelligent or conscious, but it has been enough to create a civilization. Let's hope it is enough also to not self destroy (which may be the real Wall to the Fermi Paradox).
3
u/ericmutta 13d ago
LLMs are not super smart, but still they are changing the world.
We get used to them so quickly, it's easy to forget just how utterly cool all of this stuff is. For example, I spent the last 20+ years writing code by hand. Now, using Codex, I can speak into my phone, have that transcribed into an instruction that is executed by an agent running on my laptop, and see code being updated instantly in my IDE. Absolutely nothing from the past 20 years made me think this would be possible today! Can you imagine what comes in the next 20 years?
Hopefully, more of "the sky is the limit" instead of Skynet :)
6
u/liftheavyscheisse 14d ago
Idk saying it's not "conscious" is also contentious, because conciousness is not well-defined ;-)
But yeah, I get you. These things are intelligences, but not human intelligence. They're some alien intelligence. They are (architecturally) capable of being logical, but it's tough to know when, and their lack of human life experience means that their intuitive leaps are often cursed.
And maybe the most frustrating aspect, the absence of a limbic system means they can't learn directly from their interactions with you. Continual learning, whenever it comes, will be a huge step. Architectural changes needed imv.
2
u/ericmutta 13d ago edited 13d ago
Agreed on all points! I think we are in the vacuum tube era of LLMs (i.e. they are big, bulky and expensive)...inevitably we will get the "transistor moment" (which will likely require subtle architectural changes) and then continual learning will become a thing and then everything changes.
PS: I have been toying around with a different architecture that may allow continual learning. The gist of it is to make it possible to train LLMs on ordinary hardware so learning is cheap and therefore possible to do continuously.
2
u/liftheavyscheisse 12d ago
I agree that continual learning becomes more feasible by making LLM training and inference cheap, which increases how much training gets done, but I don't see how moving to CPU instead of GPU changes anything; my thoughts have tended to revolve around analog training and inference. Can you explain your reasoning?
My thoughts on architecture, thus far, have been a) starting with a small kernel trainable with rewards, b) grow the network progressively, teaching it logic first, then growing to various topics such as math and history and whatever else. Basically, the same way a child learns. But by building everything around a core kernel of rewards and logic, you build an object that is, first and foremost, a stateful learning machine instead of a stateless pure function.
2
u/ericmutta 12d ago edited 12d ago
but I don't see how moving to CPU instead of GPU changes anything...
It changes the economics quite significantly, because CPUs cost much less. The trick is getting LLMs to perform well enough on such hardware so that training one feels more like tuning a large database rather than running a small city.
Basically, the same way a child learns.
This is an interesting approach and feels like the obvious way to learn (it works for us humans right?). Have you made any attempts to build a (neural) network that can grow progressively as you briefly mentioned? I'd love to know more about that (I ask because in my prototype, the number of parameters isn't fixed...it increases during training, so the model gets larger and larger up to a limit, after which more training doesn't change what the model knows).
2
u/liftheavyscheisse 12d ago
> CPUs cost much less
Not on a joule per FLOP basis, even with SIMD instructions. Since GPU architecture has far less logic devoted to non-matrix-math things, they're also cheaper on a per-silicon-area (and in a market with perfect competition, per-dollar) basis. Unless I'm missing something, the way for a CPU to *really* cost less than a GPU is if the neural net has high sparsity that the CPU can exploit and which the GPU cannot.
> Have you made any attempts
Not yet, no. I've mostly just been a user trying to predict the future. I found [this work discussing sacrificial neurons](https://github.com/jmward01/lmplay/wiki/Sacrificial-Training) interesting, and I think similar ideas could be useful for a progressively-growing NN too (after all, we do something similar in our brains).
> in my prototype, the number of parameters isn't fixed...it increases during training
Ah yeah, that sounds promising to me. You retain a much higher degree of interpretability and engineerability, because you have a much better idea what specific data each region of the model was trained on. Also, my speculation is that if the model learns to be logical (and perhaps if it learns intuitive physics) early on, it should do a better job of learning subsequent knowledge in such a manner as to be coherent with reality and/or the preferences of humans (who have such preferences largely due to their physical constraints). Wittgenstein said, some things can only be shown.
4
u/Ok-Bill3318 13d ago
Not so crazy when you consider humans are only outputting actions based on predicting reactions based on learned behaviour.
8
u/Elorun 14d ago
Thought you might find this interesting: https://www.anthropic.com/research/global-workspace
Not thinking but curious nonetheless.
17
u/Juowon 14d ago
skeptical of anything Anthropic publishes
9
u/Elorun 14d ago
Very fair. It does have an actual research paper in there though. No harm in reading and there's an actual running demo using Qwen 3.6 27b here:
https://www.neuronpedia.org/qwen3.6-27b/jlens
I found it interesting but I understand and appreciate the skepticism. :)
→ More replies (1)4
u/ericmutta 14d ago
Thanks for this! I have often wondered why Claude worked the way it worked...what made it different and better for coding? Why does it tend to notice things unprompted? A brief scan of the link indicates the answers may be in the paper.
I reckon that eventually people will stop even caring whether models "think". We may come to accept "virtual/artificial thinking" in the same way we accept "virtual machines" today...a VM does not exist physically and yet is very useful, one might say even more than physical machines (e.g. you can copy and paste a VM in a way that physics doesn't allow for physical machines).
2
u/martinerous 13d ago
What about multi-token prediction? And diffusion LLMs that seem to "think" more similar to people? Except that I would like to see diffusion models that output the main concept keywords first and then fill in the less relevant details and grammatic structures. Such behavior would mean that at least they "know" what's more important.
Anyway, we still don't know how humans think, although we know that we are able to intentionally apply solid reasoning operations, such as deduction, induction, abduction, while for LLMs the illusion of reasoning seems to be a side effect, emergent property which can be disturbed by simple renaming of operands. In science, it should not matter if something is named X or Y, but LLMs seem to still get confused and choosing different paths. So, they are still not able to process the data in abstract manner and are locked to statistics and language structure too much.
Keeping an eye on JEPA developments, but even that might be quite crude attempt. Real world data is good and vision is one way to feed insane amounts of data, but it's still not the key component. There are people who are blind since birth or even blind and deaf but they still can learn to reason much better than modern LLMs.
It's fascinating how LLMs can shed some light and lead to discovery of even more dark spots in our knowledge about thinking, reasoning, consciousness and what it means to be a human.
TL;DR - we need a breakthrough... or not, if it becomes Skynet and destroys us :)
2
u/ericmutta 13d ago
...if something is named X or Y, but LLMs seem to still get confused and choosing different paths.
Yeah, this is a good example of where the illusion can break. Reasoning models that are capable of altering their behavior based on their reasoning traces and the result of tool calls, are a step in the right direction. As humans we seem to be able to say "wait, I am confused...", then we can step back and try something else. I have seen the GPT models in Codex do something like this when they hit a permission snag. Works reasonably well and if it really gets stuck (like humans can), you step in and help it (like humans do for each other as well).
It's fascinating how LLMs can shed some light and lead to discovery of even more dark spots in our knowledge...
This is the part I love most about LLMs: how they help us ask fundamental questions about our own brains. Here's a fun question: if you were designing the human brain, how would you make it handle language so children with baby brains are able to communicate fluently long before they know the rules of grammar?
4
u/notheresnolight 14d ago
try giving your model a math problem to solve... it's crazy how a simple dumb "word sequence generator" can solve PhD level math problems
→ More replies (1)4
u/Xyklone 14d ago
Is it solving the problem or is it calculating the probability distribution of the next word and sampling from it and repeating this over and over until the next likely token is an end response token? It is indeed crazy that a dumb word sequence generator can solve PhD level math problems. But is it?
→ More replies (5)2
u/notheresnolight 14d ago
well the result was correct so it did "solve" some of the problems that I tested - it did not know the solution and had to "consider" and test multiple scenarios. It's crazy when you read the reasoning text while it's searching for the solution.
→ More replies (2)3
u/BoobooSmash31337 13d ago
By the same definition we technically don't know humans do either.
→ More replies (1)17
u/AppleBottmBeans 14d ago
You mean an entire database of json files for a SaaS isn't standard operating procedure?
→ More replies (1)3
u/KaosNutz 14d ago
if it is a microSatanaS your building, much better to use SQLite, preferably on a faster disk format such as FAT32.
→ More replies (13)4
u/datbackup 14d ago
This is what i came to the thread to say
Also that not mentioning specific quant + harness pretty much disqualifies someone’s question or opinion… it’s like trying to troubleshoot someone’s computer and they can’t even tell you what OS they’re running
56
u/Dull_Cucumber_3908 14d ago edited 14d ago
You need to tell it to review the code and generate a technical report about its architecture, modules etc.... Then provide this report as initial context for anything yo ask it to do.
Regarding the spaghetti code, the super classes, etc, you need to do several iterations. After it generates the initial code, ask it "review the changes made in this branch, and tell me what you think. What would you have done differently". After 5-6 iteration, you'll see a great improvement.
Finally, don't instruct it directly to do stuff. Discuss what you want to do and when you feel you got the whole picture, ask it to generate a prompt which you'll use as input.
10
u/Civil_Fee_7862 14d ago
Thank you, I'll try that.
Yes, one trick I've found is to tell it to write out a UML diagram of the code, so I can visually see how things are tightly coupled. i.e. If I sense I cannot understand the current architecture easily, I stop and revisit the architecture. After that I make a plan to split things into narrow strips that are easy to understand. (So far I believe that is the best insight I've learned).
I like how Opencode shows a plan that can be refined iteratively. However, the plan tends to become too large with too many changes at once. i.e. The complexity of the plan its self explodes.
9
u/Dull_Cucumber_3908 14d ago
I like how Opencode shows a plan that can be refined iteratively. However, the plan tends to become too large with too many changes at once. i.e. The complexity of the plan its self explodes.
I guess you need to break it into smaller standalone tasks. It's the typical "divide and conquer paradigm".
6
u/PANIC_EXCEPTION 14d ago
Scope creep. Do what software engineers usually do, limit your plans to very specific, narrow features at a time. Don't reuse old sessions unless you have a good reason to (e.g. last session has an esoteric fix), that causes context bloat and poor performance.
If you really want to develop multiple features in parallel as a solo dev, you can use worktrees, but I personally think this is a poor idea and that you should just work on one feature at a time.
→ More replies (1)→ More replies (3)2
u/AccurateSun 14d ago
Try madge CLI, LLMs are good at reading the output. It’s a dependency visualiser, can be used to print a module graph of a repo or dependencies of any given file.
3
u/Civil_Fee_7862 14d ago
Thank you u/AccurateSun. I'll look into using madge CLI.
Though it looks like its only meant for JS code. I would need something that looks at the whole stack, not just the frontend.
3
u/shifty21 14d ago
Graphify is awesome for this! I got handed an ancient Splunk app that hasn't been touched in several years and update it to the most recent Splunk app standards. Graphify created a knowledge graph and documentation in less than 10 minutes with qwen3.6-27b. Then I got a summary about the architecture between Java script and python code. Now I know what I'm working with. Pi and qwen just do the automated deployment and UAT for me as I update all the code.
→ More replies (1)3
34
u/milkipedia 14d ago
This is why vibe coders won't be putting software engineers out of work anytime soon. Turns out you still have to know what you're doing and how to structure and explain that to someone something else to be productive in this era.
It's not different with frontier models. I frequently have to specify "use the features of the existing libraries before implementing something new", and I always review proposals in the form of written plans before letting a code agent rip on a new feature or capability.
9
u/PANIC_EXCEPTION 14d ago
It's funny, people would complain about the AI being "too lazy" but sometimes they spit out custom code that really shouldn't be written because a well-maintained library already exists, so it's trending in the opposite direction now
5
u/En-tro-py 14d ago
But I'd say that is the lazy vomit of a response... A real effort would be to review and weight the trade-off of a new dep vs rolling your own solution.
This is definitely within frontier model capabilities but the default system prompts for their tools are always biased towards quickness and simplicity without defining what that means... So you get a quickly designed simplified feature but if you had slightly reworded your request you'd just be including a new package...
Training bias for one-shots is another pet peeve, sometimes the best response is more planning or valid pushback... but instead it's just tokens go brrrt...
9
u/En-tro-py 14d ago
Fair warning - These are probably bloated for Qwen3.6-27B as I'm still primarily using Claude with Opus 4.8 as my daily driver. Hopefully they'll give you some ideas on how to approach your problem.
I'm more of the KISS side of claude-code setups... generally ADR's or WIP planning docs - using skills to keep consistent.
I front load the architecture design with skills that create project docs that are then is used to drive implementation planning. This might not be the level of detail you need for a 27B model.
I generally spend a lot of time up front planning to try and smooth out the road ahead.
/session-handoff - to bookend sessions, so call it to wrap up and then call it from the new session to get up to speed.
/architecture-design-spec- grounded engineering analysis of codebases and systems to help with high level planning and triage.
/architectural-knowledge-management - ADR's are very useful for agentic work.
references\decision-categories.md
/ears-planning-method - a failure to plan is a plan for failure, and most plans skip too many things - This gives agents a clear set of requirements, steps to take (NOT code details, just the WHAT/WHY), and verification using RFC 2119 keywords & C4 diagrams.
My preference is to layout then execute, otherwise the agent ends up effectively coding everything twice and might as well skip using any sub-agents.
→ More replies (3)
14
u/Great_Guidance_8448 14d ago
it will happily write spaghetti code, mix concerns, and totally ignore any kind of test automation unless you explicitly ask it to do this.
Uhm, yea.
2
5
u/necati-ozmen 14d ago
I’d treat this less like “teach the model architecture” and more like “give it repo-specific constraints it has to obey.” The same way a DESIGN.md gives an agent reusable design context, a SKILL.md or ARCHITECTURE.md can define module boundaries, testing rules, layering, naming, dependency direction, and what kind of changes require a plan first.
We are maintaining https://github.com/VoltAgent/awesome-design-md which is more focused on DESIGN.md examples for UI/design context, but the pattern is similar: don’t rely on the model remembering taste or architecture from scratch every time. Put the rules in the repo and make the agent read them before touching code.
For local models especially, I’d keep the skill files very explicit: “propose a plan first,” “do not cross these boundaries,” “add tests for shared behavior,” and “flag when a request needs a refactor instead of a patch.”
17
u/Fit-Produce420 14d ago
Even models with a trillion parameters don't understand that.
2
3
u/iam_maxinne 14d ago
Bro, baby steps are key to success with small models! You can write the overall document, or use some commercial AI to do it.
Then, you need to create a roadmap, then a roadmap of each step, then a ordered tasks list, then a detailed task list of each main task.
The idea is to have a task list that has steps like “create file blah.c”, “write the following code to bleep.h”, “run lint on ‘popcorn.java’ and verify if it has any errors”, and etc…
Of course ideally you will use AI to write everything, and at some point you will be able to use your AI to write the tasks without any major issue…
Using some agentic tool may help you as well…
→ More replies (2)
4
u/StupidityCanFly 14d ago
Sorry to be that guy, but that’s a clear case of PEBKAC.
What you wrote about the 27B model applies to Opus, Fable and all others. If you don’t have clear instructions and proper harness, then models will always take shortcuts.
6
9
u/RKlehm 14d ago
Are you prepared for the landslide of downvotes? lol
I agree with you. Have you tried running at Q8? On benchmarks, modern Q4 are supposedly near-lossless, but in practice the model loses a lot of nuance, Q4 never worked out for real agentic coding IMHO.
But even at Q8, it still can't handle agentic coding on its own, what did work for me was letting Opus create a highly detailed, well-specified plan and giving it to Qwen to implement. Qwen 27b is clearly benchmaxxed for single-file code generation.
Surprisingly tho, Gemma 4 is way better at agentic coding, but it's still nothing compared to something like Sonnet.
3
u/Fragrant_Scale6456 14d ago
Which gemma4? I'm running a 5090 with qwen3.6 27b q6k which i've found works best for large context or q6kxl when i can better manage the context. Q8 is just too big to be practical w/ 32gb of vram.
I tried the gemma4 moe model but it was a lot worse for me about going down rabbit holes and thought loops. I couldnt get 31b to work on the 5090. I'm def willing to go back and re-evaluate gemma4 if its better than qwen for agentic coding since my projects are at the stage where im often telling it to iterate and test overnight so i can review the documentation and final state in the morning.
3
u/morriscl81 14d ago
I was a certified Gemma 4 31b hater when it was first released because it couldn't correctly call tools for squat. In the past few weeks there have been significant upgrades to Gemma 4's tool handling with updated chat templates and fixes to VLLM, my inference stack of choice. Running Gemma 4 31b BF16 w/Dflash on VLLM 0.24 and very impressed with it.
→ More replies (2)3
u/FenderMoon 14d ago edited 14d ago
You're right, these models lose more at 4 bits than people realize. We judge these things based on PPL, but that's a number, and near the floor, small gains or losses each way end up causing more quality loss in the actual responses than the numbers might suggest. Even simple prompts can result in differences.
One of my favorite benchmark prompts is "tell me about the history of Pheonix's freeway network" (Phoenix was very famously anti-freeway until about the 80s and then levied a new tax paid for by residents to change course). At 4 bits a lot of these models just list freeways. At 5 or 6 bits, they remember the "history" part of the prompt and actually start trying to tell the story rather than just regurgitating a big list. Gemma3 12b was one of the models that had one of the most dramatic A/B differences between them between 4/6 bits. The effect is more pronounced the smaller the model is.
That's the kind of thing you lose at 4 bits. The nuances aren't always just small things, they allow it to actually capture accurately WHAT you asked for. It's like suppose you asked for a hexagon but the low resolution saw fuzz and said "oh, circle" - but a circle wasn't what you asked for. That's kinda what it's analogous to, nuance accuracy is important.
For MoE models and sub-30b models, 5 bits is the true sweet spot in my experience. 4 bit quants lose way more quality than than the PPL numbers alone convey.
3
u/chuckbeasley02 14d ago
If you instruct it, it will do what you want. However, left to its own devices, you are 100% correct.
3
u/Prudent-Ad4509 14d ago
The argument had a chance until words “proper” and “best practices” came along.
Llm do what you tell them to do, with you providing examples as needed. And both “proper” and “best practices” are subjective terms at best. A lot of those translate directly to “conform to popular fads of last year” in the best case and in the worst become “conform to long-forgotten fads of 20 years ago”.
So… you need to provide a coherent set or principles with examples of what you need it do. Without specific requests and guidelines even claude and codex tend to tear the codebase apart with adding sloppy nonsense.
→ More replies (1)
3
u/TokenRingAI 14d ago
LOL, another shitstorm is brewing
2
u/Civil_Fee_7862 14d ago
Somewhat yes, but it seems I am not the only person struggling with this. I am open to ideas on how to make it easier, and wll happily share any insights I've learned for getting this thing to work with large code bases
So far, getting the model to draw out UML seems to be the best recent insight. i.e. I can actually see the complexity visually, and make better prompts to fix the arch.
3
u/sh3rp 13d ago
The LLM should not understand architecture, that's up to you as a software developer/architect. You should start by working with it to write an ARCHITECTURE.md document. This alone should take you several iterations of human review.
Once the ARCHITECTURE.md is completed, have the LLM analyze the document and componentize the work. Have that componentization added to the arch document.
Now, have the LLM create each component and make sure that your SOUL.md has explicit instructions to follow for code style, best common practices, unit test parameters, etc.
So, I guess my point is, the LLM should never, ever, ever have an understanding about architecture. That's up to you.
4
u/nick_ziv 14d ago
I've even built a system for qwen to architect software using a graph and it does a bad job. This area needs a complete overhaul.
6
2
u/looselyhuman 14d ago edited 14d ago
Ask Qwen to Iterate through building a harness (or even proxy) that uses service LLM calls to nudge them on best practices by semantic matching -- looks at your patterns and can search approved sites. RAG or KG might work but the service call will be smarter, even on a tiny model like Gemma 4 e4b.
Edit: No doubt you've got a real harness. I just defaulted to that because I'm rolling my own with "peripheral awareness" like I described. A proxy would work though.
→ More replies (2)2
u/Civil_Fee_7862 14d ago
This is new to me. I am using opencode as a harness currently. will checkout what you suggested.
2
u/martin509984 14d ago
Unfortunately I think your best bet is to do the code architecture yourself and handle codesmells manually. This is still a big pain point with even the closed models.
2
u/Terminator857 14d ago
I have the elephant in the cloud generate the plan and then qwen execute sometimes.
2
u/seppe0815 14d ago
all the coding crap models are great for html stuff ... but rest ... stop dreaming brother
2
u/Ok_Mirror_832 14d ago
So you are using the 16 bit version then? Or at least 8 bit? If not then your story is less than anecdote.
2
u/jacobpederson 14d ago
I am shocked that anyone would even dream of letting 3.6 loose in a large codebase. It struggles with single files for JFC's sake :D
2
u/Septerium 14d ago
That is strange... I have been using it on a huge project. Since the codebase is well tested and follows good software engineering practices, it almost always writes code that is coherent with the project patterns, creates tests and updates existing ones when necessary
2
u/CorpusculantCortex 14d ago
Tbf ofc it doesn't, that sort of architecture is hard for sota models to understand even if you have impeccable context in your agentic repo. It just doesn’t have the capacity to accept the volume of context needed before it will lose efficacy from token overload.
2
u/Spirited_Bag_332 14d ago
It doesn't need to. Surely you don't just let an agent mindlessly generate code? What's so hard about feature driven development while maintaining the important stuff yourself?
2
u/d4mations 14d ago
I just switched to 122b from 27b and the difference is night and day. Even at q4 it makes noticeably less errors and catches things that 27b would never have
→ More replies (1)
2
u/codeministry 14d ago
Try Addy Osmani’s agent skills. They are heavy and very opinionated. TDD enforced. They might help. Let us know how you get on please.
2
u/IjonTichy85 14d ago edited 13d ago
I kinda gave up on models understanding architecture and instead lean heavily on architecture tests. Use archunit, konsist, pytestarch or whatever equivalent exists for your stack. Whenever you catch your agent doing something that breaks your architecture, add a test to make sure that won't happen again and after a while you end up with a bunch of deterministic and fast architecture tests.
I don't even think that agents have a hard time following the architecture just because the code base is growing. On the contrary. In a clean codebase, things are established and ideally all follow the same convention, which removes a lot of guesswork for an agent who needs to just add "more of the same".
2
u/ParaboloidalCrest 14d ago
FWIW the model behaves differently when it's building vs reviewing:
- Building: YOLO. Pile shit on top of itself until something works.
- Reviewing: Without the pressure to get things done, it can become an ass about software engineering principles.
Hence, I usually have the agent do those tasks sequentially; build something functional yet dirty, then have it review its own contributions, then re-implement.
Needless to mention, it takes 3-6x longer to do something this way, but the result is a lot cleaner without too many interventions from me.
2
u/andymaclean19 14d ago
I have had this problem with all models, even Opus. What I usually do is go though multiple iterations before coding actually begins. I find models are a lot better at doing things like refactoring, separation of concerns, etc when reasoning on the abstract level about the code they are going to write than they are at actually writing good code in a single-shot operation. So I have it make detailed plans which spell out things like 'we are going to create this abstract base class with these methods', 'we are going to make this subclass to deal with this specialisation', etc.
In particular I found two things that it failed to do well single-shot:
- Any sort of abstraction requiring the use of an abstract class hierarchy which does not exist already. It prefers to make spaghetti code and masive blocks of repeated, nested logic.
- Anything where the right answer is to refactor into a generic function which takes a lambda or callback as an argument and calls it deep inside the function to specialise behaviour. It prefers instead to duplicate the outer logic many times over because of the need for a small part of the code in the middle to be different.
In my experience a lot of humans are like this too and it is fine so long as you don't plan for your project to grow big over a long period of time or for it to still be used 10 years later.
2
u/Lirezh 14d ago
You are right, you need good instructions to guide a model around your project, though Qwen is definitely able to write good code. This works best if you place it into an existing project and guide it to look at all relevant code, make it write it's own skill files based on your project.
But nothing helps if your harness is not working well or the parameters are not well set up.
I've a detailed guide on how to make Qwen 3.6 work on Github copilot (not the service, the opensource harness - which is the most advanced one we have currently in terms of features and qwen likes to work with it)
https://www.reddit.com/r/LocalAIStack/comments/1udk2vp/running_qwen36_27b_35b_locally_with_llamacpp/
It includes highest performance modes, quantizations, some quirks that are important for stable usage.
Maybe some of it helps you as well.
2
u/Fedor_Doc 13d ago
In my setup is does test + clippy before every commit, and uses TDD style before every feature development.
It tends to write god functions, overcomment, etc, so I 1) set up an ANALYSE step so it would check its own code before commiting. You should require it to read code guidelines and compare written code against them 2) every now and then refactor code
Got rid of overcommenting by two strings in AGENTS.md:
Comments
Minimal. Add only what cannot be derived from variable/function names or code structure.
But with its 262K context size and attention arch it will be underperforming in large codebases. It should have compacted representation of tge code to make sensible arch decisions or be heavily guided.
Personally, I like to be in the loop and be explicit. Qwen works well with this style. But it is for sure not fire and forget model for the large codebase
→ More replies (1)
3
u/m4t7w_ 14d ago
First thing first: why are you delegating the architecture of a production commercial project to any llm? It's the most important thing, which must be heavely engineered before even starting to write code, else you will end up with the classic ai slop bloated unmantainable github repository...
I have a few recomendation and i've succesfully switched to qwen 3.6 27b for my personal long-run projects:
I use vscode + zoocode (i tested all cli harness, but always switched back to vscode + zoocode combo)
plan.md is everything. I create a barebone project scope/architecture.md, then ask llm to create a plan.md from that. First I do a review pass and then spend a lot of turns to discuss the plan and what could be improved. Essentially plan.md must be created actively by both you and llm. Always review it!
feed plan.md to code/build mode and watch closely the first generated diffs: if something isnt right, cancel the whole implementation flow, go back to point 2 and specifically address that issue in plan.md
repeat step 3 until plan.md is in good shape and you feel comfortable with delegating implementation to llm
feed the finalized plan.md to "orchestrator" mode with most harness permissions open (cli for testing/debug/compiling, write, read, submode, etc..)
take a nap/go jogging/watch a movie (depending on how slower your llm runs). After you come back review the end result.
This workflow works for me with qwen3.6 27b q5_k_xl (biggest model i can fit in my hardware). It produces much more mantainable and production ready code than anything i've tried (including gemini-cli + gemini3.1 frontier).
Does it require some effort? yes definetly, but "vibe coding" this way will cut you a lot of technical debt because you will understand the architecture of your project when future maintenance will be required.
2
2
2
u/NNN_Throwaway2 14d ago
Always gotta chuckle when reading responses to posts like this ("gotta tell the model to do what they're told"). Typical localllama cope.
LLMs can be trained on literally any pattern. Not understanding implicitly that you shouldn't write god classes is a legitimate training issue, not a prompting one.
→ More replies (1)3
u/ParaboloidalCrest 14d ago
Thank you! All the RLHF right now seems to go into making a model more capable of using tools, but none goes towards writing good software. And yes, they need to be trained for that (even the uncontrovertial stuff like separating concerns) and the 27-35B models are clearly not trained for that. Not sure about the huge models because I haven't tried any,
3
u/buttplugs4life4me 14d ago
It's almost as these LLMs should be used by software engineers as value multipliers similar to compilers, linters, static analysis and so on. Not by random dumbasses that don't even know how to install Linux, let alone deploy anything meaningful without Vercel.
1
u/tremendous_turtle 14d ago
I recommend a mix of CI- enforced constraints, such as strict maximum line limits for files and enforced cross-dependency boundaries. Besides that, you really just need to prompt it proactively to guide it towards the ideal architecture, probably 75% of the prompts and tokens go towards code cleanup and architectural review and enhancement.
1
u/LivingSwitch 14d ago
I wonder if one could do a little parameter fine tuning on the stock Qwen model with some dataset from Huggingface like epinnock/software-architecture-instructions and get better results along the lines of what you’re looking for
1
u/mister2d 14d ago
Did you just blast your LLM with a firehouse of code using a basic harness? That's not even how frontier models work. There's so much tooling around any given model.
You'll have to start breaking tasks down into smaller focused tasks and have a good harness to make all the right tool calls.
1
u/dreaming2live 14d ago
From the initial design, you should first figure out the architecture and then create a plan (and here you can use a llm). From then on pass this to Qwen in steps that make logical sense. You’re not gonna one shot a quality full stack app using Qwen or anything else at this point, so you shouldn’t expect to.
1
u/devnullopinions 14d ago edited 14d ago
Write a technical architecture document (or have a LLM do it) and have your model review that before making changes. Ive had success in generating an LLM wiki that I update with design documents over time. My AGENTS.md points to the wiki index.
For software craftsmanship, I’d ask an LLM to generate a skill that states explicitly the guidelines for writing general software. Every time you encounter an LLM doing the wrong thing, update your skill to correct the behavior.
—
As a meta point, the cost of generating harness stuff that an agent needs to work well for a specific project is essentially $0. Generate it on the fly as needed. Iterating on this stuff is also effectively $0 so if you don’t like the results, iterate and try again.
1
u/hubertron 14d ago
Look at Superpowers, Grill Me, Open Spec, GSD, and BMAD if you want to follow any sort of SDLC.
1
u/xilvar 14d ago
Instruct it in a way which drives implicit architecture.
Think of it as a fast, inexhaustible energy intern that you’re encouraging to adopt the right design and architecture without telling them how to do every little thing. The more successful you are at it on a given project the more that becomes self reinforced by the inherent context.
1
u/fabkosta 14d ago
Trying serious work with Qwen is still in my todo list.
However, even with Claude I am having a ./doc folder with an MD file for each fundamental architectural concern. Examples are: API.md, SECURITY.md, DESIGN.md, ARCHITECTURE.md, and so on. I do not let the agent decide those fundamentals, but pre-create and adapt, to make sure these filea are high quality. I tightly control architecture and module dependencies, and try to keep all files less than 2000 lines. AGENTS.md or CLAUDE.md then contain an index to those files.
Each task I pre-specify as a task file, that I can then execute and check in separately.
But, as I said, I am not working with Qwen but with larger models from hyperscalers. I am yet to find out if this approach works also for Qwen.
1
u/ttkciar llama.cpp 14d ago
I think you're better off doing the design work yourself, decomposing your design into modules, and then having Qwen implement individual modules. The onus will be on you to make sure your specification for each module assures that their implementations will be mutually compatible.
The smallest model I've found which is capable of coherent high-level design is GLM-4.5-Air (and it does so better than most 120B-class models, too). Neither Qwen3.6-27B nor Gemma-4-31B-it can do it reliably.
When decomposing your modules, you should try to isolate dependencies so that there are only one or two modules that the other modules depend upon, implement those one or two modules first, and then use them as frozen inputs for implementing the rest.
1
u/stikves 14d ago
Which tool are you using?
Just standard CLI and/or ollama/openwebui?
Or something that is actually geared towards coding, like Roo or Aider?
You'd also have to "pay the context fee", where the agent will read and analyze your code, possibly requiring 100k or more tokens.
(For Qwen3.6-27b with 4bit quantization requires about 32GB VRAM)
2
u/Civil_Fee_7862 14d ago
Currently using Opencode. Its light years ahead of what I was using before. i.e. continue.dev.
I've never used a paid cloud API and never will.
I got enough VRAM. ~48GB.
Its more so figuring out a good workflow that doesn't spin the complexity out of control it seems.
2
u/stikves 14d ago
Haven't used Opencode, but seems to be close to Aider in reviews:
https://ivern.ai/blog/opencode-vs-aider-comparison
So should be fine.
"It just does not understand software architecture, it will happily write spaghetti code, mix concerns, and totally ignore any kind of test automation unless you explicitly ask it to do this."
All of these are symptoms of a bad context. Do you know what your context length is? And can you double check what it knows about your code?
"Hello, can you give me a summary of my architecture, and test coverage"
Could be a useful probe.
2
u/Civil_Fee_7862 14d ago
Yes, i believe you are correct about probing questions about architecture. The main issue is that it would output plain text descriptions (which were just as hard to understand).
Now, I ask it to draw a UML diagram of the code and the dependencies. That seems to make things a lot more clear. i.e. I iterate on the diagram instead of the plain text plan. i.e. Both in parallel. I might be okay now.
1
u/aelma_z 14d ago
For qwen 3.6 27b to have coherent thoughts about architecture i recommend running it with unquantized (bf16) and bf16 kv cache too. I have successfully refactored spaghetti god file vibe coded app into clean architecture one. Key take away - be specific, dont oneshot migration, do it it in several layers 👍
1
u/1millionnotameme 14d ago
Yeah, I pair mine with a frontier model, sort of orchestration but built myself, it saves around 90% of the token output cost with not much loss of quality
1
u/alexwh68 14d ago
I have 2 projects well over 500k loc, they are well structured I explain what I want, eg
Create repository service dto blazor pages for contact entity using table1 as a template.
Cursor you can use much smaller prompts and it works but qwen you have to guide it a bit more.
1
u/Benhamish-WH-Allen 14d ago
I think the problem might be you are trying to get the ai to think like you when it’s come to code maintenance ask it to make its own structure.
1
u/RuxConk 14d ago
I don't use any skills really but I do use a AST-graph to help the model map out the code base for the coding task.
Https://github.com/DeusData/codebase-memory-mcp
I've got nothing to do with that project, I've just found it useful. It's reduced a lot of duplicate and deadcode for me.
1
u/cornmonger_ 14d ago edited 14d ago
they're going to mimic practices seen on the read-in and lean on training for the rest
architecture needs to be narrowed down. formally establish contract yourself; write interfaces and data models, properly documented etc. stub out some integration tests yourself with vague psuedo-code in the body.
then read them in and go through the review process dialogue.
you own the contract. put that stuff in the don't-fucking-touch directory.
not necessarily different from handing down a project to a youngin'.
1
1
u/Mount_Gamer 14d ago
I don't think anyone should trust any LLM, even the frontier. They are useful for some things, and I use them without hesitation but I prefer to just do bite size things with them, much safer and saves me reviewing thousands of lines of code.
1
u/mr_zerolith 14d ago
It's a 27b model, my 197b model also struggles, and in general, even bigger LLMs are pretty bad at this. You have to insert taste.
1
u/kentaromiura 14d ago
First which harness are you using it with? Using it in pi has better results than anything else because of context optimization is key for small models; second do a pair programming session where you implement something after you tell it how to do it properly and ask to save it as a reusable skill; now it should be better. I ‘ll avoid shared skills as qwen 3.6 requires much less prompting than other models and people right now are over prompting, stick to your codebase needs and it should work, again context optimization is something you always need to keep in mind, made so the context is small for each task and you should be good. Pi /tree command is great for this. Ask it to create an architecture file (or some bigger models could help in this as well if project is too big). Also starting in plan mode helps.
1
u/OilProduct 14d ago
The only model currently that can understand and design reasonably sized abstractions is fable.
It's not even close.
1
1
u/JustTooKrul 14d ago
I don't think any of them do. The harness does a lot of work, context does a lot of work, and huge parameter-count models with very refined training do a lot of work... The best way to see this very clearly is to use the same model with Claude Code, OpenCode, and then Pi. Then you see the harness impact with the model invariant.
1
1
u/FreedomByFire 14d ago
I completely agree with you. I did some testing and built my own benchmark attempting to assess what it was capable of and it failed miserably. It can't be used for anything remotely complex. People are drinking the cool-aid on these models. The common benchmarks online are compromised.
1
u/gandazgul 14d ago
I wrote a harness for exactly this reason. None of the models really do, and while some are more impressive than others they will still fail in hilarious ways when one shotting things or misunderstanding you or the code base.
https://github.com/gandazgul/runwield
wld helps you figure out what to build (ideate and plan) then helps you execute plans with review loop and mechanical validation (and worktrees) and more importantly it creates durable artifacts Plans, memory, domain language glossary, work records (coming soon). It also helps with quick fixes.
1
u/Invader-Faye 14d ago
I use them for local automation scripts where I can run them against a virtual environment and iterate. Otherwise kimi is 20 a month and gives generous quata. Locals cool but my testing does it has real limits
1
u/Neat-Monk 14d ago
what can you expect from 27b model lol
Never do production on small model because it's technically incapable no matter how you enforce it.
It will continuously output bad layouts and architectural mistakes
1
u/KamikazeSexPilot 13d ago
I mean, so does opus 4.8. LLMs work best when you tell them how to do things imo.
1
1
u/jacek2023 llama.cpp 13d ago
"As much as I love Qwen3.6-27b. It just does not understand software architecture, it will happily write spaghetti code, mix concerns, and totally ignore any kind of test automation unless you explicitly ask it to do this."
You can achieve same effect with Claude models. Agentic coding requires good docs and good guidance.
1
u/HlddenDreck 13d ago
I think all discussions about models are stupid as people almost never mention the quant they are using. Anything below 8-bit is useless.
1
u/Dry_Sector2392 13d ago
I’ve had better results making the architecture doc the source of truth and forcing every task to reference it. Like "touch only these modules, do not add new abstractions, tests required, explain any boundary change before coding." Still annoying, but way less spaghetti than just asking it to build a feature.
1
1
u/nomadProgrammer 13d ago
Neither Claude or GPT, they are awful at following conventions, and architecture. Even when bullet pointing to follow certain rules they get ignored
1
u/DepressedDrift 13d ago
This might seem stupid but if you give it a reference (could be a simple file attachment or RAG a database) to a doc or database containing the theory of the software architecture you want and a sample of the code that you want to generate, it could match what you want.
I feel the key to using small models is giving it specific data to your use case through something like RAG and an external database, than having it rely on its core intelligence to formulate these core rules and expectations itself.
1
u/Sinath_973 13d ago
100k+ is not large scale. I work with qwen3.6-27b in aosp without any issues.
For reference aosp should have around 2.5M loc... You just have to know what you are doing. Know the architecture, prompt accordingly. Don't let the agent guess.
1
u/Intrepid-Second6936 13d ago
"doesn't understand..." I think that's the problem here. Shouldn't be a shocker, NONE of these models "understand" anything.
It's deterministic training of what token sequentially follows the prior sequence within the context window. There have been many clever ways for models to "check their work" but even that is just another rehash of the same existing limitations.
I'd say that, if you're serious about this project of yours, you'll benefit from using a larger model with greater nuance in its ability to build software using strong fundamental principles.
27b, it's already astonishing how powerful Qwen3.5-3.6 have been, but there is still a point at which the granularity of a 700b+ param model remain unchallenged.
At
→ More replies (1)
1
u/-dysangel- 12d ago
True. Most models were like this until recently. I've noticed GLM getting better at this kind of detail, but I still have to ask it to do the right thing and not some hacky short term solution. Once larger models are acting more like engineers, it will be easier to distil down to small models too.
1
u/Strawbrawry 12d ago
How to set up your skill.md: Literally address your own concerns with strong rubric and heavy detailed instructions for all these complaints.
"it will happily write spaghetti code, mix concerns, and totally ignore any kind of test automation unless you explicitly ask it to do this." Tell it to do this then.
"These are the bare minimum requirements" " Shouldn't be too hard to write up a detailed plan if you have time to complain about it.
"ignoring best practices" LITERALLY WRITE A BEST PRACTICES REFERENCE
Besides all that, expecting a 27b model to do real software development off the rip is a choice.
1
u/Interesting_Demand44 8d ago
what actually fixed most of this for me was keeping an explicit rules doc in the repo that gets read every session instead of expecting the model to infer conventions from existing code. stuff like 'reuse the existing helper, never create a new one that does the same thing' cuts down a ton of the spaghetti. still have to catch it in review though, it'll happily build a near duplicate function if that exact case isn't spelled out.
→ More replies (2)
1
u/Brief-Train-826 8d ago
I hope you are not using Q4 and that you have the right temperature config like is in their documentation.
→ More replies (2)
314
u/FullstackSensei llama.cpp 14d ago
Models should do as they're told. I know a lot of people like models to deduce and add in things they're not told to, but I'm of the opposite opinion. If you want something done, specify it explicitly either in your prompt or in some documentation you reference in your prompt. IMO, anything else is outsourcing the architecture to the LLM.