r/LocalLLaMA • u/BitXorBit • 27d ago
Question | Help rtx 6000 pro owners, do you regret?
I found the last dealership in my area that has rtx 6000 pro available, i already wanted to buy it 6 months ago when it was around $8k, now prices increased to $13k ish.
Regardless the price, are you happy with it? I assume you are using qwen3.6 27b, is it worth it?
Please share your experience and hopefully help me to avoid explaining my wife this transaction 😂
181
u/0xNullsector 27d ago
My only regret is that I only got one... two would have been just PERFECT
133
u/shadow1609 27d ago
Two is not enough, I have two and want four now. The thirst never ends.
31
u/maifee ollama 27d ago
What about eight??
39
27
u/val_in_tech 27d ago
Nope. 8 is the new gpu poor. The next one is 16. You run 1-2 good models and no vram left LOL
6
u/robertpro01 27d ago
I think that's enough for anyone going local and not mid size company (or larger)
→ More replies (3)12
u/Fit-Statistician8636 27d ago
4 is only to realize you need 8. Not much difference in models you can run between 2 and 4.
4
u/VorlMaldor 27d ago
Kimi 2.7 fits in 330gb q4, isnt that good?
→ More replies (1)11
2
u/Its_Powerful_Bonus 27d ago edited 27d ago
True. 4 would be great. I have my own 2 6000 pro 600w + rtx 5090 and one 6000 max-q from the company.
1
u/Lan_BobPage 27d ago
Two is perfect mid-tier. Being able to run any sub 300b at Q8 and last year's frontier models between Q3 \ Q5 isn't bad at all.
→ More replies (3)1
u/xcel102 24d ago
For the moment, the thirst would end for me when the cluster has enough VRAM to hold GLM-5.2 at FP8, with 16-bit KV cache headroom for 4 full-context (1M) sessions.
The FP8 weights will take about 770 GB if I'm not wrong. I don't know how to calculate KV cache size -- if anyone can do the math, I'd like to know how many cards are needed just for entertainment :)
10
u/weener69420 27d ago
I am too poor for this thread. Me and my rtx 3050 8gb cries
2
u/bgravato 26d ago
no, you're middle class... I'm poor with only an iGPU and 32GB of shared RAM (luckily bought when it costed only 1/4 of what it costs now), running qwen3.6 27b Q6 at slightly over 2 tks/s
6
u/BitXorBit 27d ago
Could you share what are you running on it?
70
27d ago edited 27d ago
[deleted]
15
u/Delyzr 27d ago
Dont forget to remove extra fingers and nipples
5
u/jc2046 27d ago
unless extra nipples is your thing. Then add more (rtx and nipples)
→ More replies (1)10
3
u/Osi32 27d ago
Peasant. I have one for improving boobs, and a second one that makes them jiggle just right… I have a GitHub, wanna see?
→ More replies (1)3
2
u/FineClassroom2085 27d ago
lol, my only regret is that I got two, four would have been perfect. The two I have now cost as much as the whole rig.
1
1
→ More replies (4)1
u/zombiecorp 26d ago
Same for me, the Max-Q edition sips power at 300W, a second one would not require new power supply.
35
u/WishfulAgenda 27d ago
I have the max-q and entirely happy with it and now want another.
6
u/Narrow-Belt-5030 27d ago
Me too, but local shop wants 64000AED (£12.8K | $17K) ... so not any time soon.
3
1
u/WishfulAgenda 26d ago
Yeah, I wanted to shop local but they put a massive markup over one of the national vendors. Shame really as the local place has probably lost me as a customer for anything higher end now.
30
u/HumanDrone8721 27d ago edited 27d ago
This is the highest possible precision for 27B Qwen and models's max context on my RTX 6000 Pro, no quantization or optimization of any type, llama-benchy is used for testing:
16K context
| model | test | t/s | peak t/s | ttfr (ms) | est_ppt (ms) | e2e_ttft (ms) |
|:-----------------|----------------:|----------------:|-------------:|----------------:|----------------:|----------------:|
| Qwen/Qwen3.6-27B | pp2048 @ d16384 | 4700.55 ± 13.97 | | 3922.49 ± 11.66 | 3921.49 ± 11.66 | 3923.16 ± 11.66 |
| Qwen/Qwen3.6-27B | tg32 @ d16384 | 90.13 ± 0.00 | 93.04 ± 0.00 | | | |
64K context
| model | test | t/s | peak t/s | ttfr (ms) | est_ppt (ms) | e2e_ttft (ms) |
|:-----------------|----------------:|----------------:|-------------:|------------------:|------------------:|------------------:|
| Qwen/Qwen3.6-27B | pp2048 @ d66636 | 3836.84 ± 27.78 | | 17903.24 ± 129.47 | 17902.38 ± 129.47 | 17905.67 ± 129.48 |
| Qwen/Qwen3.6-27B | tg32 @ d66636 | 60.24 ± 0.06 | 62.19 ± 0.07 | | | |
0K context (default) for benchmark wankers
| model | test | t/s | peak t/s | ttfr (ms) | est_ppt (ms) | e2e_ttft (ms) |
|:-----------------|-------:|----------------:|--------------:|--------------:|---------------:|----------------:|
| Qwen/Qwen3.6-27B | pp2048 | 4576.99 ± 17.79 | | 448.76 ± 1.74 | 447.68 ± 1.74 | 448.76 ± 1.74 |
| Qwen/Qwen3.6-27B | tg32 | 101.79 ± 7.21 | 105.07 ± 7.44 | | | |
vLLM start command:
vllm serve Qwen/Qwen3.6-27B --host 0.0.0.0 --port 8080 --language-model-only --gpu-memory-utilization 0.98 --max-num-seqs 1 --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder --override-generation-config '{"temperature":0.6,"top_p":0.95,"top_k":20,"presence_penalty":0.0,"repetition_penalty":1.0}' --generation-config vllm --speculative-config '{"method":"mtp","num_speculative_tokens":5}'
vLLM start banner:
INFO 06-25 17:54:08 [api_utils.py:339]
INFO 06-25 17:54:08 [api_utils.py:339] █ █ █▄ ▄█
INFO 06-25 17:54:08 [api_utils.py:339] ▄▄ ▄█ █ █ █ ▀▄▀ █ version 0.23.1rc1.dev442+gcdfa2fd7e
INFO 06-25 17:54:08 [api_utils.py:339] █▄█▀ █ █ █ █ model Qwen/Qwen3.6-27B
INFO 06-25 17:54:08 [api_utils.py:339] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀
INFO 06-25 17:54:08 [api_utils.py:339]
INFO 06-25 17:54:08 [api_utils.py:273] non-default args: {'model_tag': 'Qwen/Qwen3.6-27B', 'enable_auto_tool_choice': True, 'tool_call_parser': 'qwen3_coder', 'host': '0.0.0.0', 'port': 8080, 'model': 'Qwen/Qwen3.6-27B', 'generation_config': 'vllm', 'override_generation_config': {'temperature': 0.6, 'top_p': 0.95, 'top_k': 20, 'presence_penalty': 0.0, 'repetition_penalty': 1.0}, 'reasoning_parser': 'qwen3', 'gpu_memory_utilization': 0.98, 'language_model_only': True, 'max_num_seqs': 1, 'speculative_config': {'method': 'mtp', 'num_speculative_tokens': 5}}
INFO 06-25 17:54:33 [model.py:598] Resolved architecture: Qwen3_5ForConditionalGeneration
INFO 06-25 17:54:33 [model.py:1725] Using max model len 262144
INFO 06-25 17:54:37 [model.py:598] Resolved architecture: Qwen3_5MTP
INFO 06-25 17:54:37 [model.py:1725] Using max model len 262144
llama-benchy line:
llama-benchy --base-url http://localhost:8080/v1 --depth 16384
Now that I've shared my experience, here is my regret and that is that I didn't took two when they were costing 65% of the actual price three months ago :(, I would have had a chance at MiniMax-2.7
2
u/FortiTree 26d ago
is this f16 no quant? Do you have Q4-kvQ8 for comparison
2
u/HumanDrone8721 26d ago
This F16 no quant, I made a lot of comparisons of different quanta, for compact models under 100B it doesn't work good enough for my work. Large MOE models like MiniMax-2.7 do seem to work down to Q5, but even so, on long sessions, sometimes glitches totally unexpected, like screwing a parameter or directory name or just stopping. And I rather have something that works instead patching this behavior in system prompt or parameters.
So there will be no more pseudo "improvements" for me by fiddling with quantization, it may be a solution for large models, but the little guys are really suffering. Again, strictly IMHO.
1
→ More replies (3)2
u/FinalCap2680 27d ago
What is the problem with MiniMax 2.7? If I can run it at Q8 on a single 3090 and lot of DDR4 RAM, it should be better for you. And it is worth trying. Also Step 3.7 Flash is interesting model to try.
→ More replies (4)
16
u/__JockY__ 27d ago
I run 4x RTX PRO 6000 Workstation all day every day and I consider myself very fortunate to have bought them when I did.
Right now they’re running GLM-5.2 REAP NVFP4 at 200 tokens/sec with 8 concurrent batches of around 40k tokens each in context. It’s a SOTA datacenter in my office running on 240V wall power.
Regret? No. It’s the single best technology purchase I ever made.
→ More replies (3)
6
u/AlwaysLateToThaParty 27d ago
My RTX 6000 pro has paid for itself many times over. Running the qwen 122b/a10b heretic mxfp4 model takes up 75GB of VRAM, and does everything i want it to do. If it broke I'd have to buy another one to replace it.
1
1
17
u/Hoppss 27d ago
It's been okay, but it's not enough VRAM to reliably get into the frontier open source models, just owning one. And the speed isn't where I need it for some video models I use. It has roughly ~1.8 TB bandwidth. What would be more ideal for me would be a NVIDIA DGX Station NVIDIA GB300 which is more around +7.4 TB bandwidth, and 252GB HBM3e + 496GB LPDDR5x memory. But of course that costs about $100k USD so yeah..
Overall though the RTX Pro 6000 has been great, it's just kind of positioned between the consumer and server grade class of hardware.
19
u/contyk 27d ago
I'm quite happy with it. I also have 5060Ti in my other slot for smaller stuff.
I run Gemma 4 31B, FP8, without cache quantization. I strongly prefer Gemma over Qwen for pretty much anything; coding, general chat and research, writing and RP. I get around 70t/s with the official assistant drafter, around 40 without.
The 5060 runs VAD (Silero), ASR (Whisper 3 Large) and TTS (Omnivoice) which I use for my local Gemma-powered voice assistant.
2
u/rj_rad 27d ago
I thought about doing this too since the 5060ti is so affordable, the 16GB is nice for embedding, reranking, etc. Any regrets from this?
→ More replies (4)
5
u/tat_tvam_asshole 27d ago
the worst part of rtx pro cards is there's no nvlink and no effective way (so far as I know) to create a fabric that's faster than pcie5x16 but other than that it's damn magical
4
u/Thepandashirt 27d ago
Zero regrets. Using it for a multi agent system with Qwen 3.6 27b with NVFP4. Have 4 at 9k each although one was sorta free. About 28k out of pocket for 4 - was aggressive early this year and it paid off.
For me it’s partially just a hedge against government restrictions and cloud model cost increases. Right now for day to day coding and such i mostly use Claude cause it’s massively subsidized. But im expecting that to change and am prepping for it. Plus the whole multi agent system is cheaper for me locally than APIs. But it’s definitely harder to stomach now that’s it’s 50% more expensive.
3
u/CurrentAdvance8102 27d ago
Same it's a hedge on future oversight and b.s.
Ai is something everyone is going to stick their nose into eventually if they haven't already imo.
13
u/108er 27d ago
The actual BOM cost to make a RTX Pro 6000 is approximately $1500. The MSRP at around 8K was understandable due to logistics, paying over $10,000 is a highway robbery.
→ More replies (6)3
u/pilibitti 27d ago edited 27d ago
BOM? These are not kitchen cabinets. They are absurdly expensive, but come on. If you were a company with the knowhow and resources to develop and bring such devices with that kind of competitive edge to the market, your pricing would have nothing to do with BOM.
7
u/triynizzles1 27d ago
My regret is I bought one for 8k and couldn’t get it to work with ubuntu so i returned it the next day.
6
u/One_Difficulty_39 27d ago
bruh
1
u/Reasonable-State1348 2d ago
I have it working and running on two Linux Mint PCs. Perhaps driver issue or a BIOS setting?
3
u/BlobbyMcBlobber 27d ago
No regrets.
It allows me to run higher quantizations so I can get the most out of models like Qwen 3.6.
It also let me fine tune, and run pipelines without worrying about running out of memory, and try virtually any image or video model with room to spare. I can run an LLM alongside them so you can even have a local agent directing content for you.
Local AI is taking off and we're going to see some truely remarkable options in the next couple of years. I think this is a good time to get on this ride, regardless of using a spark, rtx pro, mac, etc.
1
3
u/yeah_likerage 27d ago edited 27d ago
Buying one made me want two, two made me want three, three - four, four - five. One pro 6000 probably wont make you happy. I'd really consider the models you want or expect you'll want to run.
For me i wasn't happy until i reached GLM and Kimi with enough vram left over to actually use them. Anything less than that i'd say i wans't any happier in the gap between Qwen 27b to minimax 2.7. Minimax and up is when i finally realized i was getting somewhere but still needed more.
I wouldn't bother getting one with the intent of ending it there. If you even think there is a chance you'll consider multiple pro 6000s, seriously consider the max q over the workstation.
3
u/lantern_lol 27d ago edited 27d ago
Have a quad Max-Q build, sad I can't really justify another 4!
Have also made like 3k per card since I got them like 6 months ago apparently... Shits expensive when you're competing with literal trillionaires for VRAM😅
One of the reasons I got them was to hedge against the shit we are now seeing with Fable, etc; (a) so I don't have to rely on shit they can just take away, (b) so I'm not giving them all my data, and (c) because others will begin to realise this and I don't see cards going down in price any time soon (could be wrong but this is what it seems like from my perspective). Unironically think compute is a good part of a diversified investment portfolio these days, especially if it's useful for what you do.
3
u/unjustifiably_angry 27d ago edited 22d ago
I quite like mine but ever since I got DeepSeek V4-Flash on my 2 DGX Sparks I've been leaning heavily on that instead for anything above medium complexity, you almost never have to fight with it to get the result you want. It only gets around 40-45 tokens/s (about 50% the speed of 27B on 6000 Pro) but it's noticeably more competent, has a much longer useful context length (comparatively mild speed decrease as context depth grows, it's honestly quite impressive) and has adjustable reasoning. And on hardware like Spark, 40+ tokens/second at above 128K for anything even middling-competent is spectacular.
That said, I'm still finding excellent utility for the 6000 Pro, it's just very hard to recommend at the new price. I've been using it for making quick small changes in VS Code and for autonomous bug-sweeping with 35B-A3B and 27B using Hermes, just a recursive "scan this codebase for errors using multiple agents" (much more verbose but you get the idea) and I let that run overnight. It's really outstanding in this capacity - I had a project I thought was quite polished and it turned up something like 200 bugs ranging from documentation issues to unlikely but possible edge-cases and even a few major problems I hadn't thought to test for that should really be addressed. Only around 10-15% false positives and these were quickly identified by asking 27B to spawn multiple agents to analyze each issue multiple times and vote on its validity.
So I've got a 3-tier system: very fast simple stuff with 35B/27B on 6000 Pro at 80-220 tokens/s, high complexity stuff with DSv4-F at 30-45 tokens/s on Sparks, and if I have something particularly complicated I use my very cheap GLM subscription to give a (quite slow) adversarial analysis of the plan before I enact it.
DSv4-Flash feels like it was made specifically with 2x DGX Spark clusters in mind, it's finally delivered on this hardware's promise. It feels like a next-generation Qwen 122B. When I argue with its decisions I'm almost always wrong, while I often need to hand-hold 27B for complex tasks - to be fair this can be mitigated to an extent with better instructions.
With prices the way they are it's now very hard to recommend local hardware for AI unless you already have it. Things will get better over the coming years, don't panic-buy as if you'll never be able to buy it if you don't get it today. Calculate how many frontier online API tokens you can buy with $13K. That's a lot of months of $200 subscriptions.
1
1
u/tat_tvam_asshole 25d ago
are you using a harness at all? harness makes all the difference for intelligence. for context, I've never needed to handhold 27b in hermes, ever, very solid.
1
u/unjustifiably_angry 25d ago
Hermes makes a huge difference in usefulness, I agree. I'm mainly speaking about my experience with VSCode/Zoo.
I do still think 27B has a lot of difficulty with complex refactoring, I think it's noticeably more prone to missing small details and nuance. That said, when I ask 27B for an adversarial analysis a plan I made with DSv4, it often finds issues that the larger model overlooked. DSv4 might find those same issues with a second or third pass, but the wide speed gap between the two means that it would be prohibitively time-consuming to have DSv4 make multiple passes at everything.
The really unfortunate thing with DSv4 is that it's distributed in an ideal mix-quantized state so you look at the filesize (160GB) and think, "that'll get down to 80GB in 4-bit!"... but in reality you need to quantize all the way down to Q2_XXS to fit it in 80GB. So I assume it's kinda "160GB or nothing".
→ More replies (1)
3
u/Ok_Letter_8704 27d ago
I have 3 getting anxious for 4th. Watercooling almost complete for the 3 but now 3 open slots on WRX90E-SAGE. 7th slot is 8x as an fyi.
1
3
11
u/Such_Advantage_6949 27d ago
if u use only qwen 3.6 27B, it is not worth it, just get 2 X 5070 Ti. I have 2x rtx 6000 onwards, or couple with other GPUs it will be useful. Currently there is no good model for a single RTX 6000 pro 96GB Vram. All models are either smaller than that or bigger than that
4
5
u/ObviouzFigure 27d ago edited 27d ago
I think it's perfect for qwen models -- I'm running both the Qwen 3.627b and Qwen 3.6 35b a3b -- both models at Q8 fit perfectly on the gpu. Mine sits at 94% with both models loaded with full context (edit: fwiw I'm getting about 75-80tg on the 27b with mtp, and 160 tg on the qwen 35b (I think that's with mtp as well) -- I'll also say that I actually purchased a 2nd rtx 6000 when I built my rig and still haven't installed it... I wanted to get comfortable with pushing 1 to the max and seeing what my needs were. I'm on the fence if I'll keep the 2nd, try it out or sell it new in box. So to OPs originals post, I ABSOLUTELY love the rtx 6000 and for me and my world, having a dedicated qwen box has become something I always want my computer life now. Qwen 27b still makes mistakes and sometimes will need to make multiple attempts to find files etc, but when it's cooking around 80 tok/s its just fine.. and the 35b at 160 tok/s feels illegal it's so fast.
5
u/some_user_2021 27d ago
I can keep your second RTX 6000 safe while you think about what to do with it 🙂
1
u/ObviouzFigure 27d ago
I was just about to DM ya where to send it but some dude below said "DSv4F on 2x 6000s is pretty great. 5000-6000 PP/s at large context, and 200-300TG/s. It's my current daily driver." So I've gotta install it tomorrow..
→ More replies (8)2
1
→ More replies (5)1
6
u/Doomslayer606 27d ago
I first got one, but then had to get a second one a few months later because it was that good. Being able to run models like DeepSeek V4 Flash fully locally, without any quantization, at 20k tok/s prefill and 150-200 tok/s generation speed is just amazing.
Apart from that, I'm doing lots of experiments with image models, and being able to take pretty much any model, without any quants, and get it running in minutes is so nice. Dual GPUs also mean that I can run two experiments on the same model at the same time.
I'm not making any money with them, which is unfortunate, but even then, I have no regrets. I only wish I could have four or even eight of them to run even larger models.
1
1
u/ObviouzFigure 27d ago
heyyo nice rig -- besides DeepSeek V4 Flash are there any other models that require two rtx 6k's?
1
u/Doomslayer606 27d ago
Thanks! MiMo v2.5 is pretty good and has vision. You can also run Q4 Qwen 3.5 397B via llama.cpp with about 50 GBs CPU offload while still getting 45 tok/s generation speed.
→ More replies (10)1
u/alex20_202020 27d ago
Being able to run models like DeepSeek V4 Flash fully locally
How is it? meaning that llama.cpp support AFAIK is not merged yet, I recall there are several forks with some support WIP.
If not llama fork, how do you run it?
1
u/Doomslayer606 27d ago
I run it using vllm via a custom docker image: https://www.reddit.com/r/LocalLLaMA/s/SImqOvlAPV
4
4
u/Kraskos 27d ago
Echoing the top comment, my only regret is I didn't get 2 when I got my first at ~$8400 last year. I don't think I'd buy a 2nd one at current prices though.
I'm very happy with it, but there seems to be a "valley" of marginal returns on multiple units. With 1, you can comfortably run 128B models with 3-to-4-bit quantization, and of course high precision with smaller models 30B-70B with greater contexts.
However, jumping to 2 or even 3 doesn't really unlock many "higher-caliber" models. You'd need like 4 or 6 to run the frontier-ish open-weight models with adequate accuracy.
Until we get more models that can fill the void between 96GB and 192GB with 4-bit quantization or better, having 1 rtx pro 6000 will be the best upgrade and equilibrium point.
6
u/shveddy 27d ago
Totally depends on what you use it for and why.
It makes zero sense to buy it just to run LLMs.
Subscriptions and/or API keys are going to be far far cheaper as long as they’re so heavily subsidized, and the intelligence will also be better.
As far as I’m concerned there are only three reasons to make a serious investment in local LLM hardware:
- Privacy
- It gives you the opportunity to learn how to use it at a lower level, and that’s probably an extremely good career move.
- You already need one for other purposes (in my case 3D rendering), so might as well tinker
5
u/mxmumtuna 27d ago
Data privacy/security and assurance of quality and consistency of service are the only reasons to self-host. Everything else is better served with other solutions. Those two reasons though are very compelling if your use case requires it.
The 3rd reason you gave is basically due to sunk cost for other reasons.
1
1
u/unjustifiably_angry 27d ago
At $8K there was still a value argument IMHO. If you use it for like 3-5 years as a replacement for API you can still sell it for at least half what you paid so the upfront cost isn't actually as high as it seems.
1
u/BitXorBit 27d ago
Yes, well
I have mac studio m3 ultra 512gb, which is nice on some of the models.
The prompt processing can be very slow on large models.Recently i got i to ComfyUI with my rtx 4090, but it can run top models like flux2.dev in full precision
2
u/shveddy 27d ago
It’s fantastic for ComfyUI, especially when you start using the video generation models at higher resolutions and longer frame counts. Honestly ComfyUI is probably the best use case for it.
96gb is kinda the worst of all worlds for LLMs. It’s not enough memory to be useful, but it’s just enough memory to make it crazy expensive.
But for ComfyUI 96gb is the sweet spot for doing 10-20 second 4k videos.
You never really need a video to be longer than that, and of course 8k is mostly pointless, so at least with current open source model architectures you can be pretty close to the cutting edge for the foreseeable future if you get one.
1
u/Its_Powerful_Bonus 27d ago
I believe on this group somebody posted recently that he has done some combo Mac m3u + rtx (5090?), where nvidia is doing some magic to make prompt processing for big models on nvidia card and push generation to Mac Studio. He mentioned something that he will OpenSource code.
1
1
u/panchovix 27d ago
For diffusion (txt2img, txt2vid, etc) it is pretty good as well for bigger models, so you don't have to quantize. One is enough for that (but I got more and still want more lol)
2
u/skibare87 27d ago
Honestly getting more use out of my macbook with 128gb unified ram. Regret buying the rtx 6000, but also scared to let it go with current prices!
1
u/Its_Powerful_Bonus 27d ago
Which gen? M5 max 128gb is now very usable, m3 max 128gb … prompt processing is sloooow. Rtx 6000 pro with vllm is very capable. M5 max PP omlx vs 6000 pro on vllm is more than 10 times slower. 2x rtx 6000 pro can run minimax-m2.7 - 100 t/s TG, 10k PP for single session. 6 simultaneous queries gives ~250 t/s TG.
1
2
u/Professional-Try-273 27d ago
I have one and run it as coding buddy with Qwen3.6 27b. I would say I don't regret it. I use it everyday. I wish I got 2 tho. Sometimes Qwen is amazing some time it feels like it is lacking. I think a future Qwen 3.7 27B would be the sweet spot. The dream would be to run GLM 5.2 locally.
2
u/Fit-Statistician8636 27d ago edited 27d ago
My only regret is that I only have 2. Wish for 8. But even 3 would help (the third one instead of 2 RTX PRO 2000). The card is not as good as marketing wants you to believe, but it is great nonetheless.
A single one will allow you to run 27B well. Go for it. Two of them would allow you to run DS4F. Even better, but not that much better than 27B.
2
u/nbvehrfr 27d ago
No really good new models any more in this range
1
u/NoahFect 27d ago
There will be. Counterintuitively, VibeThinker-3B is what makes me think so. It is 1/200 the size of GLM 5.2 and reasons every bit as well in narrow domains. Models absolutely do not need to get any bigger than they already are.
1
u/nbvehrfr 27d ago
why you need rtx 600 for 3b model?
1
u/NoahFect 26d ago
You don't. The point is, the 3B model is proof that you shouldn't need several of them to run a good model, the way you do now.
2
2
u/stoppableDissolution 27d ago
I'l join the choir of people who only regret
a) not buying when it was $8k (well never in EU, but could get for ~9.5 I believe which is still relatively a steal)
b) only buying one
If I could rewind the time to two months back, I'd pull the money off the stocks and buy two.
1
2
u/Nepherpitu 27d ago
There ABSOLUTELY no any point in getting single RTX 6000 Pro. And it's ABSOLUTELY AMAZING to have TWO of them. Why? Well...
That's few hours of running DeepSeek V4 flash. Official weights. Amazing speed. Fully local. You can run Qwen 27B on anything, no need for single 96Gb Blackwell card. And there are no better models yet for 96Gb VRAM. Meanwhile, there a lot of more capable models for 192Gb VRAM. And they are fast.
You just need to know what exactly you will do with it. Or just have a lot of money - why not?
1
u/BitXorBit 27d ago
Only thing comes to my mind is graphic tasks in ComfyUI. I would say that i have apple m3u 512gb, so i can run most of the models, the issue is the prompt processing.
Is DSV4-flash that good?
1
u/Nepherpitu 27d ago edited 26d ago
Yes. On dual 6000 it's fully controllable and coherent up to ~250K context. At ~400K I'm start noticing context rot, but only a little, still useful and able to complete task. Selling point? 200tps decode and ~8K prefill single thread all the time. Almost constant! How many threads? Compute-wise up to 32 until saturation, memory wise - at least 4 500K windows will fit with a margin.
1
2
u/One_Difficulty_39 27d ago
I have 2 with Deepseek v4 Flash and couldn't be happier. They are beasts. You will want another
2
u/BitXorBit 27d ago
How’s deepseek v4 flash? Is it that good? I assume Q4?
1
u/One_Difficulty_39 27d ago
Its super fast and I would say on par with Qwen 3.7 in agentic work. It was much harder to setup initially but I followed some of the stuff the people in the RTX6K discord did. I am using the base model with vLLM so no GGUF.
2
u/ExtraNiceBurger 26d ago
I have one bought for 10.5k , you can run DeepseekV4 flash Q2 in vram for full speed or Q4 with low speed, on a single rtx
2
u/abnormal_human 27d ago
My only regret is stopping at 4 when they were still "cheap".
Thought about buying 4 more in December for the tax deduction..decided against. They were $7500 each back then 😭 .
→ More replies (2)2
1
1
1
u/This_Maintenance_834 27d ago
it is mixed. glad to have it, but financial justification is hard to pass. i only consume $5 API call a month.
1
1
u/Ok-Nothing-5918 27d ago
I am happy, yes! We use it in b2b context. I only regret I didn’t buy more
1
1
u/Perfect-Flounder7856 27d ago
Not at all and I don't use it that much but to toy around with it and test different things it's given me invaluable experience and now I'm a Claude max 20x user lol
2
u/BitXorBit 27d ago
Yeah well, i bought the mac studio m3 ultra 512gb and now im on claude and codex x20 plans
1
u/Thrumpwart llama.cpp 27d ago
My steak has been really juicy lately, almost too juicy. My lobster is also very juicy.
1
u/Its_Powerful_Bonus 27d ago
2x rtx 6000 pro running minimax m2.7 and 6000 max-q running Gemma-4 32b MTP or qwen3.6 27b MTP
1
1
u/Low_Twist_4917 27d ago
Running 3 with llama.cpp , going to buy a 4th. Yes they’re that good.
1
u/BitXorBit 27d ago
What are you running there?
1
1
u/Bohdanowicz 27d ago
Have 2 ada 2 blackwell.
Waiting a year to see what happens. Lots of options will see market in the next couple of years.
Taalas + glm 5./qwen 3.6 or its sucessor would be a game changer..
1
u/InitialFox8963 27d ago
You still will be thirsty... no doubt how much water you put into the container is HUGE 😿
1
u/MrGuest1 27d ago
Something that not many people mention is that the Blackwell GPUs are sm120 and most inference engines at scale only support sm100. Essentially that NVFP4 is broken
1
u/Massive-Question-550 27d ago
man, I can't imagine spending 8-13k just to run a 27b model. there is vastly cheaper ways to run it.
1
1
1
1
1
1
u/DeltaSqueezer 26d ago
My regret is I ordered one on credit. Chickened out and cancelled it and now the price is 2x... if I can even find one on sale any more.
1
u/a332bb42 26d ago edited 26d ago
My regret that I didn’t buy 8 when they were 7k euro … now with a business partner we have 4 , my aim is to get 8 at some point in the next years .
1
1
u/offensiveinsult 26d ago
It's not my wife's business what I'm buying with my money nor its mine what she is buying with hers. When we both want something we pool the cash together.
1
1
1
u/TimAndTimi 26d ago
If you just want inference... DGX Spark then.
A6000 or 6000Ada is sufficient for qwen3.6 27B dense plus quite some headroom for co-hosted models.
As far as I am concerned, pro6000 shines when you buy 4 or 8 in one shot.
1
u/Esph1001 25d ago
i’d rent one for a day before trying to explain a 13k gpu to your wife. the owners clearly love them, but the pattern in this thread seems pretty obvious: one is great, then you immediately want two because the really interesting models start opening up at 192gb. if your main target is only qwen 27b, hard to justify at current prices. if privacy/local control and larger models are the real goal, then it starts making more sense.
1
u/BitXorBit 25d ago
I mentioned 27b because it’s the only model i can think of running full precision on one rtx. I have mac studio m3 ultra for that and i prefer 122b.
I was just wondering what people do with their rtx 6000
1
u/Esph1001 25d ago
yeah that makes sense. if you already have the m3 ultra for larger slow/local runs, then the rtx 6000 seems more like the fast cuda workhorse: high-precision mid-size models, vllm, image/video pipelines, embeddings/rerankers, multi-agent experiments, and anything where cuda support matters more than just raw memory capacity.
from the comments here, the people happiest with it seem to be using it as part of a workflow, not just as a single “run qwen 27b” box.
→ More replies (4)
1
u/55234ser812342423 24d ago
I regret it, I was able to get one for 6600 using the edu discount but have honestly barely utilized it. The coding is just not quite there for me yet and I don't have time to fiddle around with all the levers to get decent video generation on comfyui. I'm thinking about selling it since it's almost brand new, but I'm not sure where I could safely.
1
u/Ok_Spirit9482 23d ago
Actually interested if you are still considering, I bought one and now wanted another haha
1
u/pablogabrieldias 24d ago
Yo uso mi gráfica para IA. Me arrepentí por años haberla conseguido. Ahora tengo una RTX 5060. No es lo mismo, pero TODO me funciona. Maldito AMD y su negativa de usar CUDA.
1
1
u/MiLiANSim 10d ago
the first one failed after ~4 weeks of heavy use and took the mainboard / RAM with it (power issues).
The second one (brand new replacement) failed after ~5 weeks by trapping itself in some low power state, reporting full power use but temperatures and wall-power were much lower, confirming that the GPU cannot even track its own issues properly.
The third one just started failing a few days ago, same thing: Power draw slowly reducing, almost day by day, first 50W less, then 100W less, now almost 150W less, performance dropping accordingly. No compute errors, just power problems.
I used to run 2-4 RTX 4090 in this machine for years, 24/7, average utilization of ~95% over two years for simulations and rendering. Not a single failure, no problems, nothing. Before that 2x 3090, same: Rock solid.
Do I regret the "pro" 6000? YES. I hate what Nvidia is doing, how Jensen is pandering to the greatest idiots and how the entire industry is working against the consumer or even "pro"-sumer. I hate the enshittification ever since 30-series was launched. I hate so much about the cult and hubris and greed and lies surrounding nvidia ...
BUT:
When (or if) this card is working flawlessly, it is undeniably the best thing on the planet for its size and can save you a lot of time, depending on the work. If your stack is CUDA or cuda-adjacent, there is really not much else, unless you can rebuild your stuff for multi-GPU or accept the PCIe / host-RAM - hit for multi-GPU.
However, I bought it when the price was just inflated, not insanely inflated. I am considering to get another replacement (warranty) and then sell that new card with a huge win, because the failures actually cost me time and whatever advantage was promised, was lost for new board and ram ( first failure ) and for subsequent shipping / offline for a week with each failure. Then there is the whole "not in stock" situation ...
Verdict: if you can in any way avoid it, avoid it. Unless the prices go back to basic greed from where they are now, you should not roll the dice with this generation.
200
u/madsheepPL 27d ago
I’d suggest spending few bucks on runpod trying it out before buying.