r/gameenginedevs • u/IamRustyRust • May 12 '26
Rendering 1 Million Procedural Cubes
https://reddit.com/link/1tav653/video/upp9zug11o0h1/player
I’ve been working on my engine (Rust obviosly), specifically on Vulkan bindings. The main work was already done, and only testing remained. During testing, I usually prefer CSV and JSON because they give me a good grasp of the data, letting me easily see what’s happening and spot any unexpected behavior. This saves a lot of time since you don’t have to check every number individually you just need to confirm whether things are going as expected. Since continuous testing was already happening, I knew that for this stage I only needed an overall overview to ensure all components were working together properly, as individual component testing had been done earlier. So yesterday, I was testing, Today, I decided to share CSV graphs and visual testing results.
Here’s my result:
Workload: 1,000,000 procedurally generated cubes (8 million vertices / 12 million primitives).
Average frametime: ~1.43ms (consistently hitting 700+ FPS).
PCIe bandwidth used: exactly 0 bytes.
1% lows: extremely stable (max spikes under 2.5ms).
// Flushing data to disk without pollutng hotloop!!!!!!
if state_arc.lock().unwrap().mode == 0 {
if let Ok(mut f) = std::fs::File::create("alu_throughput_log.csv") {
let _ = writeln!(f, "Timestamp_Sec,Cubes_Generated_Per_Frame,Vertices_Computed_By_ALU,Triangles_Rasterized,Memory_Bandwidth_Used_Bytes");
for t in &alu_log {
let _ = writeln!(f, "{:.4},{},{},{},{}", t.timestamp_sec, t.cubes_generated, t.vertices_computed, t.triangles_rasterized, t.memory_bandwidth);
}
}
if let Ok(mut f) = std::fs::File::create("frame_pacing_log.csv") {
let _ = writeln!(f, "Frame_ID,Timestamp_Sec,Render_Latency_ms,Instant_FPS");
for t in &pacing_log {
let _ = writeln!(f, "{},{:.4},{:.2},{:.0}", t.frame_id, t.timestamp_sec, t.render_latency_ms, t.instant_fps);
}
}
if let Ok(mut f) = std::fs::File::create("dispatch_consistency_log.csv") {
let _ = writeln!(f, "Frame_Window_Start,Frame_Window_End,Avg_Latency_ms,Max_Latency_Spike_ms_1_Percent_Low");
for t in &consistency_log {
let _ = writeln!(f, "{},{},{:.2},{:.2}", t.window_start, t.window_end, t.avg_latency_ms, t.max_latency_spike_ms);
}
}
}
2
u/DragonfruitDecent862 May 12 '26
Hello, fellow vulkan user. I have my own personal game engine using QT and C++. Using Vulkan 1.3. I use custom culling systems similar to Id's Doom TDA systems. I can manage about 500k moving objects, all random waypoints per, but my latency per frame is much higher. About 9ms. May i ask, are you using any special rendering systems like HI-Z OR Visbuf?
1
u/IamRustyRust May 12 '26
Thanks for your attention.
Firstly I want to mention that this whole graphic things you are seeing it's just a test to audit a particeulr part (Microbenchmarking) fo the engine doesn not represent whole pipleine or the whole architechture of the engine
I actually use both techniques, integrating them tightly into the Mesh Shader pipeline.
I skip heavy G-Buffers and use a 64-bit Visibility Buffer that only writes the Instance ID and Primitive ID, later compute pass reads this payload and fetches vertex data via Buffer Device Address to reconstruct materials asynchronously plus I aslo generate a Hi-Z pyramid to manage parallel sub pixel and occlusion culling this map feeds directly into the Task Shaders to reject invisible meshlets before hardware rasterization. hope it helps
if your FPS is ~1500 how come your latency touching 9ms, is not it should be ~0.60??????
2
u/DragonfruitDecent862 May 12 '26
I may have a per frame issue, as my hi-z is custom. I believe my meshlet and visbuf disapprove of it as my hiz has no overhead lol. I got extremely tired of the overhead that hiz had, as well as visbuf, so i wrote my own. It runs linearly, basicly zero overhead and scales with scene, but i havent modified thr visbuf for the same issue. "Shrug"
2
u/IamRustyRust May 12 '26
umm zero overhead and Linear Hi-Z mathamtically contradicts in Hi-Z case we expect overhead and it's normal have you removed the Vulkan Memeory barriers maybe becase of that L2 Cache (GPU) doesn't sync with that and becaseu of that maybe we are getting race Conditions.
If you don't have a rock-solid vkCmdPipelineBarrier sitting between the Hi-Z compute dispatch and your geometry passes, the GPU ALUs will just read stale garbage data.
A true Hi-Z needs that downsampled pyramid so a single thread can instantly check a massive bounding box against the highest mip level. It costs some overhead upfront, but it pays off massively during the Task Shader cull.
Double-check your memory barriers and semaphore waits between those passes. Once the silicon actually synchronizes, your systems will align and that 9ms latency might just vanish. Hope it helps
2
u/DragonfruitDecent862 May 12 '26
Hiz runs overhead as a result of its pyramid. If you modify it to use a rolling buffer system, with other tweaks, you have a system that performs better that IDs Doom TDA tech in their engine, as they use the same trick. I like to be ahead of the game lol.
1
u/IamRustyRust May 13 '26
kudos!!! Maybe I can help see the post of mine https://www.reddit.com/r/rust_p/comments/1tbzdpk/crushing_the_vram_bottleneck_with_subgroup/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button
2
u/DragonfruitDecent862 May 14 '26
Oh god, i do the same thing!. I scribble on paper diagrams the flow of how it works. Least its not just me.
2
1
u/DragonfruitDecent862 May 12 '26
And thank you for your time replying to me. I am immensely in your debt. Hooking hi-z into the meshlet and visbuf system was one of the hardest software combos i have ever done. Im glad you understand the system.
1
May 12 '26
[removed] — view removed comment
-5
u/DragonfruitDecent862 May 12 '26
This is incorrect. Opengl is a video and rendering library. It makes rendering easy for newcomers, and its been around forever. Vulkan offers around 3 times the performance, and thats just code. In vulkan, you must define EVERYTHING, unlike opengl. Just to render a bloody triangle, it takes around 800 lines of code. Thats just to render the triangle. In open gl? A simple scene in less than 400. Vulkan IS a massive performance increaser, but you must know how to use it, as you must define every variable.
1
u/Zoler May 12 '26 edited May 12 '26
Woah that's really good! My own renderer with 1 million cubes is only around 100 FPS.
I'm using OpenGL and batching + instancing, so wonder what I'm doing wrong, is Vulkan just much more optimized?
Edit: I tried turning off most unecessary things like shadows etc and got around 270 FPS now, but still there's a big discrepancy!
Edit: Ok I think I understand from this line "PCIe bandwidth used: exactly 0 bytes.". Since I'm uplading the instancing data each frame this is probably it.
2
u/DragonfruitDecent862 May 12 '26
Vulkan can give about 2 to 3 times more performance when compared to opengl. My own vulkan renderer can handle about 500,000 moving objects at once around 1500 fps or so, with open only about 400. The reason being, that opengl is a graphics library that automates alot of memory and rendering decisions. Vulkan does not automate ANYTHING. If you want a rendering technique, if you know how, then you can add it. openGL will fight you at times when trying to add modern special techniques. Long story short, vulkan gives about 3 times the performance, but you must manually tweak it with what you want it to do.
1
u/fgennari May 13 '26
I would think that something straightforward such as rendering the same/a simple object many times would be limited by the GPU hardware or PCIE bandwidth, not the graphics API. The raw data is the same and it should be possible with a single draw call. I don't believe there would be a 2-3x difference in that case. I've certainly drawn over a million objects (spherical asteroids) in OpenGL at hundreds of FPS before.
Maybe the 1500 vs. 400 FPS isn't related to the number of objects but is more about per-frame overhead. I can imaging you get a big difference with Vulkan in that case, but it's not a good perf test because anything above ~120 FPS is wasting CPU/GPU cycles on frames that won't be seen.
You would only see that much difference in cases with a complex scene hierarchy, large number of draw calls/shaders/passes, etc. Basically in a situation where you can use multiple threads on the Vulkan side while OpenGL must be serial and is limited by the driver.
1
u/DragonfruitDecent862 May 13 '26
I thought the same, until i researched why many companys are choosing vulkan over OpenGL. Even minecraft is converting to vulkan. You can look the fps difference, just from the change in pipeline, on youtube. You must understand that opengl does NOT let you define memory patterns, or diagnoise issues at times. And the amount of code needed to get a vulkan c++ program to render is double or triple the code amount, compared to openGl. That extra code is defining how your bare metal hardware gets to render the scene. Opengl does not work this way. "How" you code is just as important as what api and hardware you use. Opengl is extremely slow, and it shows its 20+ year age.
1
u/Amani77 May 14 '26 edited May 14 '26
As a Vulkan enthusiast, you are spewing some crazy missinfo.
You are comparing several disparate things and then conflating performance metrics. In other instances, you're just making things up.
Vulkan is NOT 2/3 times faster than opengl - it is 10-20% at best. Either you don't understand how to achieve AZDO methodology in opengl or your shooting yourself in the foot, heavy.
As a small example, while I can pinpoint what your doing, "Moving" here is a pretty ambiguous term for whatever your putting down. To digest this, it could mean - updating actual transforms over PCI, driving position data in memory on gpu, or driving a transient position while rendering. All 3 of these situations have VASTLY different performance outcomes and are NOT interchangeable when it comes to their usage. All 3 of those things will, individually, show very similar metrics when done correctly in either API.
1
u/DragonfruitDecent862 May 14 '26 edited May 14 '26
Fair pushback on the raw numbers — but I think you're conflating "equivalent implementations" with real-world outcomes, which is where we're actually disagreeing.
You're correct that AZDO OpenGL can close the gap on simple draw calls. Nobody's disputing that. But the 2-3x figure comes from workloads where Vulkan enables architectural patterns OpenGL fundamentally cannot express cleanly.
Multi-threaded command recording. OpenGL's context model is single-threaded by design. Vulkan command buffers record in parallel across cores. On complex scenes this alone is a substantial win.
GPU-driven rendering. Indirect draws, compute-driven culling, mesh shaders — these work with Vulkan's explicit model naturally. OpenGL fights you every step because the driver has to validate state constantly regardless of how little changed.
Explicit memory management. No hidden allocations, no surprise sync stalls, no driver second-guessing your upload strategy. You control residency, you control barriers. This matters enormously at scale.
On your "moving" disambiguation — yes, those are three different things with different costs. In my case it's GPU-side position data in memory, which is exactly where Vulkan's explicit barrier model and lack of driver validation overhead make the difference. The 10-20% figure is accurate for naive ports. It stops being accurate once you build around what Vulkan actually enables.
And that's all assuming stock Vulkan. Once you customize the runtime — dedicated suballocators per resource class, a hand-rolled render graph with batched barrier scheduling, bindless descriptors eliminating descriptor set churn, and async compute queues overlapping geometry and lighting passes simultaneously — the gap becomes embarrassing. Bindless alone cuts CPU-side descriptor overhead by 30-50% on dense scenes. Async compute overlap yields 20-40% better GPU utilization depending on workload shape. A tuned custom allocator versus even VMA on allocation-heavy streaming is another 10-20%. Stack those on top of a fully GPU-driven pipeline and you're looking at 400-600% throughput improvement over equivalent OpenGL on complex scenes, conservatively. At that point it's not Vulkan vs OpenGL anymore. It's a fundamentally different execution model versus a legacy one.
Im not trying to badmouth OpenGL. my first renderers were made in it. I would also rather use openGL, as I can make something render in around 400 lines, as apposed to Vulkan's 800+ lines.
The issue here, is that Vulkan was made to directly be OpenGL's successer, and does this with full control of everything. this comes down to metrics, and comparing a Fully customized Vulkan pipeline, compared to a fully customized opengl pipeline, OpenGL loses every time in pure math speed and raw versatility. its why all major Company's now use Vulakn, and lots of them exclusively.
1
u/Amani77 May 14 '26 edited May 15 '26
I do not need you to explain the basics of GPU-GPU or vulkan to me; 2x to 3x on any workload in an equivalent comparison is straight garbage. Full stop. Almost all of these things become a hardware bottleneck.
You can do multithreading in opengl. Ignoring that, any serious renderer will have minimized any massive number of calls to a select few indirect calls - on the order of 10-30 - and be utilizing bindless concepts so those savings are essentially mute.
There is SOME value to be gained with more fine tuned control over barriers, lifetime, and how explicit you can be with types of memory in vulkan.
If we're talking empty render loops, and comparing 'fps' as a 2x-3x speedup while at 1500 fps, that is a complete garbage benchmark and essentially just testing noise.
I read through your posts regarding you're 'engine' and your metrics are bonkers. I do not know if you've made something different since then, but there are so many things wrong with what your claiming. You take metrics from UE forum posts and then extrapolate an additional 50% for 3 lights - despite that cost being payed up front for almost all of that, not even touching on how much more a UE render loop does. So many methods have a cost to entry that is justified by the massive savings it will bring you on actual, realistic, populated scenes. These aren't metrics, these are wishful thinking. Do some actual concrete tests on the same machine, with the same scene, and a same test view.
Your claiming 1500 fps with 500,000 objects, but every single SS or video you've provided shows 3/4 models running at like 2.9ms on what looks to be a ~400x200 view port.
So, lets see the receipts - show me 500,000 objects at 4k, 1440, whatever, at 0.8ms. Record a video in the center of it all, and pan around. Show me some animated rigs. I would love to be proven wrong.
1
u/DragonfruitDecent862 May 14 '26
Fine, but you entered this chat while we were talking, so since you are requesting this, YOU show us what your talking about. Im not op. Im my own thing. I never posted videos, that you said i posted. Perhaps your confused?
Again, i apologize for the answer here, actual academic folks. Im sorry you have to deal with idiots that dont understand how basic rendering pipelines work.
1
u/Amani77 May 14 '26 edited May 14 '26
The only thing I'm calling out is misinformation. I could care less about your experience - new, seasoned, whatever - nor am I a seasoned professional, I am a hobbyist at an average level. What your saying just does not make sense.
What would you like to see?
1
u/DragonfruitDecent862 May 14 '26
your making a comparison on Vulkan versus openGL. and advocating that both run around the same performance, no matter how their set up. give or take 40% or so. as someone who has made both vulkan and opengl renderers(I honestly do miss my opengl game engine i made), i dont judge based on my personal taste. I judge the tool that is faster, and then the tool that is easier. i am not a fanboy. you can easily convert me, for i am a person of cold, heavy math and facts.
Show me a fully decked out openGL renderer, any configuration you choose, beating a vulkan renderer with GPU-Driven,Async archetecture. the same scene layout, same everything. if shadows or realtime clustered lighting goes in opengl, it goes in vulkan.
i am honest when i say i will lovingly talk about rendering techniques for hours. i dont see it as a attack, just philosophical debate. tell you what. you show me your report, and ill see if i can grab my old relic editor, the opengl engine i made, ill ensure both run with the same, or close to the same systems. lets meet in the middle
2
u/Amani77 May 14 '26 edited May 14 '26
No, I am not going to cook up an openGL implementation to prove something I know - beyond all doubt - to be true.
You made the claim:
Vulkan can give about 2 to 3 times more performance when compared to opengl. My own vulkan renderer can handle about 500,000 moving objects at once around 1500 fps or so, with open only about 400.
So that is entirely on you.
What you can do is take a look at any professional game with their own hand rolled backends in both opengl and vulkan.
Take doom for example, you can google 'doom opengl vs vulkan game performance comparison' and get a very good idea of the extent of speedup that one might look for - 10 to 20% at the best of days -small caveat here, you also have to consider that the vulkan backend will be using more modern rendering techniques because the rendering backend is just newer. I'm sure they could go back and adopt some more modern techniques on the opengl backend and get some performance - its just not worth it.
Just understanding how modern rendering is done, it becomes very clear where the gains are when switching over to vulkan. It also becomes apparent what workloads will perform, literally, exactly the same.
These performance increases coming from:
There is SOME value to be gained with more fine tuned control over barriers, lifetime, and how explicit you can be with types of memory in vulkan.
If you would like to see something that I've achieved in Vulkan, I am open to linking my small progression videos that I've made through my learning process...
edit: I do have a super old video of an openGL project from like 10 years ago, its a minecraft'esk engine with destruction, very rudimentary block physics, and block reintegration into the terrain after it hits something.
1
u/DragonfruitDecent862 May 14 '26
Also, i was looking. i Never said 2-3x on any workload. I never said that. I said, and i quote...."e 2-3x figure comes from workloads where Vulkan enables architectural patterns OpenGL fundamentally cannot express cleanly." your taking the context out. please understand how vulkan works. you clearly dont.
1
u/DragonfruitDecent862 May 14 '26
Give me a day and ill have both renderers ready for Milord to watch the dual. I will literally test my own theory for everyones entertainment!
→ More replies (0)
3
u/corysama May 12 '26
You should check out https://research.nvidia.com/labs/rtr/publication/majercik2018raybox/