From OpenGL to Vulkan

In my last update, I decided to start learning something called Vulkan, or Vk for short. What's Vulkan, again? Why, it's just the hottest new way to put your graphics card to use!

I began climbing the Vulkano (so to speak) in 2023, but despite releasing multiple Vk-powered minigames since then, I'm still climbing it over two years later. Vulkan's reputation for taking nearly 1000 lines of code just to render a single triangle is no joke! But I was ready for the challenge. Let's dive in and see what's been taking me so long.

 

Early Rumblings

 

August 21, 2023: With no idea what I was doing, I searched "vulkan tutorial", clicked on the first result, and got to work. In a single week of leisurely progress, I already had my first triangle in just 845 lines of code (well, with the help of a 4913-line helper system called volk).

I may have just been copy-pasting tutorial segments together, but it felt like I was on a roll! It wouldn't take much work to add more shapes to the scene. Most of those 845 lines were for setting up the program, like establishing a proper connection with the graphics card, reserving the right types of memory, and other mundane details. Soon enough I had two triangles (aka a rectangle)!

 

By week three, the window could be resized (including full-screen mode!) and my shaders were compiling automatically without external software (don't worry if you don't know what that means). Not long after, I had textures and could even animate the scene!

 

In the past, my games had basically just been lots of shaded textures sliding around, so surely I'd covered most of the features by now. All I had to do was fill in some gaps and incorporate this project into my engine.

The Drought

 

And then the realization gradually dawned on me that I hadn't actually accomplished all that much yet. Vulkan is at its best when, just like in this tutorial demo, you know in advance exactly what your visuals require so you can set every structure in stone. When you do this, your GPU turns into The Sphinx and winks at you.

But if anything in your scene needs to change or behave unpredictably, like running scripts from a file, you are left on your own to solve the riddle of moving the entire Egyptian Pyramids around by hand in a well-coordinated improvisational dance.

I had climbed the first hill only to discover the vast desert ahead.

Welcome to Vulkan.

 

Dormancy

 

For the next five months or so, I was stuck in a loop. When I'd start writing the next critical feature, the app would naturally stop working until I finished that part. During this under-construction phase there'd be nothing to see but boring old C++ code, sometimes for weeks. Then I'd pull the feature together, only to be back to exactly where I was before: the spinning squares.

Always those same spinning squares.

I was really stuck with those spinning squares.

 

Spinning squares.

Sure, I could've changed my test app's facade now and then, but that would have been a distraction from the real goals. Every time I arrived back at this scene (you know, the spinning squares), I knew my code was another step closer to solving Vulkan's puzzles. So you may wonder... how on earth could this process be so dizzying for so long?

 

Inside the Magma Chamber

One of Vulkan's early accomplishments was that it "minimizes driver overhead", meaning it removed much of the work required for people to bring Vk support to your hardware and many other devices. That's exciting! The downside is that this work didn't disappear, but instead became the responsibility of every Vk-based program, like NetMission Engine. And that work... is less exciting.

 

To study the level of detail, we could take a magnifying glass and zoom into pretty much any portion of Vk-based code. How about this moment: our game is running at 60 frames per second, and it just directed a new frame for us to display. How do we show that final image on the screen?

WARNING: This section gets somewhat technical. If you begin to experience symptoms of discomfort, feel free to skip to the next section. I think you'll get the point pretty fast.

In OpenGL we'd use a single built-in function, usually something with a name like SwapBuffers(). And we're done! The graphics driver probably does some interesting stuff under the hood there.

 

But in Vulkan? ...

I spent too many hours making this gif.

... Well first, our image isn't in the right format yet. It's been set up to receive new pixels, not display them, which is an important distinction. To convert the image, we just need to fill out a 12-field VkImageMemoryBarrier2 form and a 9-field VkDependencyInfo form. We'll turn in this paperwork to a vkCmdPipelineBarrier2() which, just like the name doesn't suggest, offers image-reformatting services on the side. Be careful: if we're trying to use this GPU to the fullest, our image might be exclusive to only one "queue family" at a time. You see, it's further possible that rendering and displaying might operate on different "queue families", in which case we'll need to submit forms to two such pipeline barriers (warning: three in the case of non-image data) indicating an ownership transfer. Be sure to indicate the correct order twice or it might not go through.

 

Still here? Ah, sorry to say, we haven't even started. Literally! With Vk we tend to operate in a future realm where none of this is happening yet, even our image's rendering. Which means we now have a similar story for the accumulated past data referenced by that image, like the positions & colors of all our objects. The game was generating that info, but soon the GPU needs to read it. This data's properties can vary for complex reasons, so our strategy depends on which memory parts are "host coherent" or not, and/or if any are split between a "host visible" space and "host invisible" space. If you forget to do this data conversion here... it might actually still work on many devices, leaving you with a mystery for the rest. Good luck!

 

Moving forward, it's time to submit all our frame's commands into the correct queues so the GPU can eventually prepare our image. Alert: This is assuming our command-submission model is robust enough to handle all the above synchronization dependencies efficiently, or else we'll have performance warnings or impossible constraints. With this done, at long last we can use vkQueuePresentKHR() to "present" the image, and we'll attach a note saying to wait for the image to be ready first, since it's probably not yet. Whoops, tragedy strikes! Our hard-earned image might get rejected here if the window recently resized, so we'll have to react accordingly by e.g. creating a new "swap chain". Yeah, you get the idea. I'll gloss over that part. ...

... And we're done!

Critical Overload

When I researched my options in 2023, Vulkan was the winner for what I needed. I was craving that extreme degree of control & reliability in my engine's graphics & performance, and there are good reasons why so many other software projects are also switching to Vk.

That being said... Vulkan is... well, frankly it's just a really bad time. I heard its designers had some pressure to get version 1.0 out the door as soon as possible. I wish they'd been given more time to clean it up before it hit the fan.

I'd love to at least be able to rename all the unclear and misleading terminology... or fix the paradigm behind how resources get deleted... Synchronization is so confusing that virtually every Vk tutorial and program got part of it wrong. At the time of this writing, the Official Vk Validation Layer reports that the Official Vk Spinning Cube Demo has a nasty SYNC-HAZARD-WRITE-AFTER-WRITE error, and I actually don't know which side of that to believe. There's so much complexity.

Every software development story has some amount of toil and frustration, and unfortunately my experience with Vk is defined by much of that. But the internet has enough negativity already, so I'll do my best to focus on the fun parts. Plus it's worth knowing that the designers have been listening to the community and improving Vulkan over time, slowly but surely.

Anyway, this part's boring. Let's speed ahead.

Comedy Break

One day, I thought I'd be funny to grab arbitrary regions of GPU memory and interpret them as pixels on the screen. Tada, looks really cool on my laptop!

An artistic rendition

I left the glitchy effect open for a few minutes while I worked on something else. Then I closed out, and... huh, it was still flickering? Even over other windows? On my desktop wallpaper?! Even after rebooting?!?! Uh oh. Screen was busted.

The solution was to fully power-off my laptop for a week to alleviate the charge buildup. That did the trick. Screen was fixed! So I immediately made it impossible for NME to ever do this again.

Yep. Hahaha. Real funny... Anyway...

Magma Formation

February 1, 2024: I was finally able to send my half-year Vulkan experiment to others, and... *drumroll*... the scene ran smoothly on everyone's machines!

 

I added the tutorial's 3D model to keep it interesting

I had come so far, but to be clear there was one crucial stretch left: actually replacing NetMission's OpenGL code with all this Vulkan stuff. At first it was thrilling to demolish so much of NME's old code and restructure everything, but with an April 1 deadline looming, the pressure was on. Cue the montage music!

In a few frantic short weeks, I'd implemented enough to make a new game. Not quite enough to meet every feature of my old games, but that could come later. With a shiny new Vulkan-powered engine at my fingertips, I was finally ready to...

Hold on a second... That isn't montage music we've been listening to. That's more like... boss music?!

Phantom Bug: Final Form

There was a problem. About every 1 in 200 times I'd open my Vulkan-based engine, it would freeze immediately. Empty screen, no response.

Hmm, okay. "Troid," you say, "That's only 1 in 200 times, and only like a dozen people are going to play your upcoming demo anyway. The bug might never even happen. Come on, you're in a time crunch! Fix it later. Can't you just let the player close the window and try again?"

Oh, if only it were that simple. The engine would not close.

I don't mean that it simply wouldn't respond to the [X] button in the corner of its window. For that, the operating system (Windows) would notice and help close it. No, in this case, not even Task Manager could not kill the process. I began resorting to lesser-known PowerShell commands with administrative privileges to try to stab this thing. The world's sharpest knives would phase through this process like it didn't exist. Yet there it was, still clinging onto its ghostly Process ID...

I'm serious. The engine would not close.

Not even shutting down the computer could successfully close my engine. The shutdown screen would just spin its little animation at me until the end of time. I couldn't even put my computer into sleep mode for some reason. No, the only way to kill this process was to hold down the power button and hard-reset. There was no choice on the matter: I couldn't allow any number of my players to experience this phenomenon!

It was truly a finicky, elusive nightmare. Even if I attached a debugger mid-freeze, or if I had the engine output any text to show what it was doing, the bug would temporarily go into hiding. It was still there, but not in a way I could study.

Things were getting down to the wire, and I was scared. This was a total deal-breaker. What was wrong with my code? After so much effort, would I have to call it quits and revert the next game back to OpenGL?

...

Well, no. I figured it out. There's this function called vkQueueSubmit(). I already knew that I shouldn't overlap more than one vkQueueSubmit() at the same time if they share any parameters. But it turns out my machine's graphics drivers had a mistake: I couldn't overlap these at ALL, even with completely different parameters, or else I'd be haunted by the phantom. That would've been nice to know!

I probably could have waited for the next driver update for this problem to go away, but I couldn't wait. It was an easy fix! In code we use things called "mutexes" (which stands for "mutual exclusion") to avoid such simultaneity woes. So to make sure my vkQueueSubmit() functions never overlap, I merged a handful of parameter-specific mutexes into a single general one and called it a day. Slower? Maybe. Peaceful? Yes.

I was free at last.

The Vulkanic Eruption

Artwork by Team SCU

March 24, 2024: I released a preview of NetMission 5.0 (the new Vulkan version) to Team SCU, showcasing the title screen for our next April Fool's game. Other than some folks having to manually update their graphics drivers, the application ran smoothly. All systems go! Let's make a game!

April 2, 2024: Well, we missed the deadline by a day, but our little Vk-powered game was released to the world. Time to find out if the new tech holds up!

...And right away, there was a visual bug on someone's computer. Luckily that was a false alarm: it wasn't related to NetMission 5 or Vulkan. Basically I made a silly mistake in one of the shaders, which was easy to fix.

Artwork by Team SCU. Well, the left side at least. My apologies to anyone who saw the cursed one at full speed before I patched it.

(If you're curious: I was unaware that in GLSL the modulo operator (%) has "undefined behavior" (e.g. could be different from computer to computer) with negative values (which I was using), while the modulo function (mod()) is defined for both positive and negative values (same on all computers). If I could redesign GLSL, I'd make % the reliable one since it's easier to use, and maybe rename the other to positivemod() or fastmod() or something, for people who know what they're doing. Oh well. History is history. Live and learn.)

The Falling Ash

Anyway, we made it. NetMission was running Vulkan instead of OpenGL. How'd we do?

Excluding 3rd-party modules, NetMission had a net gain of 2,705 lines of code in the first draft. All things considered that isn't all that much. The whole engine is around 23,165 lines today, ignoring whitespace and comments.

My previous post focused on an older phantom bug: images failing to load in extremely rare circumstances due to OpenGL-related problems. And guess what? Vulkan ate that bug for breakfast! My engine was healed.

But one of the draws of Vulkan is its performance gains, and my engine... actually got slower! While it's true that the old NME couldn't have rendered 10,000 animated blades of grass like the new one, I could've implemented that feature back in OpenGL, too. For cases we can actually compare, the Vulkan edition was definitely worse: more stuttering, higher CPU and GPU usage, and a few really obvious slowdowns. We were lucky that our first game with it was a retro pixel-art game so it didn't matter much!

Coupled with the fact that NME no longer ran well on Steam Deck, these results were honestly pretty demoralizing. Was this worth all the effort overall? It was hard to say.

...Well the answer was yes. But it was hard to say that with any confidence. I was sad and tired.

Regrowth

With less-than-stellar results, the story couldn't stop there. I picked myself up and continued to refine my Vulkan back end one detail at a time into the present day.

• Too many command submissions, "semaphores", "barriers", etc.? Fixed.
• Doesn't display properly on Steam Deck? Fixed.
• Code-generated shapes (like shadows) are way too slow? Fixed.
• Can't run my old NME games due to missing features? Fixed.
• Obscure bugs and slowdowns all over the place? Fixed.

I make it sound quick and easy, but some of those points were fairly large undertakings. Vulkan's systems are intertwined in complex ways, where if I have a discovery or realization about the tiniest footnote in the specifications, the outward ripple effect from that might eventually have me drawing new logic diagrams and upending half my code for another redesign.

 

It's like working within a landscape of microservices that all hinge on each other in specific ways, or watching how video game speedruns change their entire routes over time due to seemingly tiny discoveries. 

...Ok, fine, I'll admit that this kind of puzzle solving can be invigorating at times.

 

Aftermath

 

Let's end this with a little Q&A.

 

Is the Vulkan-powered NetMission fast enough yet?

Yep! It has generally caught up or surpassed the former OpenGL-based version now, and I could optimize further if needed. Loading is like 2x faster (possibly for unrelated reasons), my games work on Steam Deck again, and I've thoroughly ironed out so many bugs that my engine has never felt more slick. I'm happy.

Are you done adding Vulkan into NetMission?

No, but it's become low priority until a future game needs more. Someday I'd love to finish adding 3D support, as well as "compute shaders" to boost particle simulations, fluid dynamics, lighting effects, fractals, and so on. I also want to improve my usage of "descriptors," and I should probably switch shader languages from GLSL to "slang". Might be fun to experiment with 2D vector styles or shifting a bit further toward a "retained mode" architecture. The list goes on and on.

Would you recommend that others learn Vulkan?

For most solo devs, no. I'm hoping something like a Vk2.0 would be designed better. OpenGL is still (mostly) available and capable, other new APIs exist, and there are free frameworks out there which wrap multiple graphics APIs into a single interface, not to mention the many well-established game engines that further hide the complexity. All of these alternatives are worth considering first.

On the flip side, if cutting-edge performance is crucial to your project and you have the time and resources to grapple with Vulkan directly, then go for it and good luck. I still maintain that Vk was the right choice for me and NME, and little did I know I was in good company. When I first started, Godot had just finished their 5-year evolution into a monumental 4.0 Vk release, Blender was about to begin their nearly 2-year journey to achieve a well-received Vk release, and the floodgates were opening for many others to make the switch. Even FFmpeg recently began using Vk to speed up certain decoders!

 

Is anything else new in NetMission besides Vulkan support?

Yes, luckily I wasn't stuck working on Vulkan this entire time. I completely revamped how spatial level data works, and I invented a new file format for linking text, cutscenes, voiceovers, and even translations in a natural way. Together these two features make it dramatically easier to build game content at scale, and I'm hoping to give them their own write-ups here down the road.

 

As an additional push toward usability, I granted access to a trusted individual (aka my brother) to use NetMission to build a game. So I've had to document my engine, teach it, watch another person use it, and receive their feedback. That process has definitely helped me transform NME for the better in innumerable little ways. More on that another day as well.

In other news, there's no Windows-specific code left in the engine, so in theory I could compile it for other platforms (Mac, Linux, iOS, Android, Switch, ...) if I had the time (I don't). So that's cool. Looks like I'm crossing off many items from the shortcomings list in my 2023 engine overview!

 

---

 

Anyway, that's all for now. Thanks for reading, whoever you are! See you in the next post.