There's an old mantra in game development:
It's a wise saying, because we game developers have so much fun making the underlying foundations, that we never get around to adding the actual gameplay.
This is an article about how badly I disregarded that advice, and what happened because of it.
As you can guess, I made my own game engine.
It was an gloriously unwise decision, and delayed my game for a long time. This is what's known as yak-shaving: getting so distracted by the details that you lose track of the original goal.
I learned a valuable lesson from that mistake: don't get distracted trying to make the perfect tools.
So, having learned that lesson thoroughly, I then made the same mistake again! Not being satisfied with C#, C++, Rust, and other languages in the landscape, 0 I made an entire programming language, Vale, and rewrote the game engine with that.
It wasn't on purpose, it just happened. You start out making a simple DSL, 1 and you find yourself working on it more than the game itself. Then, momentum just carries you forward, because you're having so much fun.
After eight years, I finally had an entire general-purpose programming language and engine that I could use in the 7 Day Roguelike Challenge.
These languages are mostly fine... but C# is slow, Rust can't handle a lot of basic safe patterns, and C++ doesn't have the syntactic goodies that I'm so used to.
Domain-Specific Language, usually a tiny language that helps with one specific task.
Making a programming language is like raising a child. They start out helpless and generally oblivious, and they constantly fall and spout nonsense. Adorable, indecipherable nonsense. 2
Eventually, you teach them how to walk, how to do basic math, how to follow instructions, and how to signal basic syntax errors... and then the impossible happens: they win an argument with you. They point out a mistake you made, and they were right.
It's a complex mix of emotions: embarassment because you were wrong, sorrow because they no longer need you, and pride because you taught them well.
There's a certain joy from using a new language that nobody else has ever used before. I felt like an Indiana Jones, exploring a tomb that no modern eyes have beheld.
There's a freedom one feels when not slowed down by garbage collection or reference counting, and not constrained by a borrow checker. 4 I'm free to implement what I want, and know that the language will help me do it safely. It's like I've been driving a Honda Civic in city traffic, and now I'm in a BMW cruising down the highway.
For example, those two features would make it so there's zero memory safety overhead for the entire below function, by my estimation. 5 For a language with shared mutability to have that is unheard of and makes me quite excited for Vale's future.
I have a particular wariness for the borrow checker, after learning it that it's incompatible with observers, most dependency injection, most RAII, and it can sometimes influence one into more complicated architecture with no performance benefit). This is why I made the Region Borrow Checker, which should handle shared mutability a bit better.
To reiterate, these features are not yet complete, and we only have a theoretical understanding of how many generation-checks they eliminate. Currently:
If we add parallel in front of this loop, it can perform these iterations in parallel on multiple threads, using seamless structured concurrency.
Hybrid-Generational Memory would notice that we're iterating over a piece of data we own, so it wouldn't have to keep checking it still exists.
Because we're accessing data that was a parameter to a pure function, it doesn't have to check that the data is still alive or increment any reference counters, see Zero-Cost References with Regions.
PatternMap would contain an "iso" HashMap, which means nobody outside has any references to it. Because of this, PatternMap can freely mutate it without memory safety overhead.
However, Scala is slow as heck so the Vale compiler runs slow, which means Vale code takes a long time to compile. 12
This was the biggest risk in this year's challenge, and almost pushed me past the deadline. Now I know that this year's priority should be to rewrite the compiler in Vale itself, which would be much faster.
I use Scala it in a mostly imperative fashion, more like Kotlin than any pure functional approach.
In case you're curious, the backend is in C++, because I'm a madman.
Moral of the story for language designers: Pay attention to your compile speed. Don't over-optimize, but at least track performance regressions and leave some TODOs around your codebase so you know where potential slowdowns might be when you do decide to optimize.
After the longest yak-shave in history, I still don't have much of a game. It's only about 6,000 lines of Vale code.
It's clear that if I spent all this time working on an actual game instead, instead of making the perfect programming language for game design, then I'd have three or four games by now! In this timeline, I don't have those games.
Still, I'm very glad I spent this time working on a programming language, because I ended up creating something so weird, so unrecognizable 13 that it blows people's minds, and that's a really great feeling.
I also now have something that can help a lot of people for decades to come. Speed and safety has always incurred a lot of complexity burden on the programmer, and maybe with this language, I can help with that.
Thanks for visiting, and I hope you enjoyed reading about this experience as much as I enjoyed writing it!
In the coming weeks, I'll be writing more about our "region borrow checker" which helps eliminate Vale's memory safety overhead, so subscribe to our RSS feed or the r/Vale subreddit, and come hang out in the Vale discord.
If you found this interesting or entertaining, please consider sponsoring me:
With your help, I can write this kind of nonsense more often!
Nobody would have thought that there was an alternative to GC, RC, and borrow checking!
The Vale Language Project is not just about making Vale, it's also about exploring, discovering, and publishing new programming language mechanisms that enable speed, safety, and ease of use.
The world needs more exploration here! Currently, most programming language research is in:
These are useful, but there is a vast field of possibilities in between, waiting to be explored!
Our aim is to explore that space, discover what it has to offer, and make speed and safety easier than ever before.
In this quest, we've discovered a lot of new techniques:
These techniques have also opened up some new emergent possibilities:
We also gain a lot of inspiration from other languages, and are finding new ways to combine their techniques:
...plus a lot more interesting ideas to explore!
The Vale programming language is only one combination of the features we've found. Our goal is to publish all the techniques we've found, even the ones that couldn't fit in Vale, so that other languages can make strides in this area.
Our medium-term goals:
We aim to publish articles biweekly on all of these topics, and inspire the next generation of fast, safe, and easy programming languages.
If you want to support our work, please consider sponsoring us on GitHub!
With enough sponsorship, we can: