• Alexander Mills

The World Might be Missing a Programming Language

Updated: Oct 17, 2021

See the full article on our Medium page

Written by Alexander Mills

We have all the languages we need — perfection has been achieved. Or has it?

A commonly expressed idea, since 1995 or so, is that the world does not need another programming language. We have all the languages we need — perfection has been achieved.

After working with a variety of languages in my career so far, I think there is at least one language missing. Always on a quest for that silver-bullet, I have spent some serious time over the winter holidays trying to find a new language to add to my toolbox. However, I am having trouble locating this mythical language.

The proverbial silver bullet. A lot of people are surprised and saddened by the fact that JavaScript has been eating the world. It’s not because young people are too stupid or too hip for their own good, it’s because a lot of languages just aren’t all that great. This is not to disrespect language designers — I have nowhere near the capability to be a language designer and implementer. It is precisely because it is so difficult to create a good language, that no such great language exists. Also, it takes time to get real-world feedback so that language designers can improve over time. By the time feedback arrives, the language designer is practically retired.

The reason why JavaScript is so popular is that most languages (1) cannot be run on multiple devices and platforms, (2) are not performant enough, (3) don’t have a good/versatile concurrency model, (4) have bloated and hard-to-master libraries, (5) only are useful in certain problem domains, and (6) are too fractured across problem domains and don’t have a big enough community to support users.

In other words, it was not just because JavaScript was a widely hated browser-based language that it took over servers, desktop and much of mobile. It was also because JavaScript is performant, has an easy-to-use concurrency model that does use multi-core processors, and, because it is interpreted, it is a bit easier to “run everywhere”. Less intuitively, since JavaScript does not have a module system specification, different JavaScript module systems were developed for different platforms, allowing it to run easily in different places. JavaScript is more ingenious than most people realize, whether the design was 100% intentional or not.

But … can we create something better than JavaScript?

I am looking for a language that I can use on servers, which I can also use to write mobile apps, desktop apps, web apps…and just maybe write a high-performance multi-player game with. It might be too much to ask, but I would rather have one language that I love to death and is supremely useful, than 3–5 languages, which I think are OK. Some criteria for a useful language:

  • It can be compiled/transpiled to run on multiple platforms, including UIs and servers. Easy transpilation to other languages is crucial to allow it to run in many places at the onset.

  • It allows importing more than 1 version of the same library/package name. On the job, so many people get stuck on older versions of libraries and then cannot easily upgrade sections of the codebase to newer versions of a library. Java, NPM, and other languages cannot easily upgrade library versions.

  • It makes writing asynchronous code easy — most runtimes have to handle an async event here or there, there is no avoiding it. The easiest way for the developer to avoid mistakes (and the need to put locks everywhere) is to design a language that doesn’t have (thread) pre-emption. For example, there is no pre-emption in Node.js — all your scheduled code runs first, then, when it finishes running, the event loop moves to the next tick. If Node.js had pre-emption, the platform wouldn’t have worked at all — the absence of pre-emption is one of the reasons why it’s such a powerful platform. I believe all JavaScript runtimes are bound to this constraint, but I am not 100% sure, (someone back me up?). On the other hand, Java has thread pre-emption and this makes asynchronous programming hard (especially to write libraries for general use). Pre-emptive software environments are simply bad, or morally wrong, if you will.*

To read about some of the pre-emption challenges in Golang, see:

  • It must compile to machine-code for performance. This of course does not prohibit the language from having a Virtual Machine as well, and being interpreted by that Virtual Machine instead of compiled to machine code (for the record, Java is compiled to bytecode then interpreted).

  • A VM/interpreter may be essential, because it is sometimes prohibitive to compile each and every program. Instead, for scripting we use interpreted languages because we can run programs without having to re-compile them for every machine.

  • It is structurally-typed*, not nominally-typed. Nominal-typing seems to offer few advantages to structural-typing, as far as I can tell. Structural-typing makes libraries less bloated and easier to write. FP languages are more often structurally-typed than OOP languages, and I think structural-typing is unequivocably better. Structural typing is especially useful for interoperability between libraries, otherwise both libraries need to import each other.

  • Has inferred* typing. Types should be inferred as much as possible, chaining and FP seem to help with this, whereas pure OOP seems to mean a lot of duplicate declarations.

  • It has no offside-rule — aka, no whitespace-as-syntax, aka no significant whitespace. Whitespace-as-syntax has to be one of the worst ideas in all language design, for obvious reasons. It is all about vanity and will end up killing you someday.

  • It is performant, with an easy-to-use concurrency model that can take advantage of modern multi-core processors. The event-loop model with async I/O seems to work really well, which is one big reason why JavaScript is so popular.

  • It has automatic garbage collection and memory management.

  • It avoids striving for hard-real time, instead going for soft-real-time. Hard-real time means pre-emptive or priority threading is used, which makes software development a lot trickier. Since 99.9% of software does not require hard-real-time features, this is something that a silver-bullet language can avoid, and therefore it can avoid pre-emption. Hard-real-time software might include control systems software, algorithmic trading software, etc.

  • It has the best of both worlds when it comes to functional/object-oriented, aka, why not both?

  • It has first-class functions. Template string literals. Anonymous functions, closures, etc.

  • It has solid language-level support for immutability and immutable data structures etc.

  • It has a good module system, packaging system, and dependency management system.

  • It allows for more than 1 version of a dependency loaded at runtime. Java does not allow this which makes for dependency hell — for a given class only one instance can exist, that’s how the classloader works.

  • The language itself makes writing 3rd party libraries easy.

  • It has a widely available and supported interpreter, since this makes WORA easier to implement.

  • It has easy deserialization/serialization. JSON is probably here to stay for awhile, but a lot of typed languages make it overly difficult to do I/O.

  • It is technically superior so that any barrier to entry is completely necessary, which has the helpful effect of keeping away unskilled, inexperienced, or uninitiated programmers.

  • I don’t think operator-overloading is much more than vanity (correct me if I am wrong), but I do think that method-overloading has advantages. Trying to find two or three or four different names for essentially the same routine can be annoying at best. Learning Golang has reminded me that not all languages have method-overloading — this makes me sad.

  • This is by no means a complete list and I will add more later.

*Regarding thread pre-emption : Java has it, Node.js doesn’t. I am currently investigating async runtimes like Erlang to see if Erlang processes can interrupt each other and therefore arbitrarily change the order of execution of scheduled operations. If you know about this please add a comment. Garbage collection is one task in most runtimes that could be pre-emptive.

In asynchronous development, it is critical that the order of execution is consistent. In this case, we expect step 2 to always run before step 3, no matter what happens. In pre-emptive threading environments, it is sometimes a danger that step 3 can be run before step 2. Pre-emptive environments usually require the developer to use more locks to prevent race conditions etc, which can be burdensome and error-prone. In the Node.js community, inconsistent APIs that can fire callbacks synchronously and asynchronously are known as “releasing Zalgo” or “summoning Zalgo”.

C++ I don’t know that much about C++ because I have never used it extensively, but I have read about it countless times. To my knowledge, C++ is very versatile and powerful, but lacks auto memory management which makes it dangerous to use. You can write desktop UIs, but writing UIs for Android, iOS, Web is not that easy, as far as I know. Clearly, the reason is that people choose to support other languages instead of C++ to run on their platforms.

Java Java is an excellent language for a lot of modern software. The biggest problem I have with Java is that it has a nominal/nominative type system instead of a structural type system — this is why the Java ecosystem is so bloated. Over time, tools like GWT for writing web apps proved to be fairly unproductive compared to using modern JavaScript. But Java is great for servers and Android. To this day, it seems that Vert.x is the best system for writing Java servers. Also, Java programs are not especially easy to compile, which has contributed to the fact that excess tooling is necessary to get Java programs running. This is a headache and major reason why so many Java devs don’t know much about the command line and are dependent on the IDE tooling to get anything done.

Golang Golang was promising but it is lacking in a couple of key areas. It is not very supportive of functional programming and lacks generics and inheritance which are useful OOP features. Golang does score major points with regard to being structurally-typed, proving that a structural type system can compile very quickly, however it requires redundant type declarations (see above) which other structurally-typed languages avoid (TypeScript and OCaml being some examples). Golang also has a somewhat restrictive circular dependency issue where two files in different packages cannot depend on each other (although perhaps all compile-to-machine-code languages have this restriction — OCaml seems to). So, Golang is good for writing simple high-performance servers — but what else? To read about some of the pre-emption challenges in Golang, see:

OCaml In my opinion, OCaml is the most promising of them all. However, the one thing it lacks is good concurrency. OCaml was created before the age of modern multi-core machines and never had a solid concurrency model, other than spawning another process. I believe it has threads but it also has a GIL. With ReasonML on the move, OCaml might be getting more mindshare. I see there are big efforts to improve OCaml concurrency including “multi-core” OCaml and binding the Libuv Event Loop engine used by Node.js to OCaml.

Erlang Erlang is cool — a high-performance dynamic language, like JavaScript. JavaScript and Erlang are the only two dynamic languages that make this list. However, without TypeScript and Dialyzer, they would have made it. There are a couple problems with Erlang: 1. It is useful only to build scaleable backends — not UIs or mobile apps, etc. 2. It has type-annotations (which Dialyzer uses) but those type-annotations are nominative not structural. Using 3rd-party libraries and trying to figure out how they work without types can be a headache. Type-annotations might help, but I think they would be better if they were structural annotations. We could build mobile apps with Erlang, like we do with Swift and Java, but without a real type system this could get pretty tricky.

F# F# is promising. It is a high-performance clone of OCaml and because it is Microsoft, it can borrow a lot of the nice libraries from Microsoft / C#. The problem is that F# is nominatively-typed because it borrows from existing MS libraries. Damn it! Still, nice developer tooling with VSCode. Unlike OCaml, F# uses optional significant whitespace.

C# C# is nominatively typed, like Java and F#. It is perhaps better than Java, but still has nominative typing, making it bloated just like Java. Like F#, it has nice developer tooling though.

Dart Dart is somewhat like TypeScript. One thing it lacks is method-overloading. It is not an especially functional language , at least no more so than JavaScript — it does not support immutability by default, for example.

Rust Along with OCaml, Rust is highly promising. One thing Rust does appear to lack is method-overloading — a feature that I think is useful.

TypeScript > JavaScript Besides OCaml, I believe that TypeScript and JavaScript are closer than most modern languages to reaching an ideal. TypeScript is an excellent solution to the fact that JavaScript is untyped. One major problem with JavaScript is that computation and floating point issues can make numerical libraries a bit unreliable. While JavaScript does have a good concurrency model, the concurrency model does have a couple of pitfalls that you need to mitigate — primarily errors thrown in asynchronously executed blocks. Ultimately, TypeScript can’t solve many of the fundamental problems that JavaScript has, although it is hands down the best type-system that I have ever worked with.

As we can see, none of the languages I have discussed meet all the critieria. The hardest criterion to satisfy seems to be structural-typing — only Golang and OCaml seem to meet it.

The Big Threat The obvious threat is that with more and more languages, mindshare is spread too thin. We need some genius to come in and create the language to kill all others, but any more mediocre languages are going to cause more harm than good. One of the biggest reasons why languages and libraries are good to work with is the community that can help when you hit a tough problem. When people stop using a language en masse it is effectively dead — a classic network effect problem. Identifying the counter-intuitive / lesser-known ideals

  1. I take it as self-evident that structurally typed languages are superior. Golang provides evidence that they can compile quickly and TypeScript shows us how wonderful structural typing is.

  2. It seems like transpilers can take the place of compilers in many cases, but in some cases people are wary of the added complexity. As far as I can tell, it’s better to have an interpreter like the JVM that can run everywhere, as opposed to transpiling many target languages. For example, with React Native I am wary of writing ReasonML that gets transpiled to JavaScript which then gets interpreted into native code via the bridge on Android and iOS.

  3. Why are there no compilers for interpreted languages and interpreters for compiled languages? It seems like this would allow for more Write Once Read Anywhere (WORA). Obviously, this would take work, but also forethought.

  4. Incremental compilers like tsc --watch for TypeScript are fantastic, why don’t languages like Java have incremental compilers that behave the same way?

Room for improvement

  1. Process startup time. With things like Lambdas in the cloud, a process startup time for Node.js processes of 300ms or so might be prohibitive. To improve theprocess startup time of interpreted languages we could create compilers for them. Likewise, to get compiled languages to run everywhere, we could create interpreters/VMs for them.

  2. Using non-preemption at the language/runtime level. From what I can tell, most languages allow for pre-emption which can arbitrarily change the order of execution. The developer expects code to execute in a certain order but when doing asynchronous development pre-emption can change that. The developer can handle pre-emption manually with locking (although this can be painstaking and error-prone) but writing libraries for general use in multiple environments can be a lot harder when pre-emption is something that needs to be worried about. Garbage collection in Node.js.

  3. Incremental compilation. With Java for example, you have to recompile everything. JRebel will hot-load code but only after it’s been compiled, which remains your job. As mentioned above, I would like to see more incremental compilation tools for compiled languages. The only incremental compiler I have used is for TypeScript. That means that you have some dev server, which holds compiled code in memory and listens for changes to files.

Conclusion Perhaps quantum computing will usher in this change. I would be willing to bet that only a big company will be able to get the adoption and the talent together to create a super-language.

Quantum computing aside, my untested theory is that OCaml is approaching the ideal. I have not used OCaml much yet, but it’s the next foray for me. If Java were a structurally-typed language, I think this conversation would be over. Alas, Java has a verbose type system with bloated libraries/ecosystem because the type system is nominal.

Scala is probably OK, but a bit slow to compile and not that useful outside of servers. Rust does not have automatic memory management and I believe it also has nominative typing. Lisp is untyped. Haskell too impractical. F# and C# are Microsoft and I tend to avoid MicroSoft languages because I don’t work on Windows — although it does appear that F# is a Microsoft clone of OCaml in the same way that C# was an MS clone of Java.

Is there a language that I am missing? Which language do you think is the most powerful and versatile? Can you imagine a programming language that is better than any existing ones?

27 views0 comments