I Have a Dream

16 Mar 2012, 13:49 UTC

An imaginary sales pitch from Reanimator Ltd, circa 2015 ...

Software architecture is hard. Programmers are fantastic at working within small self-contained units of work, especially when they can start with a "clean slate". But most large-scale software is poorly architected, because an architecture that made sense at the start of a project may not make sense towards the end, but architecture touches so many places in the source code that it is very hard to "refactor" your way to a modified design.

Equally, programming languages are very good at allowing programmers to express self-contained units of work, which run in a straightforward way on a single CPU, or even on multiple CPUs (for a fixed slice of time). There is little point today producing a rival to C++ in terms of writing this kind of code, and HLSL hits a particular sweet spot in terms of writing code that can run on a GPU. If a unit of work is small enough, it is easy to modify in terms of adding features, or optimising the code. In the limit one can even rewrite the entire piece, and if it's small enough this is the work of only a few hours or days.

What traditional programming languages are very bad at is architecting the large-scale operation of software which is made up of small pieces of work like this. Controlling resources in an efficient and effective way, handling asynchronicity, handling "impedance matches" between different libraries, are all aspects of a program which are more akin to "plumbing" than to the purer and simpler world of algorithms. These aspects can be handled in a variety of ways, but they pervade practical programs, touching the code here there and everywhere, so that changes in these aspects may involve edits in hundreds or even thousands of different places. Such changes can take days to implement and weeks to fully debug; and this kind of change is hard to do well, and may involve much introspection before even beginning.

An example of a simple architecture change is to move some processing out of the runtime software and into the project-wide data conditioning pipeline. This generally involves moving code between two different programs, ironing out impedance mismatches with the different runtime environments, and possibly refactoring parts to be more multi-thread friendly or to take advantage of the new environment somehow. Then we might need to introduce loading code for the new data, and perhaps remove loading code for the old data. It's a lot of work for what, on paper, is a very simple conceptual change - one module is now run on the server, not on the local computer.

It seems to me that we need to consider writing programs on two levels. On the one level are the bread-and-butter algorithms and transformations which programs perform, which are well understood, and for which existing languages are more than adequate. On the other level are the architectural decisions - the "plumbing" which puts these algorithms together to make a product.

I maintain that there is no language in existence which does a good job at solving this plumbing problem. This is because all computer languages start with the assumption of sequential execution of basic operations - that their core job is to put together chunks of assembly code which run algorithms. So they focus on sequential, step by step code. When they hit problems of asynchronous execution they then insert band-aid solutions which fit into the step-by-step code concept, but which are no help whatsoever with architecture at the level of asynchronous execution. In other words, they begin to fail.

It is at this level that our language comes into play. You write some algorithm on CPU and you write some algorithm on GPU, and keep the code nicely self-contained. You wrap the code up in some XML (or just write it in our editor) and we can compile that code for you, for various platforms, with some added wrapper code to keep our system happy. Now, you use our system to wire these algorithms together. We notice the CPU->GPU move, and allocate data areas in the correct places, and handle synchronisation and other issues. But most importantly the resulting code can be interleaved with other operations, keeping both CPU and GPU fully occupied as far as possible.

Then, you can rewrite your GPU algorithm on CPU, and set a dropdown box, and all the CPU->GPU code will go away, leaving a simpler interface. You can mark that algorithm to run on a server somewhere, and off it will go. You can place a "cache" between one algorithm and the other, declaring that the output of the first algorithm should just be stored and loaded when the program starts. The algorithm will move onto your data conditioning pipeline and be run there - right on the server - whenever it needs refreshing; meanwhile, the code will be injected into your program to load the results from the pipeline (from the server in development; from disc in production).

In this way, major architectural changes can be invoked with literally just a few mouse movements and clicks. A system that runs fine on a PC with a powerful GPU can be retargeted to a system like PS3 with a complete focus on the low-level programming task (porting some GPU method to SPU, say) and no work at all to be done on the plumbing side. At best, if you're using a library that already has these different versions, you can click away to your heart's content, moving various parts from one processor to another, testing the many possible combinations, until you find a sweet spot. You can try and fit everything into one frame to save memory, or you can spread processing out over multiple frames, to maximise processor usage, or some combination - and maybe a very different combination per platform. This is a level of architectural manipulation which is simply not possible without our tools.

A big advantage of this approach is for middleware. Some middlewares are complex and hard to integrate, requiring insertions into the asset pipeline, the game engine, the data formats on disc, etc. etc. For these kinds of middleware we offer a great advantage. You can provide a set of components and some rough guides for integration. The client simply plugs these components into the appropriate place in his software. We will automatically include "impedance-matching" adapters - to convert data formats etc. so the user gets a plug-and-play "instant gratification" type of experience. Later, as the integration is refined, these impedance-matching adapters may be removed, or at least moved out of runtime.

Our programming system is aimed at games (specifically, at software which uses the GPU and draws its own GUI, and which has complex and heavy-weight offline processing requirements, as well as multiple asynchronous runtime tasks handling internet, disc, animation, GPU control, audio, etc.). It also applies to embedded systems in a big way e.g. DVD players, PVRs, smart TV, phone, the games console itself, etc. The same system is also ideal for all kinds of server software (phone switches, video transcoding farms, smart web services, database frontends, web spider data processing, real-time analytics and event-based processing) and all kinds of application software backends (iTunes-style apps, photo and video editors, web browsers, 3D modelling packages, audio & music programs, demonstrations and interactive educational programs). Ultimately we aim to revolutionise the entire practice of software creation, through our identification of the two levels of software, and through our tools which are tailor-made to cope with the level that is undersupported elsewhere.