One of the most interesting pieces of technology currently being developed at Sutoiku is our code loader. In a nutshell, it’s a way to dynamically load code from our code repository to deployed instances of our applications. Up until recently, it did the following:

• Looks up dependencies on a per-application basis
• Fetches the latest code from GitHub whenever the UI is loaded
• Builds and compiles the code in real-time
• Caches the compiled code with user-defined cache expiration

Today, we added the ability to select which branch in our master code base should be used, allowing different customers to use different branches. Tomorrow, we will also add the ability to manage such dynamic code branch referencing, in both centralized and decentralized fashions.

The centralized option will allow us to roll-out new versions of our code on an incremental basis, giving it to a handful of customers initially, then expanding the roll out gradually. The decentralized option will allow customers to manually specify which version of the code they want, usually selecting whether they want the latest version, or the most stable.

These mechanisms will be implemented at the application level, so that different applications can use different versions of the code, but we will also add the ability to apply changes for all applications at once. And while manual overrides for all this will be offered, the default behavior will be fully automated, so that it remains invisible to most users.

An interesting side benefit is that we will be able to give individual customers access to specific branches of our code, which will allow us to make rapid changes to pieces of our code that would be developed in partnership with some customers (the large deal we’re about to close will benefit from this).

As we’re adding these features, we’re also migrating from a monolithic architecture for our code loader, to one where applications only embed some kind of micro code loader, itself responsible for the dynamic loading of the full code loader. This will allow us to make changes to the code loader in the future, without having to manually upgrade applications one by one. There is a bit of a catch 22 to work around there, but we’re starting to get a fairly good sense of how we can make it work.

A couple of weeks ago, we did a lot of work on our code loader, which allowed our user interface to dynamically load itself whenever a user would log onto our application. It’s a great architecture, but it did not cover the code of the code loader itself, which means that we did not have an easy way of supporting its upgrade. And solving this problem was akin to breaking the chicken and egg cycle. Well, that was the situation two weeks ago, but things tend to change pretty quickly in the cloud, and today we demonstrated a way to dynamically load the code of our code loader as well. It’s not perfect yet, but it’s a lot better than what we had before, and we should have it working for our very first release. As a result, we will be able to dynamically upgrade 95% of our code, and make it super easy for our users to upgrade the remaining 5% whenever we make changes to the code loader itself (which should quickly become a relatively rare occurrence).

On n’arrête pas le progrès !

We made some improvements to our Code Loader, so that applications only load the code they really need, on the fly. And we made it so that individual components of our user interface can do the same as well. This extreme codebase modularization technique allows us to compile our code on the fly (while caching it when deployed in production), which makes incremental development and testing a breeze…

This morning, GitHub is down. Since it’s used by our User Interface to build itself when being logged onto, we can’t use our application. And of course, we can’t commit any new code either. This is a good reminder that failure should always be considered as default state in the Cloud, and that we need to push our code to multiple services at the same time, while improving our Code Loader so that code could be dynamically loaded from different services, based on their availability.

Bad news: I can’t implement the feature I had in mind.

Good news: I can write this post.

Better news: GitHub is back on even before I was done writing it.

Best news: We should be able to implement a fault-tolerant Code Loader.

A couple of weeks ago, we faced a major technical challenge: the user interface framework we decided to use lacked a critical API that would allow us to dynamically update our code so that software upgrades could be done on the fly, in a fully distributed fashion.

I proposed a clunky solution that I knew would work. Two other members of the team offered an alternative that was totally unproven. I argued against it, but they forged ahead anyway. After a couple of days of intense debugging and refactoring, I was proven wrong, to my utmost delight.

We now have an amazing Code Loader that:

• Looks up dependencies on a per-application basis
• Fetches the latest code from GitHub whenever the UI is loaded
• Builds and compiles the code in real-time
• Caches the compiled code with user-defined cache expiration

And all that is done so fast that the user can’t even notice the slightest lag. Not only that, but the necessary refactoring allowed us to work around a security restriction that prevented us from accessing certain APIs in specific scenarios. This was a totally unexpected side benefit.

I’m loving it…