In anticipation to tomorrow’s poll of our backers and beta testers, all 200+ instances have been upgraded to the very latest version of our software (build 425, spreadsheets 59). Kudos to Hugues for having implemented this amazing provisioning framework on top of Cloud Foundry.
This is really, really cool…
As we’re deploying more and more early adopters, we’re building the provisioning infrastructure that will help us manage their instances. If you’ve read this blog long enough, you might remember that provisioning caused us a lot of headache in the early days. Since then, our architecture has evolved quite a bit, and the way we provision instances has changed as well.
Now that we have decided to use CloudFoundry, we have two distinct options at our disposal: multi-tenant or single-tenant. With the multi-tenant option, multiple tenants share the same Node.js server and the same database server, but each tenant has its own logical database. With the single-tenant option, each tenant has its own instance of everything: Node.js server, SQL database, NoSQL database, key-value store. Let’s take a look at the pros and cons of each option.
Multi-tenant
This approach is a bit more complex to implement at first, because you need to isolate tenants from each other at the application level. Nevertheless, we’ve done the hardest part of the work already, and it seems to be working pretty well. Once you’re done with it, adding and managing tenants is easy, because they share a lot of the underlying infrastructure. Nevertheless, sooner or later, this infrastructure will need to be scaled up, and you quickly end up with multiple copies of the same thing. So, to make a long story short, you get a little bit of R&D cost initially, then some benefits for a while, then less so as you get more tenants.
Single-tenant
This approach is a lot easier at first, because you essentially create one instance of the software per tenant, hence you have nothing to do at the application level. But the marginal cost of adding and managing a new tenant remains. It can be reduced over time with aggressive automation, but it can never be reduced to zero, as it could with a multi-tenant architecture. That being said, the approach also has some benefits, among them better isolation between tenants, which translates into higher security and better scalability (who doesn’t want that?). It also removes the need for an optional code container, because every single instance comes with one by default.
Strategy
So, which option are we going to pick? Well, at this point we don’t really know. Some of us would like to go for the single-tenant option, while others are more comfortable with the multi-tenant route. And we all have good reasons to back up our preferences. There is also a lot of things that we don’t know, and while we know that, we don’t know what we don’t know. As a result, we’ve decided to try both options, starting with the single-tenant model.
Practically-speaking, we will deploy our first early adopters with dedicated instances created on AppFog. As we do that, we will start building some automation tools that will allow us to upgrade all instances with newer versions of our code from a single click of the mouse, or a single command from the CLI (I always forget that sysops don’t have mice). We’ll also try to automate the instance provisioning process as much as possible. Then we’ll see how much manual work can’t be automated, and try to put a cost figure on it. If that cost remains within a certain range (maybe a few percents of the average revenue per customer), we’ll call it a success and stick to the single-tenant model for as long as we can. If that cost becomes prohibitive, we’ll immediately revive our multi-tenant efforts. In the end, we might have both options, a little bit like WordPress, which comes in two flavors (WordPress and WordPress Multisite).
Benefits
What does it mean for you? Well, if we succeed at implementing a proper single-tenant instance provisioning and management infrastructure, you (if you back our Kickstarter) will get your very own Node.js server, PostgreSQL SQL database, MongoDB NoSQL database, and Redis key-value store. You’ll also get 2GB of RAM, and 100MB of storage space on PostgreSQL, even if you just have a single user subscription ($25/month). Not only that, but you’ll get a full AppFog account created for you, from which you’ll be able to add more memory, more storage space, and more add-ons. And you’ll be able to use all this through your own domain name.
Wish us luck!
From there, your spreadsheets magically turn into fully-fledged applications.
Then, you will create your first STOIC application from Google Drive. It could not be simpler.
Once we go live, you will install STOIC from the Chrome Web Store.
Since we started working on our platform, easy provisioning has been our number one priority. We’re aiming our product at a truly mainstream audience, with the goal of making it accessible to non-technical users (while keeping it attractive to the most hardcore JavaScript developers). Consequently, one should be able to install it with a few clicks of the mouse, in a few seconds.
The challenge in implementing such a streamlined provisioning process is that we need to hide the complexity of provisioning accounts across the multiple cloud-based services that we’re using as underlying infrastructure, and some of them were never designed to support such a use case.
Up until last week, we had completed three major iterations, making the end-to-end process incrementally easier and faster at every turn of the crank. But after the last iteration, we were still not fully satisfied. The process required too many steps, took too long to complete, and exposed too many technical details to the end users.
As a result, Jacques-Alexandre decided to take another crack at it while I was taking some time off the grid. While a few things remain to be done, early indicators are extremely encouraging. As it stands, the entire platform can be provisioned as easily as it is to install an iPhone app, and creating a new application with our platform now requires nothing more than a click on a single menu item, from an application that over 400 million people already use today.
If all goes as planned, we should have a screencast of the end-to-end process in a few weeks.
We succeeded in automating our application provisioning process, and it’s totally awesome. The process is flawless, fast, and brain-dead simple. It’s so simple that it does not even require any login. It feels a bit magic in fact. As described earlier, here is the end-user workflow:
When a new user account is created, an email is automatically sent to the user and to us (for tracking purposes). Similarly, we receive an email whenever an individual application is activated as well. The emails we receive contain JSON fragments so that we can parse them easily. And tonight, we demonstrated the end-to-end process on an external domain, thereby proving that it works outside of our organization.
Quite frankly, before we got started, I was afraid that this thing would take us a week or two. Instead, it took less than two days, for a single developer… The productivity we’re getting from using the right cloud services never ceases to amaze me…
Awesome opossum…
Our end-to-end provisioning process is almost there…
After a full day of work, our provisioning automation is essentially working. We still need to fix a few minor issues and polish the user interface, but the end-to-end use case is fully functional. It’s also totally user friendly, which was our primary goal. The end-user workflow is as follows:
A small bug in the platform we’re using for our user interface is preventing us to both install and activate all applications at once. Instead, applications are installed in one shot, but still need to be activated individually, by clicking on a ‘Activate’ menu item when used for the first time. There is a side benefit to it though: it will allow us to keep track of when users activate the platform and when they activate individual applications, which is quite nice.
Two things were absolutely fascinating to me in this overall process:
And once again, the initial code hacked by our COO paved the way for a really cool feature. A pattern is shaping up nicely here: our COO hacks a quick and dirty proof of concept, our CTO improves upon the initial design, I do the bulk of the implementation, and one of our developers scales it up with many more features.
It will be interesting to see how we scale this process with a team in China…
“Everything should be made as simple as possible, but no simpler.”
— Albert Einstein
Our latest attempt at simplification went a bit too far. In trying to remove a couple of steps from our provisioning process, we ignored a few undesirable side effects. As a result, we’re back to our original idea, for which every step has been proven to work. Now, we just need to put them all together into an end-to-end process and verify that it works in most cases.
Just a few minutes ago, we thought that we had streamlined our provisioning process as much as was technically possible. But that was without counting on the problem-solving genius of our CTO, who just showed up at the office and turned his brain on.
Before CTO, our planned provisioning process was:
After CTO, our revised provisioning process is:
We have yet to prove that it works, but it sure looks promising…
Bad news: our work on provisioning can go to the recycling bin.
Good news: it’s for the best, because we devised a simpler process.
How comes? Well, it seems that a major bug/limitation in the platform we’re using for our user interface has been fixed overnight and made possible today what was impossible yesterday. To be more precise, a bug has been fixed and a limitation has been lifted, allowing us to do what we could only dream of just twelve hours earlier.
So, we went back to the drawing board and defined a streamlined process, which does not require this awkward browser extension (aka plugin) anymore (I can hear a few sighs of relief). We split the effort among the three of us who are at the office this morning and really hope to have something working by the end of the day. If everything goes as planned, the provisioning process should look like that:
No emails, no logins, no non-sense. Pure and simple…
PS: No, bits cannot be recycled for real life (as my daughter would say).
Since yesterday, we’ve been working fast and furious on automating our instance provisioning process. As described earlier this morning, we must automate four specific steps. A proof of concept for Step #1 was completed on Monday. Yesterday afternoon, we did the same for Step #2. And this morning, Step #3 is working as well. We’re down to the very last step, of which about 10% was demonstrated yesterday night. So, if luck is on our side, we should have the end to end scenario prototyped sometime this week.
“Anything that can fail in the cloud, will fail.”
This principle should be kept in mind whenever building an application in the cloud. This is especially true when building a composite application out of multiple cloud services, for such an architecture dramatically increases the aggregate number of single points of failure.
In Sutoiku’s case, our Achilles’ heel will be our provisioning system, which relies on unstable APIs and web scrapping techniques that will break whenever some third-party user interface is changed. As a result, we must assume that things will fail, and build into our platform some graceful failure handling mechanisms.
What this means is that any feature relying on such brittle connection points should be backed-up by an alternative implementations. They might not be as user friendly, but they will let people get things done whenever things go wrong, even if that means reading a bit more documentation and following a few manual steps. The show must go on…
Jannis Rake-Revelant had an interesting question about our application: if it’s in the cloud, what does “installing” it mean? Indeed, if it’s in the cloud, people assume that there is no software to install, and from the end-user’s perspective, this is usually correct. But from the vendor’s perspective, things can be a bit more complex.
If the application is monolithic and natively multi-tenant, provisioning new customer accounts is usually fairly straightforward for the vendor. But if the application is built upon multiple pre-existing cloud services that have different ways of dealing with multi-tenancy, provisioning can quickly become an interesting puzzle, involving the creation of multiple accounts or sub-accounts, across multiple services.
Furthermore, if the vendor’s application is based on some kind of templates that can be customized by end-users, provisioning also involves the cloning of such templates. And while some platforms might offer proper APIs for doing so, others clearly don’t.
In our particular case, we require at least three different services for deploying Sutoiku applications: a user account on GitHub for managing source code, a runtime service for our server (currently using Heroku), and a user account on the platform we’re using for our user interface. The first two are fairly straightforward, but the third one is proving to be a major challenge, mainly because it’s new and relatively immature.
At present time, automating the end-to-end provisioning process for the user interface component requires four automated steps. Step #1 is handled with a clean integration using a published API. Steps #2 and #3 are using another API and are a bit more cumbersome. Step #4 cannot use any API and will require the use of some web scrapping technique.
As of today, our best bet is to build a tiny browser extension (aka plugin) that would handle Steps #2 through #4. It would take about 30 seconds to install and would be secured through OAuth 2.0 single-sign-on. It would be used initially for creating a new instance of our platform, then later on for installing any new application. It would turn a 15 minutes manual installation process into a fully-automated, one-click provisioning procedure. Like everybody else, we don’t really like browser extensions, but in cases like this one, they tend to be the lesser of many evils.
This browser extension will be mainly used for provisioning and won’t be required for using our applications. Nevertheless, it will allow us to implement a few optional features that will significantly improve the overall user experience whenever it is enabled on the end-user’s browser. So, all in all, it’s not a bad solution.
We should know by tomorrow if our proposed solution actually works.
