At STOIC, we positively love open source. We’re using quite a bit of it to build our own software (Cf. Foundations), and we try to contribute back as much as we can. As part of our long-term intellectual property strategy, we’re planning to open source most of our server-side components. In the meantime, we’ve released a couple of smaller open source projects: jsdox is a JsDoc to Markdown generator for Node.js, and parseq.js is a simple JavaScript flow control library, as simple as possible, but no simpler. Our Server (stoic.js) and our Object Data Mapper (mapper.js) should be released within a few months. In the meantime, you can learn a little bit more about them from our architecture page. All our open source projects are released under the MIT License and hosted on GitHub Repositories (as you would expect).

PS: If you want to apply for a job, please contribute to our open source projects first.

After a discussion with our CTO, it seems that my understanding of how the Object Data Mapper works was not entirely accurate. Contrary to what my previous post indicated, full data synchronization for virtual objects is required only when the third-party object exposed by the virtual object does not provide a rich enough API to provide direct integration. But if it does, the virtual object can be implemented in such a way that no data synchronization is required at all. For example, integrating with Google Tasks requires synchronization, but integrating with Salesforce.com Opportunities does not. Pretty cool stuff…

When building a development platform, the temptation to build as many things on top of it as possible is great. After all, if the platform is really working, anything built on top of it should work a whole lot better than something that is not. One should resist this temptation though, for its underlying assumption is usually wrong.

Let’s take the example of Sutoiku Activities. It is made of three objects (Event, Project, and Task), but only one is implemented as a native Sutoiku objects (Project). The other two (Event and Task) are syndicated from Google Apps (using the Google Calendar Services and Google Task Service). The reason for it is pretty simple: we don’t stand a chance of convincing anyone to move from Google Calendar to an hypothetical Sutoiku Calendar, no matter how hard we might try. The same is true for Gmail, Google Drive, and many other Google Apps components that we’re in the process of integrating with.

From a strategic standpoint, our approach is to leverage whatever already exists out there and is good enough, and focus all our efforts on building the missing pieces and integrating them within their surrounding environment. For example, a Google Task can reference a Sutoiku Project through a 1-N relationship. And from the viewpoint of a Sutoiku developer, a Google Task actually looks very much like a Sutoiku Task (Cf. Object Data Mapper).

For our first product release, we initially focused on Google Apps as default underlying platform. If you’ve been following this blog closely since the very beginning, this is the not-so-well-kept secret that we’ve been keeping all along. Down the road, we might support additional platforms (Microsoft Windows + Office comes to mind), but for the time being, Google Apps defines the surrounding environment describe above. Here is how we’re using it:

• GMail as universal inbox
• Google Spreadsheet for the object view (list of records), and many other cool things
• Google AngularJS for the record view, the query builder, and other UI components
• Google Contacts for Contacts
• Google Groups for Groups and Roles
• Google Calendar Services for Events
• Google Task Service for Tasks and Task Lists
• Google Docs as Content Management System
• Google Drive as Document Management System
• Google Apps Control Panel as administration panel
• Google Domain as domain manager
• Google Chrome Web Store as marketplace

If we do our job well, users should not see much of Sutoiku, if anything at all. Instead, they should feel like they’re using a better, more powerful version of Google Apps — Google Apps on steroids. And as Google adds new services to Google Apps, we will integrate with the most relevant of them, constantly improving the overall end user experience.

Now that the cat is out of the bag, we better deliver something soon…

Our Object Data Mapper (ODM) is starting to take shape. As its name implies, it’s a middleware layer responsible for mapping data entities to JavaScript objects. Mapping in this context is a little bit misleading, since we’re not starting from a JavaScript object and mapping to it. Instead, we’re staring from a data entity managed by any datastore, then projecting it to our object meta-model. As a result, the object appears as a first-class JavaScript object, for which a CRUD API is made available automatically.

The data entity is managed by a datastore. What’s unique about our ODM is that datastores can either be a SQL database, a NoSQL database, or an external application, while all look exactly the same from the viewpoint of the developer using objects exposed by them. For external applications such as Google Apps, virtual objects (eg. Contacts, Events, Tasks, etc.) are actually duplicated into our SQL database and automatically synchronized in a bi-directional and real-time fashion. This duplication is made necessary by the fact that not all applications provide a rich enough API (especially with respect to queries), and some impose certain constraints from an access control standpoint that make direct integration simply impossible.

Implementing a new datastore for a database or an application is a fairly straightforward exercise for an experienced JavaScript developer and just takes a few hundred lines of code. Furthermore, once a datastore has been implemented for an object of a given application (we’re almost done with Google Tasks and Task Lists), adding support for another object of the same application (Google Contact or Google Calendar Event for example) is incrementally easier.

Another interesting aspect is that the ODM does not stop at the CRUD API. It also provides support for relationships that can span actual and virtual objects (eg. a Task referencing a Project) and support for advanced queries (of the kind generated by our query builder). Last but not least, the API exposed by the ODM is available not only from the Node.js server-side runtime, but also from the JavaScript code embedded within web pages and running on the web browser, in a way similar to what Meteor is doing.

UPDATE: More details on data synchronization on this later post.

The most critical component of our server will be its Object Data Mapper, which will be responsible for mapping the records of an object to a database (Cf. the databases we like), be it relational or not (SQL or NoSQL). Initially, we started with Node-ORM. Over time though, we found ourselves rather limited by its codebase. Recently, we decided to start from scratch instead. This will allow us to support the rather unique lifecycle of our objects and records, while implementing the mechanisms we need for data synchronization. The latter are rather fancy, mainly because of our unconventional three-tier architecture. This project, which is currently in its early stages of development, will eventually be released under MIT License.

A reader just asked me which database we prefer at Sutoiku. Clearly, there are plenty of options to chose from in this NoSQL era. Nevertheless, we’re still very fond of SQL, and PostgreSQL is by far our preferred choice there.

Nevertheless, our upcoming server will most likely have to support Oracle and SQL Server, at least as soon as we can afford the porting, certification, and maintenance costs (never to be underestimated). In the meantime, our CTO is making sure that our Object Data Mapper (ODM) is clean enough to support these deployment options.

On the NoSQL front, we will start with MongoDB and Redis. On the longer term, CouchDB and Riak are attractive options as well. But what gets the database geeks among us really excited nowadays is Google F1. If Google were to make F1 available to the public, we would certainly be among the first to give it a spin. Anyone from the F1 team reading this?