Remember the STOIC Pod? Well, it had a very short life, because it’s now called STOIC Rack, since the team (Hugues and Jim) would like to use the word “pod” for something totally different: the Kubernetes pod. It is defined as follows:
A pod (as in a pod of whales or pea pod) is a relatively tightly coupled group of containers that are scheduled onto the same host. It models an application-specific “virtual host” in a containerized environment. Pods serve as units of scheduling, deployment, and horizontal scaling/replication, share fate, and share some resources, such as storage volumes and IP addresses.
Since the provisioning of the STOIC Platform is so complex, it is really important that we all agree on a common terminology. In such a context, I’ve been pushing hard for the establishment of a standard deployment model, which we will now refer as a STOIC Rack. So, what is a rack?
First, a rack is something that is deployed in a single availability zone, within a single geographic location (US West Coast, US East Coast, Ireland, Singapore, etc.). As such, a rack is deployed within a single hosting provider, which is either Amazon Web Services or Digital Ocean. The reason why we’re still working with both is that Digital Ocean offers better performance at a lower price when using on-demand instances, and we’re not ready yet to use reserved instances.
When our achitecture becomes a bit more mature and our customer instances are used in production, it is likely that we will move from on-demand instances to reserved instances, and consolidate everything on AWS, but for the time being we will hedge our bets and work with both IaaS providers. Better safe than sorry…
Second, a rack is a unit of deployment where some resources are shared across tenants:
- Single Elasticsearch cluster
- Single copy of read-only datasources like Platform, Ontology, or Pelias
- Single inbound email server (for the email facet)
- Single Docker container for managing upgrades
- Single Docker container for monitoring the rack using cAdvisor
Alongside these shared resources, tenants will have their own Docker containers for Node.js and Redis. Initially, each and every tenant will have a single Docker container running both Node.js and Redis. But down the road we will add the ability to split Node.js and Redis across two separate containers, then deploy them across multiple containers for clustering purposes.
All this will be deployed on Docker containers which will be configured into pods in order to enforce better host affinity across resources. In turn, these pods will be deployed onto multiple virtual machines provisioned on AWS and Digital Ocean using Ansible.
Initially, the provisioning of these virtual machines will be done manually. But as we deploy more and more tenants on more and more virtual machines, we will automate this part of the work as well, even though we have yet to pick a tool for doing that.
The rack deployment model is what we will use for dedicated instances as well, including:
- Elasticsearch server
- Standard datasources (Platform, Ontology, etc.)
- Node.js server
- Redis queue
- Inbound email server
- Upgrade manager
- Monitoring tool
Of course, the configuration of a multi-tenant rack might differ from the one of a single-tenant rack as far as Kubernetes pods are concerned. And when you deploy a local development instance on your laptop, we might have yet another configuration. These packaging details are something that we will have to figure out over the coming weeks.
That’s all for now…