Building applications that scale through proper scheduling
As we transtion to building distributed systems, we need to shed some of the anti-patterns of web application development that we have applied for ages.
The ease of compute access made most of us think that we can process every job interactively, and when things slow down, we just scale by virtue of adding instances. Now when that wasn’t the case, and scaling compute actually meant that a real estate operation was required (AKA “get more room to get more mainframes”) back in the day, we actually had process differentiation. Batch jobs would run on specific time windows when CPU time was available.
Now traditionally, if your application is super complex, you have most likely been through this. You are probably using a message queue or ESB. This post might not be for you. Now if you are still just piling up code in Rails or Node, there are things that you can probably offload as batch (or in my case, parameterized dispatch, but I’ll explain that later). So here I’ve got this library app that I use at home to keep track of my books, and adding a book is, let’s say, a lot of work.
Now from a user perspective, sure I’m going to see the spinning wheel for a while, but from a compute perspective, it is even worse as I’m wasting a thread that could be used for interactive traffic. Then again I could spawn a thread in the system, but the Linux scheduler has a somewhat “local” view of the world as it would be expected.
So how about a scheduler that has a more global view? Weren’t you all using Kubernetes?
Well not me, clearly, but then again this pattern would work with any scheduler.
Nomad has this functionality of dispatching a previously defined job, with an added parameter. So if I wanted to do a “distributed” ping
and I can dispatch it using the CLI like so:
And the result would be something like:
In my case… I’d spawn a Nomad Job upon receiving an API Call:
As you can see there I’m getting a Consul Token that I’m passing to the UI (it lasts for a minute) and I’m getting another Nomad token from Vault (which expires in 5 minutes) to do the request.
Final result looks fantastic
About the author:
Nicolas Corrarello is the Regional Director for Solutions Engineering @ HashiCorp based out of London.