Aanand Prasad, an engineer from Docker, spoke about Abstractions Over Scale and Docker’s open-source software suite at Leaseweb’s TechSummit in Amsterdam:
We wanted to get a little more insight into how tools like these are affecting the traditional ways of scaling technology:
Leaseweb: A lot of times you see a gap between the engineers who manage servers and the developers who write the code. Why do you think this issue continues to exist and what are the common problems people face trying to fit two different environments?
Aanand: I think the reason it persists is two-fold: firstly they are dramatically different environments and there are very different problems with completely different concerns, at least underneath. My quixotic endeavor is to make the case that they don’t need to be. Which is an uphill struggle, because usually the job of getting something running in production and the job of getting something running in development are done by completely different people, completely different teams even who might not spend much time talking to each other. It’s nonetheless my hope that with tools like Docker and with abstractions like containers and networks and volumes, we can get to the point where the differences between those environments are minimized.
I think that it could, given time, lead to completely different organizational styles within companies. There’s a lot of momentum behind the idea that developing software and deploying software need to be two different jobs. But as the underlying technology in both environments becomes more and more similar, the reasons for separating the jobs become fewer and fewer, and so that momentum will slow.
LSW: Do you think that this kind of abstraction can cause complications?
A: I think it is almost inevitable that when you bring in a new abstraction you’re not going to design it brilliantly from the start. That means that that abstraction is sometimes going to get in the way between the developer and their job (developing software), and between the operations person and their job (deploying software). It’s going to paper over things that just can’t be papered over. I think the solution to that is very boring, it just involves an incremental process of design wherein you improve upon the abstractions you have, you talk to people who are struggling with the abstractions that you’ve given them, and also there is going to be some degree of software coming the other way to sort of meet in the middle.
“Abstraction is sometimes going to get in the way between the developer and their job”
LSW: How would you recommend a company adopt a container-based approach”? By starting with one service or component to dip their toes in and moving on from there?
A: This is what we did at the startup I was at before we got acquired by Docker. That’s exactly how we embraced Docker, one thing at a time. At first we did it by, ‘we’ve got this one web process, let’s put it in a container.‘ So a few things changed; we’re no longer keeping a Python version up to date on a server somewhere, instead we’ve got Docker. We’re not installing a ton of Python packages on that server anymore because we’ve got Docker and the init script for the web process no longer says python it says docker run python. That’s a very small change. Other than that you have a process that behaves exactly the same and you take it from there. So, that’s a very very small step, it’s minimally disruptive and you can do that and then bring in Docker for another service and for another service and you can eventually get to a point where ‘ok, we’ve got quite a few services running on Docker now, do we really need the same infrastructure that we had before for deploying them or can we switch to using something like Docker Compose that will do all of that at once for us and keep all of the architecture and configuration in one version controlled place?’
LSW: Have you seen real world companies talking about how they’ve been able to grow faster using this technology and that they’ve found it helps them with their scaling issues?
A: Every week or couple of weeks I hear about a company that has embraced some or all of the Docker toolkit. Of course they’re using Engine, they might be using Compose, they might be using Swarm, they might be using Compose just for development or they might be using Compose with Swarm to deploy an app on a cluster. The overwhelming feeling is that ‘we love it.’
LSW: So they’ve been able to face growth challenges that they weren’t able to as easily overcome?
A: The main thing it’s enabled them to do is move faster. My feeling is that it’s not necessarily enabled people to scale up beyond what they had before – because I don’t think Docker is there to enable larger scale than you had before. When it comes to growing your app there’s the simple fact of ‘how is your software going to scale, how is your software going to work at scale, how is your database going to behave when you replicate it or cluster it or shard it, how is your load balancer going to behave?’ And those are problems that Docker does not solve because Docker does not dictate what your software is or does but what it does do is make it an order of magnitude easier to manage.
Watch all the TechSummit talks on Youtube.