Posts Tagged ‘DevOps’
Aanand Prasad, an engineer from Docker, spoke about Abstractions Over Scale and Docker’s open-source software suite at LeaseWeb’s TechSummit in Amsterdam:
We wanted to get a little more insight into how tools like these are affecting the traditional ways of scaling technology:
LeaseWeb: A lot of times you see a gap between the engineers who manage servers and the developers who write the code. Why do you think this issue continues to exist and what are the common problems people face trying to fit two different environments?
Aanand: I think the reason it persists is two-fold: firstly they are dramatically different environments and there are very different problems with completely different concerns, at least underneath. My quixotic endeavor is to make the case that they don’t need to be. Which is an uphill struggle, because usually the job of getting something running in production and the job of getting something running in development are done by completely different people, completely different teams even who might not spend much time talking to each other. It’s nonetheless my hope that with tools like Docker and with abstractions like containers and networks and volumes, we can get to the point where the differences between those environments are minimized.
This is the second part in a 2-part series. Read part 1.
In my last post I gave an overview of how we started the process of changing the way we do agile development at LeaseWeb by setting goals, engaging our engineering teams, and developing a maturity model to get everyone working to the same standard. In this post I’m going to talk about how we set up our scrum teams, how we get all levels of the business involved in the development process, and how we calculate the cost to provide the most value for the company.
Our current Scrum teams are set up as follows:
- Product Owner – defines the priorities of the team; responsible for the order in which features are built.
- Scrum Master – in charge of the scrum process (coach): making sure the team does retrospectives, sprint planning, refinement, coordinating meetings.
- DevOps – Development and Operations in one team of about 5-8 people.
LeaseWeb’s annual Amsterdam TechSummit took place on June 2 at the Pakhuis de Zwijger, an old warehouse converted to a high-tech multimedia event center. The summit was sold out with over 315 attendees who came to hear a variety of presentations from professionals focusing on this year’s theme: Designing for Scalability.
Those who attended were a diverse assortment of software developers, operations engineers, and managers from companies both large and small. Many of the attendees were local but a good percentage of them had traveled from other countries including Germany, Spain, and even as far as Liberia. All of them were looking to learn about ways to help them grow not only from a technology perspective but how to scale up their engineering teams and how to anticipate and deal with the issues that result from that growth. The summit also provided a good opportunity to network with peers and learn about the challenges they face and what they’ve learned from past mistakes.
In this blog post I’m going to cover how we are scaling our engineering department at LeaseWeb: where we started, the lessons we’ve learned, and how we are hoping to move forward. LeaseWeb was founded in 1997 and we currently have over 350 employees throughout the world with the majority working at our headquarters in Amsterdam. The engineering department currently has about 100 employees and manages 65,000 servers in seven locations (with more to be added soon).
In the past development was based on shifting priorities rather than planned business cases. Developers weren’t able to concentrate their efforts on thoroughly building, testing, documenting, and presenting a demo for one project before another more urgent one was moved to the head of the line. Operations and Development were working on different schedules and weren’t able to meet each other’s requirements to their mutual satisfaction. We wanted to change this.
Last year, we merged our existing operations and development departments into one Product Engineering department. Since then we have been focusing a lot on coaching all 13 teams and improving their effectiveness
In october last year, we attended an excellent talk by Bol.com at Velocity Amsterdam. In this talk they explained their ongoing transition towards DevOps. One of the concepts they introduced, was a maturity model to measure and incentivise continuous improvements within a team.
Inspired by the Bol.com talk, we have since developed and implemented a maturity model within our Product Engineering organization which consists of a matrix of four levels in four categories.
Throughout my career I’ve had the opportunity to work at a variety of different companies both large and small. They each had their own set of unique challenges regarding growth but one thing I noticed with time and experience was that the solutions to the problems they faced were not specific to the company itself. The approaches that were taken and the lessons that were learned could be extrapolated and applied to many of the situations facing a company looking to expand and grow technically.
There is a concept in some religions that before you save a sinner you have to tell them how they have sinned. In other words, if someone doesn’t know what the problem is they won’t be able to change. For a company just starting out, there are no wrong ways to build and deploy your app or product. Once you begin to grow however, you realize there are things you didn’t know and that some or all of the decisions that you made at the beginning were mistakes. This is the point where you need to decide how to address these issues. New companies are started all the time so I decided to draw from my experience to put together what I call the Seven Deadly Sins of Web Scale using seven real world examples from my career.