Ansible to rule the world

by Mikel Larreategi

Our background

2001 ~ 2005: 3 shared servers
2005 ~ 2009: n VPS
2009 ~ today: m physical servers (20 < m < 30)
today ~ : ∞ cloud servers

When we started, we had an agreement with a local hosting company and we had there 3 servers where we could install our application ( they were plain Zope applications), but we had no sysadmins on our team, our clients bought the hosting service directly, we just deployed the application.
But our needs grew and we had to find something else. We found a US company called Zettai (later HighSpeedRails), that provided FreeBSD based VPSs where we started deploying the applications (we have already started selling Plone sites, but they were also Zope sites). The provider was specialized in providing hosting for python/zope/plone applications so, we did not have to worry about the low-level things.
But the company closed on 2009, and for at that time we started to think about providing us directly the hosting service to our clients.
We started looking around, and started buying physical bare-metal servers at a spanish provider (Dinahosting), that provides an excelent service and good prices.
But as everything has a but we had to start doing hosting services, so we needed to prepare the servers ourselves, provide backups, configure services, etc.
We started collecting our procedures in a document, but we still needed to do a lot of things manually.
We could have gone to a cloud provider, but we didn't find any which was a real good value for money. And no, the pricing system of Amazon AWS was crazy itself... We had many small clients, not one big project, so it wasnt' easy for us.
But after some incidents with our Backups, and some possible problems with the physical components of the servers, our system administrator was not confident enough to follow the bare-metal path, so we started looking for some cloud services; and found DigitalOcean, which offers well-priced cloud servers based on SSD disks, so we started migrating our bare-metal servers to DigitalOcean

But...

This was meant to be a nightmare

Automate, ok...

but WHAT to automate?

Application deployment

zc.buildout, fabric, ...

Already achieved

Our needs

Server provisioning
Configuration update/change

Ansible to the rescue

Ansible

Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchestration, and many other IT needs.

Ansible

Write a YAML file
Run it
Everything done

Ansible example


---
- hosts: all
  tasks:
    - name: Add user csmant
      user: name=mikel state=present shell=/bin/bash

    - name: Ensure additional packages via apt
      apt: pkg=nginx state=present

Extending ansible: Ansible galaxy

pypi for ansible
Roles for almost anything: package configuration, application deployment, swap files, exim configuration, ...

Our ansible

We ended having 5 main configuration files for our servers:

Common setup
Setup for Plone project
Setup for Django projects (Common + Mysql)
Setup for WordPress projects (Common - nginx + Apache + PHP + Mysql)
apt package update (we use Debian)

Our first time

After some tests, we run it on production
4 servers with special firewall rules
10 servers with old ssh keys
n incorrectly configured exim mail servers

Dynamic inventories

Dynamic list of hosts
No need to keep a stone-writen list of hosts
Everything automated

One of the drawbacks of ansible is that we need to keep an updated "hosts" file, that says which are our servers.
Fortunately, ansible has an option to use the so called "dynamic inventories" that are no more than python scripts which provide ansible the list of hosts where it needs to run things.
For instance we have a script that connects to Digital Oceans' API and gets the IP addresses of our servers, so we do not need to keep un updated-everyday-growing list of servers.
It is just a python script, so you can do it for whichever provider you use (there are premade scripts for mayor Cloud providers)
So we even get one step ahead: we even automated the list of servers.

(small) problems

Server specific setups
Different firewall rules
Domain-dependant mailserver configuration
Project specific requirements

Current status

We create all our servers using Ansible
We regularly run Ansible to ensure that nobody has changed the configuration
We keep updating our Ansible config file to address a few glitches

Future developments

Automate Plone hotfix installing
Automate django deployments
Automate nginx configuration
Autocontained projects: server setup + application deployment

Ansible to rule the world

Our background

But...

Application deployment

Our needs

Ansible to the rescue

Ansible

Ansible

Ansible example

Extending ansible: Ansible galaxy

Our ansible

Our first time

Dynamic inventories

(small) problems

Current status

Future developments

Q & A