Ansible - useful tool to automate server configuration and deployment

In many cases configuring a new server is a repetitive task. Once the operating system is installed, we have to add accounts for administrators, install some basic packages, setup the application and secure everything. Even though there are some differences between servers designed for two different applications, certainly there will be some common things too. Especially in case of the same technology (not to mention exactly the same stack, like Symfony + nginx + MySQL).

For almost two years we've used Ansible at FINGO for preparing configuration of servers. This allowed us to spend more time on creative and interesting things, since the repetitive tasks were written once and now we can just include them from some common modules.

Our hero - Ansible

Ansible is a very powerful tool which mostly operates on special YAML files. A group of files (including tasks, variables and file templates) is organised to be a role designed for one particular purpose (like installing and securing ssh or creating users for administrators and uploading their public keys). More information about Ansible can be found in the introduction section of their documentation. It's evolving very fast with support for configuration of Linux, Windows and network devices.

Structure of the playbook

One of our use cases at FINGO is a fully automated server configuration that is able to expand a computing capacity of a running web application in a matter of minutes. I'll show an example of a simpler setup that consists of just one server.  Sample application will be called Simple within the scope of this post.

‍Simple has some requirements regarding the server configuration:

• PHP 5.6
• Newest version of MySQL / MariaDB
• Able to run a Symfony application
• PHPMyAdmin available
• Standard Firewall configuration with all (except ssh, http and https) ports blocked
• Production and test databases available

We decided to split the configuration using Ansible's role concept. This is how our playbook (defining what roles and in which order will be used) for Simple might look like:

---
- hosts: all                # Run a playbook against (in this case: all) a group of servers
  become: yes               # Use sudo for most tasks
  roles:
    - base                  # Install base packages, enable firewall, ...
    - base-users            # Add accounts for administrators with sudo access
    - mariadb               # Install and enable MariaDB, secure installation
    - nginx                 # Install and enable nginx with simple default html template
    - php56-nginx           # Install PHP 5.6 suitable for running with nginx
    - phpmyadmin-nginx      # Install PHPMyAdmin suitable for running with nginx
    - ssh                   # Send secure configuration for sshd
    - simple-configuration  # Configure directories, databases and users for the application
    - simple-deployment     # Deploy the application (download source code, run migrations)

Ansible runs those roles one after the other in the order that we have specified. Each role has its own tasks, templates, variables and handlers (triggering a restart of a service for example). Variables works just like in most programming languages: the most specific variable is used (i.e. the one defined within the role). However if the variable is not found, Ansible is searching for that variable in other roles (let's think about it as a global scope).

This means that we don't have to provide one MySQL root password for a mariadb role (which in that case would be used across all applications). We developed our roles with reusability in mind, so application-specific variables are defined within the scope of simple-configuration.

Inside a role

To visualize how one role might look like, let's take a look at simple-configuration role:

simple-configuration
  defaults
  files
    public_keys
      mateusz.pub
  handlers
    main.yml
  meta
  tasks
    databases.yml
    db_one.yml
    main.yml
    mysql_server.yml
    nginx.yml
    nginx_send.yml
    php.yml
    users.yml
  templates
    etc
      nginx
        conf.d
          simple.j2
    my.cnf
    php.ini
  vars
    main.yml

If our role is included in a playbook, Ansible runs tasks, starting with main.yml, but other tasks might be included too:

- include: users.yml
 tags: [simple_configuration, simple_users_configuration, users_configuration]

- include: php.yml
 tags: [simple_configuration, simple_php_configuration]

- include: nginx.yml
 tags: [simple_configuration, simple_nginx_configuration]

- include: databases.yml
 tags: [simple_configuration, simple_db_configuration, db_configuration]

- include: mysql_server.yml
 tags: [simple_configuration, simple_db_configuration, db_configuration]

And nested includes are also possible (a fragment of users.yml):

- include: users.yml
 tags: [simple_configuration, simple_users_configuration, users_configuration]

- include: php.yml
 tags: [simple_configuration, simple_php_configuration]

- include: nginx.yml
 tags: [simple_configuration, simple_nginx_configuration]

- include: databases.yml
 tags: [simple_configuration, simple_db_configuration, db_configuration]

- include: mysql_server.yml
 tags: [simple_configuration, simple_db_configuration, db_configuration]

Those items (for the loop) might be also defined within variables file. We prefer defining reusable and important variables there, as vars can be encrypted using ansible-vault mechanism. They're only decrypted during configuration of the server but remain secure inside of the code repository.

Roles that are correctly prepared, can be used in many projects. It's worth noting that there are many roles publicly available. To include, for example, ElasticSearch, we only need to add a git submodule to our roles directory (pointing to official repository with that role from creators of ElasticSearch) and include a line in our playbook:

{ role: elasticsearch, es_instance_name: "es_master_instance" }

That's it - we now can have ElasticSearch running on the localhost after provisioning of the server.

To run the playbook on the machine with IP address 1.2.3.4 we can use ansible-playbook command:

ansible-playbook -i 1.2.3.4 simple.yml --extra-vars "ansible_ssh_user=root" -t simple_nginx_configuration,simple_db_configuration

Ability to specify tags gives us the option to decide which tasks we want to run. It might be convenient to mark all tasks related to the deployment with one tag simple_deployment and provide it when running a playbook. It's also possible to exclude tasks labeled with some tags, which means this option is quite powerful.

Setup a new server

To fully automate things, we created Python scripts able to communicate with external APIs and create virtual machines. Starting a new server with Simple application running consists of running a script and waiting for 15 to 20 minutes. The script creates a new virtual server (parameters are specified through arguments), and runs two Ansible playbooks. One is for making sure that the server is ready to be configured (proper ssh port and public keys installed for root, for the first run) and the second is for configuring environment (as presented before).

It's great to know that we're able to setup a new machine in a matter of minutes and it's even better that our configuration is fully documented, just as a side effect.

One of the options

Ansible is not the only way for a better configuration. There are other configuration managers, such as Salt, Puppet and Chef. We've also tested some of them but stick with Ansible because if its simplicity and almost no requirements on the server side (Python and ssh are the only two).

We also like using Docker for some purposes. It's great to have so many great tools available and use them wisely, keeping in mind what is best for the project.