A State contains a set of instructions which we would like to be applied to our EC2 minion(s). We will use /srv/salt/states on the minion as the root of the Salt State tree. States can be stored in there in the form of a single file, for example /srv/salt/states/mystate.sls, or organized into folders like so /srv/salt/states/mystate/init.sls . Later on, when we request that mystate is executed, Salt will look for either a state_name.sls or a state_name/init.sls in the root of the State Tree. I find the second approach tidier as it allows for other state-related files to be kept in the relevant folder.

We begin the Configuration Management of our web server node with a state for managing Linux user accounts. Inside our Salt Git repository, we create states/users/init.sls:

    veselin:
      user.present:
        - fullname: Veselin Kantsev
        - uid: {{ salt['pillar.get']('users:veselin:uid') }}
        - password: {{ salt['pillar.get']('users:veselin:password') }}
        - groups:
          - wheel
    
    ssh_auth.present:
      - user: veselin
      - source: salt://users/files/veselin.pub
      - require:
        - user: veselin
    
    sudoers:
      file.managed:
        - name: /etc/sudoers.d/wheel
        - contents: '%wheel ALL=(ALL) ALL'

We will use YAML to write most Salt configuration. You will notice three different state modules used in the preceding section:

To avoid hardcoding certain values under user.present, we make use of the SaltStack Pillars system. We will examine a pillar file shortly, but for now just note the syntax of referencing pillar values inside our state.

Two other points of interest here are the source of our key file and the require property. In this example, a salt:// formatted source address refers to the Salt File Server which by default serves files from the State Tree (for supported backends, please see https://docs.saltstack.com/en/latest/ref/file_server/). The require statement enforces an order of execution, ensuring that the user account is present before trying to create an authorized_keys file for it.

Thanks to the readability of YAML, one can easily tell what is going on here:

Perhaps you could try edit this state and add a user for yourself? It will be useful later after we deploy.

We will now move on to an NGINX installation via states/nginx/init.sls.

We install NGINX using the pkg.installed module:

pkg.installed: []

Set the service to start on boot (enable: True), enable reloading instead of restarting when possible (reload: True), ensure the NGINX pkg has been installed (require:) before running the service (service.running:)

    nginx:
      service.running:
        - enable: True
        - reload: True
        - require:
          - pkg: nginx

Then put a config file in place (file.managed:), ensuring the service waits for this to happen (require_in:) and also reloads each time the file is updated (watch_in:):

    /etc/nginx/conf.d/default.conf:
      file.managed:
        - source: salt://nginx/files/default.conf
        - require:
          - pkg: nginx
        - require_in:
          - service: nginx
        - watch_in:
          - service: nginx

Note the require/require_in, watch/watch_in pairs. The difference between each of these requisites and its _in counterpart lies in the direction in which they act.

For example:

    nginx:
      service.running:
        - watch:
          - file: nginx_config
    nginx_config:
      file.managed:
        - name: /etc/nginx/nginx.conf
        - source: salt://...

Has the same effect as:

    nginx:
      service.running: []
      nginx_config:
      file.managed:
        - name: /etc/nginx/nginx.conf
        - source: salt://...
          - watch_in:
            - service: nginx

In both cases, the NGINX service restarts on config file changes; however, you can see how the second format can be potentially quite useful the further you get from the service block-say in a different file, as we will see in the next state.

Add in some PHP (states/php-fpm/init.sls):

    include:
      - nginx
    
    php-fpm:
      pkg.installed:
        - name: php-fpm
        - require:
          - pkg: nginx
    
    service.running:
      - name: php-fpm
      - enable: True
      - reload: True
      - require_in:
        - service: nginx...

Here you can better see the usefulness of an _in requisite. After we include the nginx state at the top, our require_in makes sure that nginx does not start before php-fpm does.

With NGINX and PHP-FPM now configured, let us add a quick test page (states/phptest/init.sls).

We set a few variables pulled from Grains (more on those shortly):

{% set publqic_ipv4 = salt['cmd.shell']('ec2-metadata --public-ipv4 | awk '{ print $2 }'') %} 
{% set grains_ipv4 = salt['grains.get']('ipv4:0') %} 
{% set grains_os = salt['grains.get']('os') %} 
{% set grains_osmajorrelease = salt['grains.get']('osmajorrelease') %} 
{% set grains_num_cpus = salt['grains.get']('num_cpus') %} 
{% set grains_cpu_model = salt['grains.get']('cpu_model') %} 
{% set grains_mem_total = salt['grains.get']('mem_total') %} 

Then we deploy the test page and add contents to it directly:

phptest: 
  file.managed: 
    - name: /var/www/html/index.php 
    - makedirs: True 
    - contents: | 
        <?php 
          echo '<p style="text-align:center;color:red">  
          Hello from {{ grains_ipv4 }}/{{ public_ipv4 }} running PHP ' . 
          phpversion() . ' on {{ grains_os }} {{ grains_osmajorrelease }}.  
          <br> I come with {{ grains_num_cpus }} x {{ grains_cpu_model }}  
          and {{ grains_mem_total }} MB of memory. </p>'; 
          phpinfo(INFO_LICENSE); 
        ?> 

We will use this page post-deployment to check whether both NGINX and PHP-FPM are operational.

Now let us look at the main mechanism for storing variables in Salt-the Pillars. These are:

We will be using /srv/salt/pillars as the root of our Pillar tree on the minion. Let us go back to the users state and examine the following lines:

- uid: {{ salt['pillar.get']('users:veselin:uid') }} 
- password: {{ salt['pillar.get']('users:veselin:password') }} 

The uid and password attributes are set to be sourced from a pillar named users. And if we check our Pillar Tree, we find a /srv/salt/pillars/users.sls file containing:

users: 
  veselin: 
    uid: 5001 
    password: '$1$wZ0gQOOo$HEN/gDGS85dEZM7QZVlFz/' 

It is now easy to see how the users:veselin:password reference inside the state file matches against this pillar's structure.

Unlike Pillars, Grains are considered static data:

We have already made good use of Grains within our preceding test page (states/phptest/init.sls), getting various host details such as CPU, memory, network, and OS. Another way of using this data is when dealing with multi-OS environments. Let us look at the following example:

pkg.installed: 
  {% if grains['os'] == 'CentOS' or grains['os'] == 'RedHat' %} 
    - name: httpd...  
  {% elif grains['os'] == 'Debian' or grains['os'] == 'Ubuntu' %} 
    - name: apache2 
  ... 
  {% endif %} 

As you see, Grains, much like Pillars, help make our States way more flexible.

We now have our States ready, even supported by some Pillars and ideally would like to apply all of those to a host so we can get it configured and ready for use.

In SaltStack, the Top File provides the mapping between States/Pillars and the minions they should be applied onto. We have a Top file (top.sls) in the root of both the state and pillar trees. We happen to have a single environment (base), but we could easily add more (dev, qa, prod). Each could have a separate state and pillar trees with separate Top files which get compiled into one at runtime.

Let us look at a top.sls example:

base: 
  '*': 
    - core_utils 
    - monitoring_client 
      - log_forwarder
  'webserver-*':
    - nginx
    - php-fpm
  'dbserver-*': 
    - pgsql_server 
    - pgbouncer 

We are declaring that in our base (default) environment:

Minion targeting gets even more interesting when you include Pillars, Grains, or a mix of these (see https://docs.saltstack.com/en/latest/ref/states/top.html#advanced-minion-targeting).

By specifying such state/pillar to a minion association, from a security standpoint we also create a useful isolation. Say our Pillars contained sensitive data, then this is how we could limit the group of minions who are allowed access to it.

Back to our Salt repository, where we find two top.sls files:

base: 
  '*': 
    - users 
    - nginx 
    - php-fpm 
    - phptest 

base: 
  '*': 
    - users 

We can allow ourselves to target *, as we are running in masterless mode and essentially all our States/Pillars are intended for the local minion.

We enable this mode with a few settings in a minion configuration file (/etc/salt/minion.d/masterless.conf).

These effectively tell the salt-minion process that the Salt Fileserver, the state tree and the pillar tree are all to be found on the local filesystem. You will see how this configuration file gets deployed via UserData in a moment.

This concludes our SaltStack internals session. As you get more comfortable, you may want to look into Salt Engines, Beacons, writing your own modules and/or Salt Formulas. And those are only some of the ninja features being constantly added to the project.

At this stage we already know how to use Terraform to deploy and now SaltStack to configure.