r10k is an automation tool for Puppet environments. It is hosted on GitHub at https://github.com/puppetlabs/r10k. The project is used to speed up deployments when there are many environments and many Git repositories in use. From what we've covered so far, we can think of it as librarian-puppet and Git hooks in a single package. r10k takes the Git repositories specified in /etc/puppetlabs/r10k/r10k.yaml and checks out each branch of the repositories into a subdirectory of the environment directory (the environment directory is also specified in /etc/puppetlabs/r10k/r10k.yaml). If there is a Puppetfile in the root of the branch, then r10k parses the file in the same way that librarian-puppet does and it installs the specified modules in a directory named modules under the environment directory.
To use r10k, we'll replace our post-receive Git hook from the previous chapter with a call to r10k and we'll move our librarian-puppet configuration to a place where r10k is expecting it. We'll be running r10k as the puppet user, so we'll set up the puppet user with a normal shell and login files, as shown here:
[root@stand ~]# chsh -s /bin/bash puppet Changing shell for puppet. Shell changed. [root@stand ~]# cp /etc/skel/.bash* ~puppet/ [root@stand ~]# chown puppet ~puppet/.bash* [root@stand ~]# sudo -iu puppet [puppet@stand ~]$
Now, install the r10k gem as shown here:
[root@stand ~]# puppet resource package r10k ensure=present provider=gem Notice: /Package[r10k]/ensure: created package { 'r10k': ensure => ['2.0.3'], }
Next, we'll create a /etc/puppetlabs/r10k/r10k.yaml file to point to our local Git repository. We will also specify that our Puppet environments will reside in /etc/puppetlabs/code/environments, as shown in the following snippet:
--- cachedir: '/var/cache/r10k' sources: control: remote: '/var/lib/git/control.git' basedir: '/etc/puppetlabs/code/environments'
Now, we need to create the cache directory and make it owned by the puppet user. We will use the following commands to do so:
[root@stand ~]# mkdir /var/cache/r10k [root@stand ~]# chown puppet:puppet /var/cache/r10k
Now, we need to check out our code and add a Puppetfile to the root of the checkout. In each environment, create a Puppetfile that contains which modules you want installed in that environment; we'll copy the previous Puppetfile as shown in the following code:
forge "http://forge.puppetlabs.com" mod 'puppetlabs/puppetdb', '5.0.0' mod 'puppetlabs/stdlib', '4.9.0'
We'll check the syntax of our Puppetfile using r10k as shown here:
[samdev@stand control]$ cat Puppetfile forge "http://forge.puppetlabs.com" mod 'puppetlabs/puppetdb', '5.0.0' mod 'puppetlabs/stdlib', '4.9.0' [samdev@stand control]$ r10k puppetfile check Syntax OK
Now, add the Puppetfile to the Git repository using the following commands:
[samdev@stand control]$ git add Puppetfile [samdev@stand control]$ git commit -m "adding Puppetfile" [production 17d53ad] adding Puppetfile 1 file changed, 3 insertions(+) create mode 100644 Puppetfile
Now, r10k expects that the modules specified in the Puppetfile will get installed in $environment/modules, but we already have modules in that location. Move the existing modules into another directory using the following commands; dist or local are commonly used:
[samdev@stand control]$ git mv modules dist [samdev@stand control]$ git commit -m "moving modules to dist" [production d3909a3] moving modules to dist 6 files changed, 0 insertions(+), 0 deletions(-) rename {modules => dist}/base/manifests/init.pp (100%) rename {modules => dist}/hostname_problem/manifests/init.pp (100%) rename {modules => dist}/resolver/manifests/init.pp (100%) rename {modules => dist}/syslog/manifests/init.pp (100%) rename {modules => dist}/virtual/manifests/init.pp (100%) rename {modules => dist}/web/manifests/init.pp (100%)
Now that our modules are out of the way, we don't want a modules directory to be tracked by Git, so add modules to .gitignore using the following commands:
[samdev@stand control]$ echo "modules/" >>.gitignore [samdev@stand control]$ git add .gitignore [samdev@stand control]$ git commit -m "adding .gitignore" [production e6a5a4a] adding .gitignore 1 file changed, 1 insertion(+) create mode 100644 .gitignore
Ok, we are finally ready to test. Well almost. We want to test r10k, so we need to disable our post-receive hook; just disable the execute bit on the script using the following commands:
[root@stand ~]# sudo -u git bash [git@stand ~]$ cd /var/lib/git/control.git/hooks [git@stand hooks]$ chmod -x post-receive
Now we can finally add our changes to the Git repository, using the following commands:
[git@stand hooks]$ exit exit [root@stand ~]# sudo -iu samdev [samdev@stand ~]$ cd control [samdev@stand control]$ git push origin production Counting objects: 9, done. Compressing objects: 100% (7/7), done. Writing objects: 100% (8/8), 946 bytes | 0 bytes/s, done. Total 8 (delta 2), reused 0 (delta 0) remote: production remote: production change for samdev To /var/lib/git/control.git/ 0d5cf62..e6a5a4a production -> production
Note that the only remote lines in the output are related to our pre-receive hook since we no longer have a post-receive hook running.
We will be running r10k as the puppet user, so we'll need to ensure that the puppet user can access files in the /var/lib/git directory; we'll use Filesystem Access Control Lists (FACLs) to achieve this access as shown here:
[root@stand ~]# setfacl -m 'g:puppet:rwx' -R /var/lib/git [root@stand ~]# setfacl -m 'g:puppet:rwx' -R -d /var/lib/git
Before we can use r10k, we need to clean out the environments directory using the following commands:
[samdev@stand control]$ exit logout [root@stand ~]# sudo chown puppet /etc/puppetlabs/code [root@stand ~]# sudo -iu puppet [puppet@stand ~]$ cd /etc/puppetlabs/code [puppet@stand code]$ mv environments environments.b4-r10k [puppet@stand code]$ mkdir environments
Now we can test r10k using r10k deploy as follows:
[puppet@stand code]$ r10k deploy environment -p [puppet@stand code]$ ls environments master production puppet_sync quiet thomas
As we can see, r10k did a Git checkout of our code in the master, thomas, quiet, and production branches. We added a Puppetfile to the production branch; so, when we look in /etc/puppetlabs/code/environments/production/modules, we will see the puppetdb and stdlib modules defined in the Puppetfile:
[puppet@stand code]$ ls environments/production/modules puppetdb stdlib
We have now used r10k to deploy not only our code but the puppetdb and stdlib modules as well. We'll now switch our workflow to use r10k and change our post-receive hook to use r10k. Our post-receive hook will be greatly simplified; we'll just call r10k with the name of the branch and exit. Alternatively, we can have r10k run on every environment if we choose to; this way, it will only update a specific branch each time. To make the hook work again, we'll first need to enable the execute bit on the file, using the following commands:
[root@stand ~]# sudo -u git bash [git@stand root]$ cd /var/lib/git/control.git/hooks/ [git@stand hooks]$ chmod +x post-receive
Next, we'll replace the contents of post-receive with the following script:
logout
#!/bin/bash
r10k=/usr/local/bin/r10k
read oldrev newrev refname
branch=${refname#*/*/}
if [ -z "$branch" ]; then
echo "ERROR: Branch undefined"
exit 10
fi
exec sudo -u puppet $r10k deploy environment $branch –pNow, we need to edit our sudoers file to allow Git to run r10k as puppet, as shown here:
Defaults !requiretty git ALL = (puppet) NOPASSWD: /bin/git *, /usr/bin/puppet-sync *,/usr/local/bin/r10k * %pupdevs ALL = (puppet) NOPASSWD: /bin/git *, /usr/bin/puppet-sync *, /usr/local/bin/r10k *
Now, to test whether everything is working, remove a module from the production environment using the following command:
[root@stand ~]#
m -rf /etc/puppetlabs/code/environments/production/modules/stdlib
Now, make a change in production and push that change to the origin to trigger an r10k run, as shown here:
[root@stand ~]# sudo -iu samdev [samdev@stand ~]$ cd control/ [samdev@stand control]$ echo "Using r10k in post-receive" >>README [samdev@stand control]$ git add README [samdev@stand control]$ git commit -m "triggering r10k rebuild" [production bab33bd] triggering r10k rebuild 1 file changed, 1 insertion(+) [samdev@stand control]$ git push origin production Counting objects: 5, done. Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 295 bytes | 0 bytes/s, done. Total 3 (delta 2), reused 0 (delta 0) remote: production remote: production change for samdev To /var/lib/git/control.git/ 422de2d..bab33bd production -> production
Finally, verify whether the stdlib module was recreated or not using the following command:
[samdev@stand control]$ ls /etc/puppetlabs/code/environments/production/modules/ puppetdb stdlib
Keeping everything in r10k allows us to have mini labs for developers to work on a copy of our entire infrastructure with a few commands. They will only need a copy of our Git repository and our r10k.yaml file to recreate the configuration on a private Puppet master.