The Direct Puppet initiative aims to make some improvements in how clients and servers communicate. Most of the efforts will be focused on having more control on how and when catalogs are recompiled, trying to do it just when needed.
This initiative includes a change in the protocol that will make it more efficient. Instead of asking the server for the catalog, the client will have the initiative of sending (before anything) an identifier of the last catalog executed. The communication will follow these steps:
- Client agent sends last executed
catalog_idto the server. As part of the direct Puppet initiative, catalogs are intended to change only if a new version of the catalog is released, that means that while the same code is deployed, the same catalog will never be computed for the same node (even if facts change), so Puppet will be able to uniquely identify it. - Puppet server checks if this
catalog_idcorresponds to the currently deployed code, if it is, it doesn't need to compile it again; if it isn't, then it has to be compiled as usual, but a newcatalog_idis also created. - Puppet server answers to the client agent; at this step the agent knows what catalog has to be applied. Note that if it was the cached catalog, the communication until now has been way faster.
- Local files are checked to see whether they match the expected state. But, instead of asking the master to compute all the files and templates and compare them with the local content, a hash is stored for any of the files in the static cached catalog, so the agent just has to compute the hash of the local files and request for the files that do not match with the hash in the catalog.
- The server retrieves the requested files from its cache.
The main idea behind these changes is to detect as early and efficiently as possible whether some work needs to be done by the master. It will also increase control on when the catalogs and files are computed and built. The only moment that new catalogs will be required will be when new code is deployed; it is a way to give administrators the control to know when changes will be applied. Changes in facts won't trigger new compilations, as it would go against the decision of the administrator to apply changes; this also helps to assume that the same code will always generate not only the same catalog but also the same files.
With static catalogs, new possibilities for code deployment and file caching appear.
When having multiple Puppet masters serving the same code, we may face the problem of not being sure whether the code is synchronized between them, and in most of the cases, we can even be sure that the code is going to be unsynched at least while we release it, as not all files change at once in all servers. The result of code compilation while the code is being updated is unpredictable.
File sync provides mechanisms to improve how code is deployed:
- It does atomic updates, it allows to update all servers at once while they serve requests
- It ensures that the code has been deployed completely and correctly before starting to serve new requests based on it
- It allows to know exactly what code generated a catalog
- Cached catalogs can be safely invalidated
It basically works by requesting a master of master for confirmation to know if code has changed and is transparent for agents.
When code is deployed, these steps are followed:
- Code is deployed to a staging area
- An HTTP POST request is sent to the new endpoint
/file-sync/v1/publishin Puppet master to start publishing the code - When masters have to compile, they ask the master of masters for the new code; every request processed after the moment the notification is sent will use this code
A new code manager will also be released to support this new way to release Puppet code, but in any case, it will be very easy for any deployment tool to implement file sync, as it will only need to push the code to a known path and send the request to the Master to start publishing the new code.
In principle, these features will only be available in Puppet Enterprise.