Skip to content, sitemap or skip to search.

Personal tools
Join now
You are here: Home Bulletins 2011 Fall 2011 Bulletin Automating systems at the FSF

Automating systems at the FSF

by Ward Vandewege Contributions Published on Nov 28, 2011 04:54 PM
The sysadmins at the FSF have put great effort into consolidating our server infrastructure over the past 12 months. We have retired more than a dozen servers in the process, and now host most of our infrastructure on three potent machines with many CPU cores, ample RAM, and plenty of disk space. As you may know, we use Xen to virtualize our servers. Virtualization allows us to securely partition our servers into many virtual machines, each dedicated to a limited number of tasks.

We have also embarked on a journey to automate our systems configuration as much as possible. We have selected Puppet, a systems configuration management and automation tool, to help in that task.

One of the advantages of Puppet is that one can start small. Once a system is under Puppet's control, it is easy to expand the Puppet configuration over time, and thus automate more and more of it. We started out with a very limited configuration that defined some settings that are common to all our systems -- for instance, making sure the sshd configuration is secure. Puppet makes it easy to differentiate rules based on "facts" about a machine, like the version of the operating system it runs, or whether the system is a physical server or a virtual machine. Each system under Puppet's control gets its own configuration stanza, so it is also possible to do things that are specific to one machine.

Here are some of the things that we now do with Puppet, rather than by hand:

  • install appropriate software packages, and standard GNU/FSF configurations for them
  • create and manage user accounts
  • distribute SSH public keys and SSL certificates
  • update xen-tools configurations on virtualization host systems

We create new virtual machines with xen-tools, which pulls in the Puppet packages. We then add the new virtual machine to our Puppet configuration, which pulls in the default GNU/FSF configuration without any additional work on our part. We save a lot of time setting up new machines, and we get peace of mind: all our systems under Puppet's control are guaranteed to have our standard configuration.

We are currently working towards the goal of generating our automated systems monitoring configuration from our Puppet configuration. This requires us to migrate more service configurations to Puppet. With enough of that done, it should be possible for Puppet to know that, for instance, runs a web server on port 80. With that knowledge, Puppet can instruct our monitoring hosts to check for the availability of that service, all without manual intervention from the sysadmins.

I would like to conclude this article with a brief word of thanks to Bernie Innocenti, who left the FSF for another job in September. We are extremely grateful for his contributions as an FSF sysadmin; he was instrumental in the server consolidation and Puppet setup efforts described here.

Document Actions

The FSF is a charity with a worldwide mission to advance software freedom — learn about our history and work. is powered by:


Send your feedback on our translations and new translations of pages to