Six months of equipment upgrades at the FSF
In the last year, we have given some venerable machines a well-earned retirement. One office server we replaced was a sixteen-year-old Pentium III, our Nagios server called Klaxon. We also retired our cranky virtual machine server Pegasus, and our mystical router Valis – which had been on for over two thousand days! Those services now run on a Respects Your Freedom-certified server running an ASUS KGPE-D16 motherboard, with Libreboot and Trisquel.
D16 motherboards are at the center of another larger infrastructure renovation project that took over a year of research and development from the whole tech team. Special thanks to our 2016 summer intern Samuel Cantero, who got the research rolling, and our 2017 summer intern Andrew Cabey, who helped wrap it up. The result is a High-Performance/High-Availability cluster of Ceph storage servers attached to host machines running Qemu-KVM/libvirt.
Stored on large, fast, solid-state replicated disks, and shared through a reliable 20Gb/s network, this new server stack will replace and extend our existing main infrastructure. It takes full advantage of redundancy for all components: anything from a network cable to an entire machine could fail without services going down, and the duplicated components also boost performance and throughput. The combination of Ceph and libvirt also provides great features like live migration of services between different hosts for maintenance or load-balancing, and the ability to instantly clone a virtual machine into a development copy where new features can be tested, lowering the maintenance cost.
We made great progress on the software front as well. Our Web developer, Andrew, has upgraded and polished the services we rely on the most every day, including RT, our ticket tracker; Brains, our wiki; and our CiviCRM instance, which hosts our members, donors, events, and campaigns. Now that we have the room and the processing power to host new virtual machines, we're ready to begin the second phase of renovation. We'll be upgrading the software on the many public-facing services that we host, and making deep changes to the deployment system to automate background tasks including deployment, backup, and monitoring. This will simplify maintenance and make the service more robust.
The first machines that got moved into the new infrastructure were the Savannah servers hosting the GNU Project collaboration and code-sharing services, and they will be followed with a new stack of mail servers focused on better spam filtering, improved reliability, and a modernization of our Mailman, hosting over three thousand mailing lists for hundreds of free software projects.
All of this investment and effort could not have been done without the generous donations that sustain us. Big renovation projects require extensive research and development, and expensive hardware as well, but are necessary to keep our efforts going and growing. Thank you for your contribution to this work!