How we are addressing a mistake we made while running defectivebydesign.org
On Wednesday, October 25th, we received an email letting us know that an old Drupal database backup file was publicly accessible on defectivebydesign.org, a site operated by the Free Software Foundation. This backup file contained contact information and other details that should not have been public, submitted from 2007-2012.
Within minutes of receiving the report, we removed the file and started auditing defectivebydesign.org and the rest of our sites. The file did not contain any passwords or password hashes, financial information, mailing addresses, or information about users who interacted with the site without ever logging in.
On Friday, October 27th, once we were reasonably confident we understood the scope of the problem and had fixed the most urgent issues, we sent a notification email to every address that was in the database backup file. We explained what had happened, took responsibility, and apologized.
If you did not receive such an email, then your address was not in the exposed file.
The file included (from both real and spambot users' profiles):
- ~28,000 email addresses;
- contact names;
- some IP addresses associated with comments on posts;
- ~120 phone numbers;
- some information users shared about whether they participated in a particular campaign action (like a call-in), and
- timestamps of users submitting data.
While some of this information was intended by users to be public, some of it definitely was not.
I and the rest of the FSF staff are deeply sorry for this mistake. We know how important privacy is to our supporters; we fight on your behalf every day against restrictive and invasive technologies that threaten it. We also don't believe in covering up our mistakes, so we wanted to let everyone affected know as soon as possible, and then share our mistake and what we learned from it here, publicly.
Even though we are a small team, under pressure to move fast against extremely large forces, this kind of mistake is absolutely unacceptable. We have made many improvements in our security practices since 2012, and in light of this failure will be taking a deeper look at what else we need to do.
I'd also like to share some of the technical details about what happened, because in just a few minutes of searching, we found others who are making the same mistake we did.
A backup of defectivebydesign.org's Drupal database was made with the backup-migrate module in 2012, likely to assist migration of the site to a new host. We failed to delete or move that file.
In 2014, or some time before then, the directory name of our Drupal installation was manually changed as part of an upgrade. However we didn't update the part of our Apache configuration that enabled .htaccess files for specific directories. Drupal's .htaccess file normally hides files by disallowing directory indexes. The site appeared to work normally despite the disabled .htaccess file because our main Apache configuration contained functionality normally performed by that file. We also mistakenly didn't have another .htaccess file to fully disable access to the backup. As a result, the backup file was left exposed.
The documentation for backup_migrate has a "VERY IMPORTANT SECURITY NOTE" indicating that "Backup and Migrate attempts to protect backup files using a .htaccess file," which we failed to mind.
We currently don't use this module, and instead backup the site as part of our global backup procedures. We are reviewing and improving several other policies and procedures to both avoid making similar mistakes again, and to detect them should they be made. This includes, for example, deleting personal data from sites where we no longer use it or need it, and accelerating our progress toward full coverage by our centralized server configuration management system.
Thank you all for your support and trust. Our technical team can also use more hands on some of their work to help expedite improvements; if you have expertise in systems administration and are interested in volunteering some time to help, please let us know at email@example.com.