I'm not going to lie - sending out nearly a million emails announcing that you've had a security issue is no fun. Honestly, it's one of the worst things I have ever had to do in my professional life. It's true that cyber-crime is growing fast, and we are not the only organization to face this dilemma, but this isn't the kind of club membership you want to keep.
While the situation is one I hope to never have to relive, I have been grateful for a couple of things as we move through the process of discovery, to communication, to remediation.
I learned from the best. Leading a team of volunteers and staff through a situation you've never had to face before is daunting. Luckily, the DA has a remarkable board and fantastic community members who were wonderful coaches.
The secret sauce is community. I really consider the response to this incident a love-letter to the Drupal community. Volunteers worked alongside DA staff to craft a response plan (technical and communications) and execute on that plan quickly and ably. Our shared commitment to the Project fueled a lot of sleepless nights, helped us work together with grace and humor, and allowed for some really impressive work.
As I told Dries last week, I am forever grateful that I got to go through this process and witness the kind of teamwork they write mangement books about. While countless folks took part in the response effort, I wanted to specifically mention a few key players:
- Lance Albertson (ramereth)
- Heine Deelstra (heine)
- Neil Drumm (drumm)
- Rudy Grigar (basic)
- Michael Hess (mlhess)
- Ben Jeavons (coltrane)
- Gerhard Killesreiter (killes)
- Greg Knaddison (greggles)
- Kieran Lal (amazon)
- Narayan Newton (nnewton)
- Joe Saylor (jsaylor)
- Tatiana (tvn)
It's also important to note that the Open Source Lab at OSU made significant contributions as well. They have been tremendous partners in this (and every) process.
Of course, we also owe YOU a huge round of thanks. As news outlets picked up the story and theories began to fly on Twitter, the Drupal community stepped up to correct misinformation, to deflect undue criticism, and generally support the Project and volunteers as we worked through all of this. All in all, this went about as smoothly as it possibly could, and that's due to the community.
Although this is a post about thank you's, it doesn't mean our work is done. Our next steps at the DA are to support the community in continuing both the remediation for the incident as well as the investigative work we need to do. We'll share what we can, when we can, and appreciate your patience as we continue this work.
| Attachment | Size |
|---|---|
 pdx_group.jpg | 38.82 KB |
Comments
Unique opportunity to learn from
Want to start with applauding the DA team as well as community members and the Open Source Lab. In my view you all did a stellar job sorting this out. Also very happy with the transparency throughout it.
However, I believe this also gives us a unique opportunity to learn a lot from. A few days a go I published Much to Learn From the drupal.org Password Reset where I wrote about my own UX (User eXperience) of the events. I then took the same scenario and placed it in the context of another site, a site where users have much different, but much more common, reasons to be a user of a site.
I think this gives us a unique opportunity to learn how the password reset itself was conducted and also how we can improve Drupal itself for when this is happening the next time.
Your blog post was great
Hi Thomas -
Your blog post was fantastic, by the way. I 100% agree that there is a lot we can learn, and some of your suggestions are right on the mark. Right now, we're still focused on learning from what happened and understanding all the deatils. We currently have a security firm conducting an audit and once we have those findings, the staff and volunteers can conduct a post-mortem and share some of those lessons learned.
Our only hope is that we never have to implement any of these lessons learned! :)
Holly
Glad you found my post of
Glad you found my post of value Holly. Allways appreciated to hear.
Its good to know that Drupal not only have a great trackrecord of taking security issues seriously, but also act quickly to both step it up and do the right thing when things like this happens. Particularly as people with questionable intentions seems to get more interested in targeting the big open source CMS's. While the WordPress And Other CMS Platforms Give Attackers Room For Creativity focuses on WordPress, Drupal i mentioned in it as a target for these things. Its an interesting read and I am, unfortunately, sure we will see articles like that more frequently from now on.
PS: Would be great if the Comment Notify module could be added to your site to improve the UX for discussions.