Our History With Security and What We Do Now

This is the story of Privacy International's journey to building more secure services. Data collection and administering sensitive data on the open web is risky, and PI had to learn this the hard way.

Many companies say that the privacy of their audiences is their top priority. But do they mean it? Do they invest in it? Doing security on tight budgets is incredibly hard. But it is the natural state of the non-profit sector. We learned this through challenging experiences. But it is worth all the lessons along the way. 

For years, PI has worked to get our website right for public engagement. Our first website, launched 25 years ago, was managed directly by our deputy director who directly edited static HTML files. Eventually we got onto a Content Management System ("CMS") and a shared service with other NGOs. At the time, we had no office or resources, but we had a logo, some HTML, and a server.

From our very first website, we have worked to minimise the data that we collected about our visitors, while also trying to gain insight into what types of content and advocacy were most engaging.

In 2010, aiming to expand our methods of engagement and freshen up our website, we moved to the CMS Drupal. 

Like everyone, we wanted a website that was visually attractive. We wanted to present more long-standing artefacts rather than just news: global studies, comparisons, explainers that were more likely to be of interest to our global audiences rather than commentaries and press releases on latest developments. 

To achieve our shiny new website, we began to outsource development but keep administration in-house. We hosted our website with our friendly neighbourhood social justice internet service provider, GreenNet. (We still do! They are great!)

We remained adamant that the personal data we collected about our readers and followers would be minimal and kept secure. We have, to my knowledge, never failed at that. For example, in addition to our website, we tried many platforms for mailing lists that practically by design required us to spy on our users in ways incompatible with our goals. 

So we decided to, again, administer our own service: using CiviCRM. We believed that hosting this public-engagement service, CiviCRM, in the outside world was a risk. The financial data of our supporters combined with their personal contact details are a treasure trove for those who want to do harm to or supporters and to us. 

<Enter disaster> 

The perfect storm in our approach to website development occurred in the fall of 2014. Despite the protests of our tech experts, we launched a new snazzy website that was so overly customised, we could not effectively run it. And then Drupal announced its services had a critical vulnerability. 

At first, I tried to ignore it and say that we should hold off updating our website, particularly as we were heading to a key fundraising period: the holidays. But eventually, on advice from our Tech Team, we decided to bite the bullet and update our website to address the vulnerability. We ran the update and everything came crashing down: the website stopped working and so did the integration with CiviCRM. Dead in the water. (Yes, on the evening of our first fundraising event.)

Now we did something called 'failing well, disastrously'. We could have reverted to the pre-updated website, risking not only our website to the Drupal vulnerability but anything running on that server including CiviCRM. We could have just prayed that nothing would happen - though we'd be none the wiser if anything had happened. Instead we decided (and I recall the conversation taking place at Bogota airport quite late at night, after far too much to-ing and fro-ing) to take down our site and rebuild from scratch. 

So for the next two years we rebuilt our website, always using a vanilla Drupal install and only using a simple style sheet to get the fancy aspects working. Importantly, we started afresh running only services that we administer in-house and could update as soon as practicably possible. 

What we built

We also decided that we would store as much data inside our office and beyond reach from the outside world - periodically bringing the data from our website down and deleting all the outside data (including the Drupal installation). 

In this period, our organisation struggled to find the right path on how to manage our website, mailing list, and other services. We could have outsourced their development, hosting, and upkeep - at great cost financially and at the loss of control over how our audience's and supporters' data was treated. Or we could have ignored security and just hoped for the best. 

Instead our Tech Team developed a means of managing our services and network through launching and managing virtualisation in an open and maintainable way. This system, that we call Thornsec, is now being used inside PI and on our external servers. 

It's our honest attempt to practice what we preach and run secure services which treat our readers' and supporters' data with great care. It also protects the data of the organisation and the people we work with across the world (but not on servers directly facing the outside world). 

And so now with this new foundation, we are finally able to change the way we run our external services. Only now do I feel confident enough to ask our audience and supporters to volunteer their data (beyond IP logs of course). In the coming weeks, we are launching new public-facing services to disseminate our work, campaign together, and fundraise. 

Our new sites

What are we doing differently now? Essentially, we will be compartmentalising running services, through virtual machines in a manageable way.  

On one, we will have our vanilla install of Drupal and a style sheet that will be our website. We will start generating analytics based on user visits but in a data-minimising manner. And every time we add new content to our site, we will wipe the existing Drupal install on the outside server so if there's an infiltration, it is likely erased. 

On the second virtual machine, we will start a new campaigning service, action.privacyinternational.org, running CiviCRM, to allow people to engage with us. After people on our current mailing list have had the chance to opt-in to our new service, we will be deleting all previously held mailing list subscription data. We will ask people to volunteer information about themselves to sign up to new ways of receiving news about our work. We will also create new ways of working with our supporters, through petitions and communications campaigns, amongst other methods. All of this data about people will be persistent data on our external server: aspects of the issues that people are interested in and parts of the world they want to hear more about. It is sensitive data and by keeping our services up-to-date and monitoring regularly for problems, we feel we are now in a position to take these calculated and measured growth steps, if people agree. 

On a third virtual machine, we will run another install of CiviCRM that will only process the financial data of our donors. We don't want this data to be persistent on our external servers, so we will provide direct interfaces to payment processes and bring the data into PI as soon as possible. We will delete any remaining data from our outside server regularly.  

There was a time when we wished our website was prettier and had more functionality. We've taken the long road around to understanding security: appearance and ease of use is certainly important, but we must not sacrifice the trust that people place in us. I hope everyone will eventually be impressed with the new content and services on our not-quite-so limited site. 

It's still not perfect. This is what the business of NGOs is today. And if you're going to fail, do so well.  Hopefully with Thornsec, any organisation can now bounce back faster.