Methodology: Social Media Monitoring by Local Authorities

The Freedom of Information Act Request

1. In 2016 the Rt Hon Lord Judge, then Chief Surveillance Commissioner, wrote to all Local Authorities regarding use of social media in investigations. Please confirm whether you are aware you received this letter and:

a. Provide a copy of your response; (please confirm if you did not respond)

b. Provide a copy of any internal audit relating to social media use arising out of Rt Hon Lord Judge’s recommendations; (please confirm if you did not conduct an internal audit and state whether any internal audit of social media use has taken place since 2016).

c. Provide a copy of your corporate policy on the use of social media in investigations. (please confirm if you do not have one)

d. Please confirm whether a follow up audit was conducted by the Surveillance Commissioner’s Office which was exclusively or partially related to social media use in investigations by your Local Authority.

2. Does your Local Authority conduct overt and/or covert social media intelligence in some or all of its work?

a. If yes, please specify whether this includes profiling individuals, conducting investigations, monitoring individuals, monitoring groups, monitoring locations, gathering intelligence, for recruitment purposes.

b. If your Local Authority does conduct social media intelligence/monitoring, please specify whether this includes both or either overt or covert monitoring of social media.

c. If the Local Authority has conducted covert social media monitoring, please confirm the number of RIPA warrants obtained in the last two years for this purpose.

3. If the Local Authority conducts social media intelligence, please provide a copy of any current guidance/policies/internal guidance/code of practice or any other such written material used by/available to the local authority or those working on behalf of the local authority to conduct SOCMINT, the monitoring or accessing of information published on social media that is either publicly available or requires additional access e.g. to be friends with an individual, to have password and login details.

4. If you conduct overt or covert social media intelligence relating to social media platforms, please provide a copy of:

a. Relevant [sections of the] privacy policy;
b. the data protection impact assessment;
c. privacy impact assessment;
d. equality and human rights impact assessment
e. training materials for those conducting social media intelligence.

Please state if you do not have any of the above.

5. Please provide a copy of any other template/form/document currently used (or to be used with the next three months) by the local authority or fraud investigator (or team) in the conduct of social media monitoring.

6. Please confirm whether or not your local authority has purchased or uses software and/or hardware to conduct social network / social media monitoring and/or in relation to sentiment analysis.

a. If yes, please state the name of the company / provider.

b. If no, please state whether the local authority has developed internal methods to conduct social media / social network monitoring.

7. Please confirm, if not stated in the guidance (question 3), the policy on deletion of data obtained from social networking sites.

8. If no documents (question 3) exist, or if the following is not covered in the documents which do exist, please explain:

a. In what areas of the local authority’s work is social media monitoring used
b. What criteria must be satisfied in order for social media monitoring to be carried out
c. Who must authorise the request to conduct social media monitoring
d. What is the process for conducting social media monitoring
e. How long is data collected and retained?
f. Is there any process for requesting deletion?

9. Are you able to state how regularly social media monitoring is used? If so, please provide the figures.

METHODOLOGY

Participation: how many Local Authority responses did we look at.

At the end of November 2019, we reviewed all responses received to date from local authorities. This amounted to 136 responses.

The participation results can be viewed in this spreadsheet.

How did we break down the responses to the Freedom of Information request

Using those 136 responses - We read each response and any accompanying disclosures such as guidance documents. Each response was input into an Excel spreadsheet. We then normalised the data and used the responses to obtain results for the following:

1. Local Authorities who use overt social media monitoring.

2. Local Authorities use covert social media monitoring.

3. Local Authorities who do NOT use overt OR covert social media monitoring AND do NOT have a policy (on overt social media monitoring).

4. Local Authorities who do NOT use overt or covert social media monitoring AND DO have a policy (on overt social media monitoring).

5. Question 3 & 4 Relatively: Among the local authorities who do NOT use overt OR covert social media monitoring: how many DO or DO NOT have a policy (on overt social media monitoring).

6. Local Authorities DO use overt social media monitoring and DO have a policy (on overt social media monitoring).

7. Local Authorities who DO use overt social media monitoring and do NOT have a policy (on overt social media monitoring).

8. Questions 6 & 7 relatively: Among the local authorities who DO use overt social media monitoring: How many DO or do NOT have a policy (on overt social media monitoring).

9. Local Authorities who DO use overt social media monitoring and DO have a policy and do NOT review overt use

10. Local Authorities who DO use overt social media monitoring and do NOT have a policy and do NOT review overt use

11. Question 10 & 11 relatively: Among the local authorities who DO use overt social media monitoring and do NOT review overt use: How many DO or do NOT have a policy?

1. Local Authorities who use overt social media monitoring

[spreadsheet]

The distinction between ‘overt’ and ‘covert’ social media monitoring is key to determining what activities can take place without any authorisation. 

The Regulators’ Guidance states that:

“Where privacy settings are available but not applied the data may be considered open source” or publicly available; the authority has a reasonable expectation of privacy if access controls are applied…Where privacy settings are available but not applied the data may be considered open source and an authorisation is not usually required. Repeat viewing of “open source” sites may constitute directed suveillance on a case by case basis…"

'Yes' = do use overt social media monitoring

'No' = do not use overt social media monitoring

'Not clear' = not clear from their response if they do or do not

'Refused' = refused to respond to the question

'NCND' = response was 'neither confirm nor deny

This informed key finding: A significant number of local authorities are now using 'overt' social media monitoring and this substantially out-paces the use of 'covert' social media monitoring. 

2. Local Authorities who use covert social media monitoring

[spreadsheet]

The Regulators’ Guidance states that:

“Where privacy settings are available but not applied the data may be considered open source” or publicly available; the authority has a reasonable expectation of privacy if access controls are applied…Where privacy settings are available but not applied the data may be considered open source and an authorisation is not usually required. Repeat viewing of “open source” sites may constitute directed suveillance on a case by case basis…"

'Yes' = do use overt social media monitoring

'No' = do not use overt social media monitoring

'Not clear' = not clear from their response if they do or do not

'Refused' = refused to respond to the question

This informed key finding: A significant number of local authorities are now using 'overt' social media monitoring and this substantially out-paces the use of 'covert' social media monitoring. 

Of those who are using covert social media monitoring, in response to the question “(c) If the Local Authority has conducted covert social media monitoring, please confirm the number of RIPA warrants obtained in the last two years for this purpose” we received the following updates:

• Blaenau Gwent County Borough Council: 2
• Bromley Borough Council: 1
• Devon County Council: 2
• Havant Borough Council: 1
• Isle of Anglesey Council: 3
• Leicestershire County Council: RIPA warrants: in the period between 1 October 2017 and 1 May 2019, 1 directed surveillance and 5 covert SOCMINT were approved.  OVERT SOCMINT: 13 authorisations granted between Sept 2016 - Sept 2019.North Yorkshire County Council   The following approvals were sought and granted for directed surveillance and use of CHIS in connection with the sale of goods via Facebook. Date Number of approvals  1 October – 30 September 2018/19 1 (counterfeit goods) 1 October – 30 September 2017/18 2 (fireworks)
• Oxfordshire County Council: 6
• Southampton City Council: 1
• Suffolk County Council: 6
• Surrey County Council:1
• West Dunbartonshire Council: 3
• West Sussex County Council: 8

3. Local Authorities who do NOT use overt OR covert social media monitoring AND do NOT have a policy (on overt social media monitoring).

[spreadsheet]

We wanted to find out whether local authorities, even those not using overt or covert social media monitoring, nevertheless had a policy on overt social media monitoring. 30 local authorities do not use overt or covert social media monitoring and do not have a policy on overt social media monitoring. 

4. Local Authorities who do NOT use overt or covert social media monitoring AND DO have a policy (on overt social media monitoring).

[spreadsheet]

We wanted to find out whether local authorities, even those not using overt or covert social media monitoring, nevertheless had a policy on overt social media monitoring. 9 local authorities who do not use overt or covert social media monitoring, nevertheless do have a policy on overt social media monitoring. 

5. Question 3 & 4 Relatively: Among the local authorities who do not use overt or covert social media monitoring: how many do or do not have a policy?

Out of the 39 local authorities who do not use overt or covert social media monitoring, 9 do have a policy and 30 do not have a policy.

It is interesting that despite not using overt social media monitoring, local authorities have nevertheless introduced a policy on using this form of surveillance. 

6. Local Authorities who do use overt social media monitoring and do have a policy

[spreadsheet]

Turning to local authorities who do use overt social media monitoring (85), we looked at how many of those have a policy in relation to this form of surveillance. Whilst some local authorities have policies that relates to covert use, these do not always address 'overt' use. Some local authorities have a standalone policy on overt social media usage, others rely on sections in other policies and guidance on surveillance. 

Out of the local authorities who told us that they use overt social media monitoring, 63 responded that they have a policy to guide use of this form of surveillance. 

7. Local Authorities who do use overt social media monitoring and do not have a policy

[spreadsheet]

We looked at how many of those who use overt social media monitoring (85), do not have a policy in relation to this form of surveillance. 15 local authorities responded that they do use overt social media monitoring but do not have a policy. 

8. Questions 6 & 7 relatively: Among the local authorities who do use overt social media monitoring: How many do or do not have a policy?

9. Local Authorities who do use overt social media monitoring and do have a policy and do not review overt use

[spreadsheet]

We asked local authorities whether if they were able to state how regularly social media monitoring is used and if so to provide figures. We then looked at those who do use overt social media monitoring, do have a policy and yet are not able to tell us how regularly they use social media monitoring, indicating that there is no review of overt use of social media monitoring. We further examined their policies and other responses to assess whether they review overt use of social media monitoring. 

10. Local Authorities who do use overt social media monitoring and do not have a policy and do not review overt use

[spreadsheet]

We asked local authorities whether if they were able to state how regularly social media monitoring is used and if so to provide figures. We then looked at those who do use overt social media monitoring, but do not have a policy and also are not able to tell us how regularly they use social media monitoring, indicating that there is no review of overt use of social media monitoring. We further examined their policies and other responses to assess whether they review overt use of social media monitoring. 

11. Question 10 & 11 relatively: Among the local authorities who do use overt social media monitoring and do NOT review overt use: How many do or do not have a policy?

[spreadsheet]

ANNEX A: FULL FOIA

(also available here)

Dear Sir/Madam,

Freedom of information act request

RE: Social media monitoring / social media intelligence

FOIA REQUEST

For definition of social media intelligence please see background explanation below. We further note the comments of the Office of Surveillance Commissioners Annual Report 2016 cited below.

1. In 2016 the Rt Hon Lord Judge, then Chief Surveillance Commissioner, wrote to all Local Authorities regarding use of social media in investigations. Please confirm whether you are aware you received this letter and:
   (a) Provide a copy of your response; (please confirm if you did not respond)

(b) Provide a copy of any internal audit relating to social media use arising out of Rt Hon Lord Judge’s recommendations; (please confirm if you did not conduct an internal audit and state whether any internal audit of social media use has taken place since 2016).

(c) Provide a copy of your corporate policy on the use of social media in investigations. (please confirm if you do not have one)

(d) Please confirm whether a follow up audit was conducted by the Surveillance Commissioner’s Office which was exclusively or partially related to social media use in investigations by your Local Authority.

2. Does your Local Authority conduct overt and/or covert social media intelligence in some or all of its work?

(a) If yes, please specify whether this includes profiling individuals, conducting investigations, monitoring individuals, monitoring groups, monitoring locations, gathering intelligence, for recruitment purposes.

(b) If your Local Authority does conduct social media intelligence/monitoring, please specify whether this includes both or either overt or covert monitoring of social media.

© If the Local Authority has conducted covert social media monitoring, please confirm the number of RIPA warrants obtained in the last two years for this purpose.

3. If the Local Authority conducts social media intelligence, please provide a copy of any current guidance/policies/internal guidance/code of practice or any other such written material used by/available to the local authority or those working on behalf of the local authority to conduct SOCMINT, the monitoring or accessing of information published on social media that is either publicly available or requires additional access e.g. to be friends with an individual, to have password and login details.

4. If you conduct overt or covert social media intelligence relating to social media platforms, please provide a copy of:
   (a) Relevant [sections of the] privacy policy;
   (b) the data protection impact assessment;
   © privacy impact assessment;
   (d) equality and human rights impact assessment
   (e) training materials for those conducting social media intelligence.

Please state if you do not have any of the above.

5. Please provide a copy of any other template/form/document currently used (or to be used with the next three months) by the local authority or fraud investigator (or team) in the conduct of social media monitoring

6. Please confirm whether or not your local authority has purchased or uses software and/or hardware to conduct social network / social media monitoring and/or in relation to sentiment analysis.

(a) If yes, please state the name of the company / provider.

(b) If no, please state whether the local authority has developed internal methods to conduct social media / social network monitoring.

7. Please confirm, if not stated in the guidance (question 3), the policy on deletion of data obtained from social networking sites.

8. If no documents (question 3) exist, or if the following is not covered in the documents which do exist, please explain:
   a. In what areas of the local authority’s work is social media monitoring used
   b. What criteria must be satisfied in order for social media monitoring to be carried out
   c. Who must authorise the request to conduct social media monitoring
   d. What is the process for conducting social media monitoring
   e. How long is data collected and retained?
   f. Is there any process for requesting deletion?

9. Are you able to state how regularly social media monitoring is used? If so, please provide the figures.

Background explanation: What is social media intelligence

This is a Freedom of Information Act request in relation to social media intelligence. Social media intelligence (SOCMINT) can also be referred to as social media monitoring, social network monitoring, social media listening, open source intelligence (OSINT) (focusing on social media platforms). Reference to SOCMINT in this FOIA request encompasses all these and any other terminology used to describe the viewing, collection, retention and analysis of data on social media platforms.

SOCMINT refers to the techniques and technologies that allow the monitoring and gathering of information on social media platforms such as Facebook, Twitter and Instagram. SOCMINT includes monitoring of content, such as messages or images posted, and other data, which is generated when someone uses a social media networking site i.e. meta data like location information. This information involves person-to-person, person-to-group, group-to-group and includes interactions that are private and public.

The methods for analysing social media networking sites vary. Methods may include manually reviewing content as it is posted in public or private groups or pages; reviewing the results of searches and queries of users; reviewing the activities or types of content users post; or ‘scraping’ - extracting the content of a web page - and replicating content in ways that are directly accessible to the person gathering social media intelligence.

It may include looking at content posted to public or private groups or pages. it may also involve “scraping” or grabbing all the data from a social media platform, including content and individual post and data about their behaviour (likes and shares). Through scraping and other tools, SOCMINT permits the collection and analysis of a large pool of social media data, which can be used to generate profiles and predictions about users.

We are sending this FOIA request to you as a local authority being either a London borough, metropolitan district, unitary authority, district council or county council.

Background explanation: Surveillance Commissioner Annual Reports

The Chief Surveillance Commissioner The Rt Hon Sir Christopher Rose’s Annual Report first referred to social network sites in 2012-13 stating that:

“2.4 All public authorities have struggled with the use of the Internet for investigations, particularly social network sites. A particular difficulty is the desire of national bodies to apply a doctrinaire approach which invites error if facts specific to each case are ignored or poorly considered.”

The subsequent report 2013 – 2014 stated that:

“5.30 … Although there remains a significant debate as to how anything made publicly available in this medium can be considered private, my Commissioners remain of the view that the repeat viewing of individual “open source” sites for the purpose of intelligence gathering and data collation should be considered within the context of the protection that RIPA affords to such activity.

5.31 In cash-strapped public authorities, it might be tempting to conduct online investigations from a desktop, as this saves time and money, and often provides far more detail about someone’s personal lifestyle, employment, associates etc. But just because one can, do not mean one should. The same considerations of privacy and especially collateral intrusion against innocent parties, must be applied regardless of the technological advances. It is worth repeating something I said in my 2011-2012 report, paragraph 5.8:

“There is a fine line between general observation, systematic observation and research and it is unwise to rely on a perception of a person’s reasonable expectation or their ability to control their personal data. Like ANPR and CCTV, the Internet is a useful investigative tool but they each operate in domains which are public and private. As with ANPR and CCTV, it is inappropriate to define surveillance solely by the device used; the act of surveillance is of primary consideration and this is defined at section 48(2-4) of RIPA (monitoring, observing, listening and recording by or with the assistance of a surveillance device). The Internet is a surveillance device as defined by RIPA section 48(1). Surveillance is covert “if, and only if, it is conducted in a manner that is calculated to ensure that persons who are subject to the surveillance are unaware that it is, or may be taking place.” Knowing that something is capable of happening is not the same as an awareness that it is or may be taking place. The ease with which an activity meets the legislative threshold demands improved supervision.”

The final report of the Chief Surveillance Commissioner in 2016 , before this was abolished and replaced by the IPCO stated:

“4.3 From time to time my Inspectorate is asked why, given that no authorisation has been granted by an individual authority since the previous inspection some three years earlier, the process of inspection and oversight is necessary. The short answer is unequivocal. While local authorities remain vested with the power to deploy covert surveillance, regardless of actual use, the appropriate structures and training must remain in place so that if and when the powers do come to be exercised, as they may have to be in an unexpected and possibly emergency situation, the exercise will be lawful. So, for that reason alone the process of inspection must continue. There is a further consideration. The inspection may reveal inadvertent use and misuse of the legislative powers. The steady expansion in the use of the social media and Internet for the purposes of investigative work provides a striking example of a potential new problem which came to light through the inspection system. Local authority officials, vested with burdensome responsibilities for, among others, the care of children and vulnerable adults, are, like everyone else, permitted to look at whatever material an individual may have chosen to put in the public domain. This is entirely lawful, and requires no authorisation. However, repeated visits to individual sites may develop into an activity which, if it is to continue lawfully, would require appropriate authorisations. Local authorities must therefore put in place arrangements for training officials into a high level of awareness of these risks. Without the inspection process this problem might never have been identified.

15.2 As discussed earlier one major consequence of the OSC inspections has been the emergence, during the course of discussions, of investigations being made by public authorities through use of social media and the Internet. For example, it may help to show whether counterfeit good are being offered for sale on a Facebook page, or reveal that someone who claims to be living alone as a single parent has a social media page which provides a different story, or perhaps, particularly sensitive, enable a check to be made whether concerns about the welfare of a child or vulnerable adult may be justified. When individuals choose to go public or advertise themselves, they cannot normally complaint that those who look at their social media sites are disregarding their rights to privacy. However if the study of an individual site becomes persistent, issues under the legislation may arise.

15.3 The resources available to the OSC do not enable me to say positively that issues relating to the use of social media sites have arisen or are arising in every local authority. What matters is that this potential certainly exists. Senior officials at local authorities should therefore be made aware of it and have the necessary policy documents and training and awareness arrangements in place to address it. This issue has been recognised in the forthcoming Home Office and Scottish Government Codes of Practice (which will be issued at a convenient date after the introduction of the Investigatory Powers Act 2016). However, in advance of the issue, and because of repeated findings in reports made to me by Inspectors and Assistant Surveillance Commissioners throughout this year, I acted on my own initiative. I therefore wrote to all local authorities in April 2017 explaining my concerns, and urging them to undertake internal checks of the use of social media by no doubt well-intentioned members of their staff, and to ensure that appropriate guidance and training should be provided.

15.4 An extract from the letter reads:

“it has become steadily more apparent that a number of officers working for public authorities, particularly those with responsibilities for the care of children and vulnerable adults have started to use the [social media and internet] sits, acting in good faith and on their own initiative. RIPA issues do not normally arise at the start of any investigation which involves accessing “open source” material, but what may begin as lawful overt investigation can drift into cover surveillance which falls within the legislation. Although the investigation of crime is not normally a “core function” of the Council, the protection of children and vulnerable adults certainly is, and any continuing and deeper study of the social media site in question would only be justified by the exercise of that protective function.

These are complex legislative provisions, and without appropriate training and awareness council officers cannot be expected to appreciate and apply them. They may therefore act unlawfully. Ignorance would provide no defence to them personally, nor to the Council for which they were working.

The Surveillance Commissioners have issued further guidance on this issue, and identified circumstances when an appropriate authorisations under the legislation would be required or advisable. The guidance is available on the OSC website as a public document, with the OSC Procedures Guidance. Note 289 is relevant and I highlight it for your attention.

It would be sensible for an internal audit of the use of social media sites and Internet for investigative or official business made across all departments be undertaken now. That would provide the necessary information about the extent to which formal training or awareness of these complex provisions is required.”

15.5 A copy of this letter was sent to the Chair of the Local Government Association and the national police lead for child protection issues, Chief Constable Simon Bailey of the Norfolk Constabulary.

15.6 As I reported last year, many local authorities have first-class arrangements in place for the use of covert tactics, even if, as a matter of policy, they do not intend to deploy them: others do not. Where necessary arrangements to ensure compliance are not in place, I require a report from the Chief Executive after a given period, say six months, about how the inadequacies have been addressed, and indicating that a further inspection may have to be arranged. There have been, and will continue to be occasional inspection revisits. One such revisit will have taken place by the time this report is published. I should, however, highlight that the problem of the non-compliant local authority should be kept in perspective. These authorities very rarely use or attempt to use statutory powers, and the occasions when they have been in breach of the legislative provisions remain very rare indeed. The point, as one of my Inspectors helpfully paraphrased it, is that while they are vested with these significant powers they should remain “match fit”. The inspection process provides both an encouragement and a check that they are.

The Investigatory Powers Commissioner replaced the Surveillance Commissioner. The Investigatory Powers Commissioner, Sir Adrian Fulford’s Annual Report 2017 states:

4.37 Local authority guidance on surveillance does not always address how investigators should use social media or where they may need an authorisation. The 2018 revised Home Office code of practice for surveillance contains helpful advice local authorities can incorporate into their policy documents and training.

4.38 Our inspectors were particularly impressed by Durham Country Council, whose senior responsible officer commissioned a helpful audit across the organisation on the “Use of social media in Covert Investigations’, to evaluate and report their system is adequate and appropriate for this purpose. We commend this approach.