Samaritans Radar and use cases

When you develop a piece of software or app, it is common to undergo some evaluative thinking about the use cases and user needs. And this isn’t simple. Use cases are often complicated, and user needs often contradictory. It’s hard to make ‘the right’ decision, especially when you are creating a tool that directly influences and alters an existing social environment and ecosystem – be that on or offline.

Earlier this week, to much fanfare in the press, the Samaritans released a new app called the Samaritans Radar, with dramatic claims about how this app is going to save lives. The discussion on Twitter, the space directly influenced by the new app, has been much more nuanced, and there has been much furore from a number of people – including those who identify as using twitter to discuss mental health issues, or who have been previously subjected to online abuse.

What is the Samaritans Radar?

The Samaritans Radar is an app that uses twitter and the feeds of twitter users who have not made their account private. The ‘user’ of the app is an individual who wishes to receive email alerts to tweets from people the user follows to certain tweets. These are tweets that the app, based upon content analysis, decides may be correlated with being depressed.

Originally any individual who uses Twitter, has not chosen to make their feed private, and who has a friend who decides to use the app, would have their tweets read, scanned and subject to analysis by the Samaritans app. There is now an process by which twitter users may opt out of being the subject of this app – although it requires contacting the Samaritans directly, and being put on a ‘white list’.

My use case and user story

I’ve written a use case and user story about myself here. It was originally part of this blog post but I removed it to make the post easier/shorter to read, and it felt clunky to have the inclusion of something so personal. However, it adds weight to the point I am trying to convey to the Samaritans so I wanted to include it somewhere.

The summary of my user story is that a piece of software that potentially analyses my tweets and alerts my friends to what it considers to be negative content has absolutely no positive effects for me. Instead it will cause me harm, making me even more self aware about how I present in a public space, and make it difficult for me to engage in relatively safe conversations online for fear that a warning might be triggered to an unknown follower.

And the use cases and user stories of others…

However I am not the only person who may be affected (either positively or negatively) by a piece of software such as the Samaritans Radar being implemented on top of Twitter.

Operating in a public space requires a degree of negotiation between the needs and wants of many individuals. And twitter is one such public space, with a very complex (existing, but always changing) social ecosystem. People use the infrastructure provided in a huge variety of ways, operating as a combination of the online ecosystem and their own personal preferences.

Therefore, to think about the Samaritans Radar and to evaluate its value, we need to think about the large number of different twitter users and potential Samaritan Radar app users. Just to mention a few possible user groups who use twitter relating to mental health:

  • Individual who has no mental health concerns and doesn’t engage in any online discussion
  • Individual who has no mental health concerns, but engages in online conversations
  • Individual with mental health concerns but doesn’t see twitter plays any role for them
  • Individual who has mental health issues, and uses Twitter as part of ongoing activism
  • Individual who has mental health issues, and engages in small conversations and gains support in small groups.
  • Individual who has mental health concerns, and doesn’t have any/little existing support mechanisms.
  • Individual who is subject to online abuse for various reasons (be that in relation to activism in other areas, gender, sexuality) – who may or may not have mental health concerns.

Each of these twitter users are likely to be followed by a variety of people – some of whom they know, some of whom they don’t, and a very large number on the whole scale in between. Although tweets from accounts which are not locked can be easily read by any individual, everyone is different in how they choose to use Twitter (as a balance of broadcast or engage), and with individual tweets, they make decisions about how public or accessible they choose to make tweets (be that in conversation, using hashtags). (EDIT – Paul Bernal has written a great post about this here)

Furthermore, individuals interact differently with different followers. A user of twitter doesn’t want or expect the same degree or type of interaction from all individuals who have made the decision to follow them.

Potential Radar users

There are also a number of different potential Radar users. These people will all be twitter users, but not everyone who has twitter will become a ‘user’ of the Samaritans Radar. For ease, I’ve pulled three main groups out:

  • Person who knows they follow people on twitter who might have mental health issues, wants to keep an eye on those people, and are able to provide meaningful support to them.
  • Person who knows they follow people on twitter who might have mental health issues, wants to keep an eye on those people, and are unable to provide meaningful support to them (this may be through not being emotionally mature/able enough to provide support to those with mental health, not knowing said people as well as they think, said recipient of ‘help’ not wishing for support from that individual etc)
  • Person who wants to hurt and abuse vulnerable people (and a lot of the conversations around online abuse show that this is a known category of individuals)

Who is the Samaritan Radar designed for?

The whole focus of the app is designed towards the ‘user’, a follower who may wish to identify when people they follow on twitter are vulnerable. Originally the privacy section of their website only mentioned the privacy of the user of the app – the person doing the monitoring – with no mention of any privacy concerns for the individuals who are being followed. You can argue that tweets are in the public domain therefore there are no privacy concerns (something I disagree with, but that’s another post), but you cannot deny that the app very clearly is designed for the follower, not for the specific benefit of an individual with a possible mental health issue. An app designed for an individual with mental health problems would be very unlikely to remove the agency of the individual in such a fashion.

Beyond this, it feels like Radar has been set up with a very specific type of Twitter user in mind and specific type of Radar user in mind. The type of Radar user appears to be the individual who will do no/little harm – either deliberate or accidental – through their engagement with a specific person with possible mental health issues that an app is flagging up. And the type of Twitter user is someone who is (for a variety of reasons) open about their mental health and depressive spells on Twitter, and wants people to look out for them and isn’t able to flag concerns to an existing support network.

And this itself is a laudable aim. And really does have the benefit to do much good.

But the major problem comes in assuming that this very specific case is the only use an app such as the Samaritans Radar could be put to, that the existence of the Samaritans Radar does no harm/limited harm or a comparatively insignificant amount of harm to other individuals (compared to the benefits that may be provided to other individuals) and assuming by default that Twitter users want to allow their followers the option to monitor their tweets in this fashion.

Radar users may not be benevolent. Online abuse can and does take place – with individuals being bullied, told they should go kill themselves. If online monitoring of individuals (with mental health problems or not) was made even easier, that seems like a significant probable cause of harm – and something a mental health charity should not be enabling.

Radar users may be less than great at identifying when and how someone with mental health issues wants to talk. If individuals need to be poked to remember to check on someone, they very much might not be the type of person someone with mental health problems wants to talk to.

The Samaritans Radar only introduced an opt out system after a number of people on Twitter raised concerns, but there has not yet been any recognition from them that the existence of such an app, opt in or opt out, affects the social ecosystem of Twitter. And this ecosystem includes a large number of individuals who are already finding ways to support themselves and some effects of mental health problems online.

A number of these individuals are now stating that how Samaritans Radar is implemented affects the ecosystem they operate. Yes, this may be self reported, but from vulnerable individuals – exactly the type of vulnerable individuals with whom the Samaritans should be concerned – it should not be taken lightly.

At the same time, we’ve not seen any evidence of the benefits of the app – despite much talk of work with academics, and time spent in the apps development.

And before potentially inflicting harm, ignoring critical comments from users with exactly the type of condition an app is supposed to help, and rolling an app out globally, there should be good reason to believe that the overall benefits outweigh the overall costs.

So I have a couple of question to throw to the Samaritans:

  • What, if any, work or assessment, have you done on evaluating the benefits that may be provided by the Samaritan Radar?
  • What, if any, work or assessment have you done on considering the user stories other than the idea of a benevolent and capable Radar user, and a Twitter user who uses Twitter broadcasting a need for help?
  • What, if any, work or assessment have you done of the negative impacts such an app may have on the existing support networks of those with mental health issues who use twitter?
  • What, if any, work or assessment have you done of the impact of having an opt in Samaritans Radar app? Was an opt in app ever considered until the initial backlash on Twitter? If so, why was it rejected?

I look forward to their response.

Lobbying and disciplinary infighting – learned societies of old

Learned societies and professional organisations have some very interesting histories written up on their websites – and sometimes these can be quite insightful (as well as providing me with something to chuckle at).

Tonight I have been reading about the history of the ‘Institution of Structural Engineers‘, set up in 1897 as ‘The Concrete Institute’. And I’ve found two quotes in the official history they publish online that I want to share, as they touch on interesting issues I’ve been thinking about.

Disciplinary infighting

Sir Henry Tanner in his presidential address made the first proposal for the Institute to broaden its scope and become the Institution of Structural Engineers but through its editorial Concrete and Constructional Engineering responded by stating it regarded the term “structural engineer” as one which described steel contractors and failed civil engineers

There is no obvious citation, so I’ve not (yet) checked the veracity of this apparent retort from the journal – but I find it interesting to see some suggestion that even back in the early 20th century there was disagreement over disciplinary boundaries and what was/wasn’t an ‘acceptable’ discipline.

Lobbying

There is a lot I want to consider about modern learned societies/professional organisations and their role in lobbying – but there is much in the research literature that discusses their role in finding public funding in the 19th/20th centuries. However, this quote from the Institute of Structural Engineers is the first I’ve come across which boasts of pushing for specific laws/regulations. While, again, I’ve not (yet) checked any original resources, at present the Institution publishes this on their website:

On 22nd February 1909, the Institute was incorporated under The Companies Act (1862-1907) and much of the time and energy of the new body was spent on ensuring reinforced concrete was accepted by the London County Council Regulations and the London Building Acts.

.

I mean, I’m sure the ‘Concrete Institute’ would have no vested interest in making sure concrete was accepted by Council Regulations and Building Acts, and that it was specifically done for ‘the good of all of London’.

Right…?

How will I know if it’s a learned society?

For a while I’ve been thinking about academic publishing and the problems faced by learned societies as a transition is being made away from libraries buying journal subscriptions. And this has started me off down lines of thinking about the activities carried out by learned societies (which will be the subject of later blog posts), accountability and the role or function of learned societies in a broader research landscape. But I’ve come up against a very fundamental question:

What actually is a learned society?

I’ve been doing some research into these organisations, and I’ve spent a few days staring at at various websites and documents.

But one thing I still don’t know is what a learned society actually *is*.

Organisations which either claim, or which others claim, to be learned societies vary wildly in many attributes.

Many are charities, but others are not. I’ve found several which have fewer than 100 members in total, many which provide you with no information about membership size, and one which claims to be over 80,000 in size – which seems to be the result of adding of the total size of each of their member organisations together, while not in any way accounting for the fact that people may be members of multiple organisations (In fairness I should add that this number seems to only exist now in past consultation responses they’ve written, and they seem to have removed it from their website).

And membership models also vary significantly. In some instances, you may only become a member after being nominated (which must be an *excellent* way to develop diversity in your organisation, and I’m sure organisations like this are the societies least filled with white, middle class, university educated, grey haired men), and in one case I’ve seen, your nomination is then put up to a membership vote in which you must achieve a 4:1 ratio of yes to no votes.

The research I’ve done seems to suggest that in the sciences, seconding by multiple existing members is fairly typical – which seems another great way to specifically exclude the ‘wrong sort’ from your group. Many societies specifically state they welcome anyone with an interest but require you to state your institution (which is not something many people outside of students/academia will have) or have membership options specifically defining job roles/experience which are only found within academia.

There are surprisingly few societies who seem to have fully thought through the idea of opening membership to even all those with a professional interest, let alone those with an amateur interest – despite the fact that almost every learned society I’ve looked at specifically talking about ‘educating the public’ somewhere in their charter or aims. There are some who clearly have thought this through, for instance the British Society of Soil Science, and I was also really impressed with their commitment to reduced fees for those in countries considered low-income by the World Bank. It’s not perfect – but much better than many societies I’ve looked at, where membership costs often increase for all countries outside the UK or EU.

Some learned societies are large, professional organisations with slick and shiny websites (like this one), with professional staff, amazing looking libraries and venues which may be rented out. Others are run by a group of volunteers, often academics working in their ‘spare time’, and are decidedly less professional in their appearance (I love this website – you can just imagine someone saying “we need a background image that will make people think of the cold”), and have no physical facilities. And there seems to be almost everything in between.

There are organisations which randomly have pictures of Harrison Ford on their front page (leading me to ask if there are perhaps no glycobiologists they are willing to show off), and others which are incredibly succinct in the history of their organisation.

I’ve seen societies which are focused on very specific sub disciplines (eg. the Society for the Study of Inborn Errors of Metabolism), and societies that are much more umbrella organisations (the Institute of Physics or Royal Society spring to mind here). There are even societies which focus on specific counties, such as the Devonshire Association dedicated to “the study and appreciation of all matters relating to Devon” – although I am slightly surprised to have not found an equivalent for Cornwall.

The sheer number of learned societies and scholarly societies is mind-numbing. My current list (which I’ll publish when I’ve done a bit more tidying up) consists of about 500 associations and organisations. And this is in no way complete, and mostly focuses *just* on organisations based in the UK. And the societies are not in any way obviously distinct organisations. For some reason, people feel the existence of both a British Mycological Society and a British Society for Medical Mycology to be necessary. And for there to be a British Society for Histocompatibility and Immunogenetics and a British Society for Immunology. And there are many, many other examples of overlapping societies with very similar remits and only slightly different names – reminding me of the People’s Front of Judea.

And for almost every UK based learned society, there is an equivalent in most other European countries and in the US (I’ve not yet looked at countries outside of this, although I intended to). And then there are also organisations covering the same or similar disciplines which operate at both the European and international levels. And all of these organisations fund and help support each others in a a really fascinating and intricate fashion.

One thing I can say for certain about learned and scholarly societies is quite how varied they are. But despite a few days exploring various UK based learned societies, I’m still no closer to what a ‘learned society’ or ‘scholarly society’ actually is. Wikipedia states that a learned society is “an organization that exists to promote an academic discipline or profession”, but it is incredibly easy to justify almost any activity as ‘promotion’ of a discipline – especially if organisations don’t step back to evaluate the effectiveness of these activities.

Many learned societies I’ve explored provide grants and bursaries, especially to early career researchers, in their specific sub-discipline, and many (although by no means all) seem to run some form of regular conference or have an ‘official journal’. But again how the journal is produced varies significantly: it may be published by the society or external/commercial publishers. It may be produced online only, or may be published in hard copy as well.

And the reason why I want to think about these activities and learned societies is that they have often been used to justify the retention of subscription based journal models or to justify a very slow progression away from them, with advocates from these societies and publishers making clearly stated, but often untested, beliefs about the value these organisations provide to the academic community. And I believe it’s important that we start to think about that.

So at present, I’ve been using a definition of a learned society/scholarly society as ‘any organisation that has been defined as being a learned society by either themselves or another organisation’, but that’s not very robust. Especially as anyone can define their own organisation or another organisation as one.

One possibility is to only include learned societies which make money from publishing – but as I want to explore business models that don’t necessarily rely upon this, that seems to negate part of the point. How do I separate a ‘learned society’ away from an amateur society, which may also have a published, peer reviewed journal? (For instance does the Amateur Entomologists Society count as a learned society or not?).

I don’t want to focus on whether or not an organisation is only open to academic members, as I’m the type of open minded individual who would like to see these societies open up to include non professional researchers and interested individuals.

And similarly, I don’t want to define a society by a specific section of activities, as that again seems to make some assumptions about what such a body ‘should’ do.

So many people say learned societies and scholarly societies are incredibly valued by academics, and a crucial part of the research landscape. But does anyone know what one actually is? Any thoughts below the line please!(*)

(*) This is not a rhetorical device – please do put any ideas you have below in the comments section below. I would love to know what I should be using as a working definition for learned societies as I continue exploring this area.

The sheer scale of hybrid journal publishing

The last few years have seen a significant rise in what are termed ‘hybrid open access journals’, where only some of the articles are freely available to read and a subscription is still required to read the remainder. As many journals require payment from authors to publish in this fashion, then university libraries need to pay subscriptions to read the remaining articles, publishers are in effect being paid twice for the same work.

With recently published data from the Wellcome Trust, the scale of this double charging has become much more clear.

In Oct 2012 – Sept 2013, academics spent £3.88 million to publish articles in journals with immediate online access – of which £3.17 million (82 % of costs, 74 % of papers) was paying for publications that Universities would then be charged again for. For perspective, this is a figure slightly larger than the Wellcome Trust paid in 2012/2013 on their Society & Ethics portfolio.

Only £0.70 million of the charity’s £3.88m didn’t have any form of double charging (ie, was published in a “Pure Open Access” journal) – with this total being dominated by articles published in PLOS and BioMed Central journals (68 % of total ‘pure’ hybrid journal costs, 80 % of paper total).

Top 5 publishers by total cost to Wellcome Trust

Publisher

No. of articles

Maximum Cost

Average Cost

Total Cost (nearest £1000)

Elsevier (inc. Cell Press)

418

£5,760

£2,448.158

£1,036,000

Wiley-Blackwell

271

£3,078.92

£2,009.632

£545,000

PLOS

307

£3,600

£1,139.286

£350,000

Oxford University Press

167

£3,177.60

£1,850.099

£300,000

Nature Publishing Group (not inc. Frontiers)

80

£3,780

£2,696.396

£216,000

Top 5 publishers by total cost to Wellcome Trust – separated into money spent on author charges for articles appearing in hybrid and pure open access journals

Publisher

Journal Type

No. of articles

Max Cost

Average Cost

 Total Cost (nearest £)

Elsevier

Hybrid

402

£5,760

2,443.28

£982,199

Pure OA

21

£3,996

2,541.48

£53,371

Wiley-Blackwell

Hybrid

263

£3,026

2,010.88

£528,862

Pure OA

8

£3,079

1,968.60

£15,749

PLOS

Hybrid

0

£0

£0

£0

Pure OA

307

£3,600

1,139.29

£349,761

Oxford University Press

Hybrid

135

£3,177.6

2,004.14

£270,558

Pure OA

32

£2,184

1,200.25

£38,408

Nature Publishing Group

Hybrid

67

£3,780

2,867.82

192,143.71

Pure OA

13

£2,880

1,812.923

23,568

Wellcome Trust pays nearly £1 million to Elsevier, and pays over £500,000 to Wiley-Blackwell to make articles freely available on point of publication, in journals that a university library will also be trying to find money to also pay subscription fees to. These are outrageously high sums of money! Especially given a recent explosion in the number of journals, and an increase in journal prices, means even well-funded libraries can no longer afford the cost of subscribing to many journals!

Journal articles should be published in a way that means they are freely available – and not just to academics, but also to wider public audiences. And I’m not critical of article processing charges. However, I’m unsure how any publisher can justify charging an academic an average cost of £2,443 to publish in a journal that is already being supported by library subscriptions from not just one university, but many universities around the world. And surely no cost based model should charge more for publication in a hybrid journal with multiple funding streams than in one supported purely on author charges (as appears to be the case with Wiley-Blackwell).

Data

Data source found here

Original data: Kiley, Robert (2014): Wellcome Trust APC spend 2012-13: data file. figsharehttp://dx.doi.org/10.6084/m9.figshare.963054

Enhancements on original data made by Cameron Neylon: https://github.com/cameronneylon/apcs

The Politics of Hack Days

One of the big values of a hack day is taking datasets away from their original owners, and getting a bunch of tech-focused people to play about with them. You remove the preconceptions that the owner may hold about the data, and see what happens when a different group of people uses that set of information.

One of the big values of a hack day is taking datasets away from their original owners, and getting a bunch of tech-focused people to play about with them. You remove the preconceptions that the owner may hold about the data, and see what happens when a different group of people uses that set of information.

The direction taken in a hack day environment is a function of those present in the room, and the datasets provided. Both knowledge and values guide what an individual or group thinks to do with a given data set, and decisions are made about what other data sets are used as comparison or in combination, what context these data sets are placed in, what assumptions (both conscious and subconscious) are made, and how any outputs are visualised or understood.

If we want to produce non-partisan outputs, we need to balance as far as possible the political biases in the input provided – both in terms of data and expertise. And this means that the decisions made by hack day organisers in terms of who is providing guidance and which data sets are provided or pointed to shouldn’t be taken lightly.

Campaigning organisations are driven by particular political objectives, and this informs the questions they ask, and the data they chose to collect and subsequently release. We can’t (and shouldn’t) collect all data about every person, group or organisation. Even organisations which are attempting to be non-political will make value judgements about what is (and what isn’t) important data to collect. And this effect will be much stronger – both subconsciously and consciously – in datasets produced by campaigning organisations. Those with strong political objectives and ideologies will also suggest different things to do with a specific dataset.

Therefore, if you run a non-partisan high-profile hack day focusing on critical analysis of Government, it might be prudent to not have this event partly sponsored by a right-wing small-state lobby group, unless you balance this. It might be wise to ensure you have an equal and opposite force with similar prominence in the room, providing guidance and datasets.

I’m not saying exclude guidance and datasets from those with strong political ideology. In fact, to ensure disruptive outcomes, you want to ensure you have many strong views in a room.

However, if the aim of your hack day is to produce something useful and informed by the widest amount of information available, you want to ensure these influences are balanced and diverse.

It can’t be that hard.