For this Module’s (2) Getting Started Challenge I would like you to start looking at potential topics for your Final Media EthicsFor this Module’s (2) Getting Started Challenge I would like you to start looking at potential topics for your Final Media Ethics

SEPTEMBER 2017 | VOL. 60 | NO. 9 | COMMUNICATIONS OF THE ACM 65

WHILE THE INTERNET has the potential to give people ready access to relevant and factual information, social media sites like Facebook and Twitter have made filtering and assessing online content increasingly difficult due to its rapid flow and enormous volume. In fact, 49% of social media users in the U.S. in 2012

received false breaking news through social media.8 Likewise, a survey by Silverman11 suggested in 2015 that false rumors and misinformation

disseminated further and faster than ever before due to social media. Polit- ical analysts continue to discuss mis- information and fake news in social media and its effect on the 2016 U.S. presidential election.

Such misinformation challenges the credibility of the Internet as a venue for authentic public informa- tion and debate. In response, over the past five years, a proliferation of out- lets has provided fact checking and debunking of online content. Fact- checking services, say Kriplean et al.,6 provide “… evaluation of verifiable claims made in public statements through investigation of primary and secondary sources.” An international

Trust and Distrust in Online Fact-Checking Services

DOI:10.1145/3122803

Even when checked by fact checkers, facts are often still open to preexisting bias and doubt.

BY PETTER BAE BRANDTZAEG AND ASBJØRN FØLSTAD

key insights ˽ Though fact-checking services play

an important role countering online disinformation, little is known about whether users actually trust or distrust them.

˽ The data we collected from social media discussions—on Facebook, Twitter, blogs, forums, and discussion threads in online newspapers—reflects users’ opinions about fact-checking services.

˽ To strengthen trust, fact-checking services should strive to increase transparency in their processes, as well as in their organizations, and funding sources.

66 COMMUNICATIONS OF THE ACM | SEPTEMBER 2017 | VOL. 60 | NO. 9

contributed articles

more political or controversial issues a fact-checking service covers, the more it needs to build a reputation for usefulness and trustworthiness.

Research suggests the trustwor- thiness of fact-checking services depends on their origin and owner- ship, which may in turn affect integ- rity perceptions10 and the transpar- ency of their fact-checking process.4 Despite these observations, we are unaware of any other research that has examined users’ perceptions of these services. Addressing the gap in current knowledge, we investigated the research question: How do so- cial media users perceive the trust- worthiness and usefulness of fact- checking services?

Fact-checking services differ in terms of their organizational aim and funding,10 as well as their areas of concern,11 that in turn may affect their trustworthiness. As outlined in Figure 1, the universe of fact- checking services can be divided into three general categories based on their area(s) of concern: political and public statements in general, corre- sponding to the fact checking of poli- ticians, as discussed by Nyhan and Reifler;9 online rumors and hoaxes, reflecting the need for debunking services, as discussed by Silverman;11

ing has scarcely paid attention to the general public’s view of fact check- ing, focusing instead on how peo- ple’s beliefs and attitudes change in response to facts that contradict their own preexisting opinions. This re- search suggests fact checking in gen- eral may be unsuccessful at reducing misperceptions, especially among the people most prone to believe them.9 People often ignore facts that contradict their current beliefs,2,13 particularly in politics and controver- sial social issues.9 Consequently, the

census from 2017 counted 114 active fact-checking services, a 19% increase over the previous year.12 To benefit from this trend, Google News in 2016 let news providers tag news articles or their content with fact-checking information “… to help readers find fact checking in large news stories.”3 Any organization can use the fact- checking tag, if it is non-partisan, transparent, and targets a range of claims within an area of interest and not just one single person or entity.

However, research into fact check-

Figure 3. Outline of our research approach; posts collected October 2014 to March 2015.

Blogs

Discussion forums

Online newspaper comments

Data Corpus 1,741 posts

Search Meltwater

Filter irrelevant posts

Content analysis

Dataset 595 posts

Findings Trustworthiness and usefulness

Figure 2. Example of Snopes debunking a social media rumor on Twitter (March 6, 2016); https://twitter.com/snopes/ status/706545708233396225

Figure 1. Categorization of fact-checking services based on areas of concern.

Fact-checking services’ areas of concern

Online rumors and hoaxes

Political and public claims

Specific topics or controversies

Snopes.com

Hoax-Slayer

ThruthOrFiction.com

HoaxBusters

Viralgranskaren – Metro

FactCheck.org

PolitiFact

The Washington Post Fact Checker

CNN Reality Check

Full Fact

StopeFake

TruthBeTold

#RefugeeCheck

Climate Feedback

Brown Moses Blog (continued as Bellingcat)

Table 1. Coding scheme we used to analyze the data.

Theme Sentiment Service described as

Usefulness Positive Useful, serving the purpose of fact checking

Negative Not as useful, often derogatory

Ability Positive Reputable, expert, or acclaimed

Negative Lacking expertise or credibility

Benevolence

Positive Aiming for greater (social) good

Negative Suspected of (social) ill will (such as through conspiracy, propaganda, or fraud)

Integrity Positive Independent or impartial

Negative Dependent or partially or politically biased

SEPTEMBER 2017 | VOL. 60 | NO. 9 | COMMUNICATIONS OF THE ACM 67

contributed articles

as through Facebook and Twitter. Fig- ure 2 is an example of a Twitter post with content checked by Snopes.

Analyzing Social Media Conversations To explore how social media users perceive the trustworthiness and use- fulness of these services, we applied a research approach designed to take advantage of unstructured social me- dia conversations (see Figure 3).

While investigations of trust and usefulness often rely on structured data from questionnaire-based surveys, social media conversations repre- sent a highly relevant data source for our purpose, as they arguably reflect the raw, authentic percep- tions of social media users. Xu et al.16 claim it is beneficial to listen to, analyze, and understand citizens’ opinions through social media to im- prove societal decision-making processes and solutions. They wrote, for example, “Social media analytics has been applied to explain, detect, and predict disease outbreaks, election results, macroeconomic processes (such as crime detec- tion), (… ) and financial markets (such as stock price).”16 Social me- dia conversations take place in the everyday context of users likely to be engaged in fact-checking services. This approach may provide a more unbiased view of people’s percep- tions than, say, a questionnaire- based approach. The benefit of gathering data from users in their specific social media context does not imply that our data is repre- sentative. Our data lacks impor- tant information about user de- mographics, limiting our ability to claim generality for the entire user population. Despite this potential drawback, however, our data does offer new insight into how social media users view the usefulness and trustworthiness of various cat- egories of fact-checking services.

For data collection, we used Meltwater Buzz, an established ser- vice for social media monitoring. crawling data from social media conversations in blogs, discussion forums, online newspaper discus- sion threads, Twitter, and Facebook. Meltwater Buzz crawls all blogs (such

and specific topics or controversies or particular conflicts or narrowly scoped issues or events (such as the ongoing Ukraine conflict).

We have focused on three ser- vices—Snopes, FactCheck.org, and StopFake—all included in the Duke Reporters’ Lab’s online overview of fact checkers (http://reporterslab.org/ fact-checking/). They represent three categories of fact checkers, from on- line rumors to politics to a particular topic, as in Figure 1, and differences in organization and funding. As a mea- sure of their popularity, as of June 20, 2017, Snopes had 561,650 likes on Facebook, FactCheck.org 806,814, and StopFake 52,537.

We study Snopes because of its aim to debunk online rumors, fitting the first category in Figure 1. This aim is shared by other such services, including HoaxBusters and the Swed- ish service Viralgranskaren. Snopes is managed by a small volunteer or- ganization that has emerged from a single-person initiative and funded through advertising revenue.

We study FactCheck.org because it monitors the factual accuracy of what is said by major political fig- ures. Other such services include PolitiFact (U.S.) and Full Fact (U.K.) in the second category in Figure 1. FactCheck.org is a project of the An- nenberg Public Policy Center of the Annenberg School for Communica- tion at the University of Pennsylva- nia, Philadelphia, PA. FactCheck.org is supported by university funding and individual donors and has been a source of inspiration for other fact- checking projects.

We study StopFake because it ad- dresses one highly specific topic— the ongoing Ukraine conflict. It thus resembles other highly focused fact-checking initiatives (such as #Refugeecheck, which fact checks reports on the refugee crises in Eu- rope). StopFake is an initiative by the Kyiv Mohyla Journalism School in Kiev, Ukraine, and is thus a Eu- ropean-based service. Snopes and FactCheck.org are U.S. based, as are more than a third of the fact- checking services identified by Duke Reporters’ Lab.12

All three provide fact checking through their own websites, as well

Consequently, the more political or controversial issues a fact-checking service covers, the more it needs to build a reputation for usefulness and trustworthiness.

68 COMMUNICATIONS OF THE ACM | SEPTEMBER 2017 | VOL. 60 | NO. 9

contributed articles

to reflect how people start a sentence when formulating their opinions. StopFake is a relatively less-known service. We thus selected a broad- er search string—“StopFake”—to be able to collect enough relevant opinions. The searches returned a data corpus of 1,741 posts over six months—October 2014 to March 2015—as in Figure 3. By “posts,” we mean written contributions by indi- vidual users. To create a sufficient dataset for analysis, we removed all duplicates, including a small number of non-relevant posts lacking person- al opinions about fact checkers. This filtering process resulted in a dataset of 595 posts.

We then performed content analy- sis, coding all posts to identify and investigate patterns within the data1 and reveal the perceptions users ex- press in social media about the three fact-checking services we investigat- ed. We analyzed their perceptions of the usefulness of fact-checking ser- vices through a usefulness construct similar to the one used by Tsakonas et al.14 “Usefulness” concerns the ex- tent the service is perceived as benefi- cial when doing a specific fact-check- ing task, often illustrated by positive recommendations and characteriza- tions (such as the service is “good” or “great”). Following Mayer et al.’s theoretical framework,7 we catego- rized trustworthiness according to the perceived ability, benevolence, and integrity of the services. “Ability” concerns the extent a service is per- ceived as having available the needed skills and expertise, as well as being reputable and well regarded. “Benev- olence” refers to the extent a service is perceived as intending to do good, beyond what would be expected from an egocentric motive. “Integrity” tar- gets the extent a service is generally viewed as adhering to an acceptable set of principles, in particular being independent, unbiased, and fair.

Since we found posts typically re- flect rather polarized perceptions of the studied services, we also grouped the codes manually according to sen- timent, positive or negative. Some posts described the services in a plain and objective manner. We thus coded them using a positive sentiment (see Table 1) because they refer to the

of more than 500 members. This limitation in Facebook data partly explains why the overall number of posts we collected—1,741—was not more than it was.

To collect opinions about social media user perceptions of Snopes and FactCheck.org, we applied the search term “[service name] is,” as in “Snopes is,” “FactCheck.org is,” and “FactCheck is.” We intended it

as https://wordpress.com/), discus- sion forums (such as https://offtopic. com/), and online newspapers (such as https://www.washingtonpost. com/) requested by Meltwater cus- tomers, thus representing a large, though convenient, sample. It col- lects various amounts of data from each platform; for example, it crawls all posts on Twitter but only the Face- book pages with 3,500 likes or groups

Figure 4. Positive and negative posts related to trustworthiness and usefulness per fact-checking service (in %); “other” refers to posts not relevant for the research categories (N = 595 posts).

Positive (total)

Negative (total)

Usefulness (positive)

Ability (positive)

Benevolence (positive)

Integrity (positive)

Usefulness (negative)

Ability (negative)

Benevolence (negative)

Integrity (negative)

Other

0% 20% 40% 60% 80% 100%

Snopes (n = 385) FactCheck.org (n = 80) StopFake (n = 130)

Table 2. Snopes and themes we analyzed (n = 385).

Theme Sentiment Example

Usefulness

Positive (21%) Snopes is a wonderful Website for verifying things seen online; it is at least a starting point for research.

Negative (10%) Snopes is a joke. Look at its Boston bombing debunking failing to debunk the worst hoax ever …

Ability

Positive (6%) […] Snopes is a respectable source for debunking wives’ tales, urban legends, even medical myths …

Negative (24%) Heh … Snopes is a man and a woman with no investigative background or credentials who form their opinions solely on Internet research; they don’t interview anyone. […]

Benevolence

Positive (0%) No posts

Negative (21%) You show your Ignorance by using Snopes … Snopes is a NWO Disinformation System designed to fool the Masses … SORRY. I Believe NOTHING from Snopes. Snopes is a Disinformation vehicle of the Elitist NWO Globalists. Believe NOTHING from them … […]

Integrity

Positive (2%) Snopes is a standard, rather dull fact-checking site, nailing right and left equally. […]

Negative (44%) Snopes is a leftist outlet supported with money from George Soros. Whatever Snopes says I take with a grain of salt …

SEPTEMBER 2017 | VOL. 60 | NO. 9 | COMMUNICATIONS OF THE ACM 69

contributed articles

crediting a service. Posts expressing positive sentiment mainly argue for the usefulness of the service, claim- ing that Snopes is, say, a useful re- source for checking up on the veracity of Internet rumors.

FactCheck.org. The patterns in the posts we analyzed for FactCheck.org resemble those for Snopes. As in Ta- ble 3, the most frequently mentioned trustworthiness concerns related to service integrity; as for Snopes, us- ers said the service is politically bi- ased toward the left. Posts concern- ing benevolence and ability were also relatively frequent, reflecting user

service as a source for fact checking, and users are likely to reference fact- checking sites because they see them as useful.

For reliability, both researchers in the study did the coding. One coded all the posts, and the second then went through all the assigned codes, a process repeated twice. Finally, both researchers went through all comments for which an alternative code had been suggested to decide on the final coding, a process that recommended an alternative coding for 153 posts (or 26%).

A post could include more than one of the analytical themes, so 30% of the posts were thus coded as ad- dressing two or more themes.

Results Despite the potential benefits of fact- checking services, Figure 4 reports the majority of the posts on the two U.S.-based services expressed nega- tive sentiment, with Snopes at 68% and FactCheck.org at 58%. Most posts on the Ukraine-based StopFake (78%) reflected positive sentiment.

The stated reasons for negative sentiment typically concerned one or more of the trustworthiness themes rather than usefulness. For example, for Snopes and FactCheck.org, the negative posts often expressed con- cern over lack in integrity due to per- ceived bias toward the political left. Negative sentiment pertaining to the ability and benevolence of the servic- es were also common. The few critical comments on usefulness were typi- cally aimed at discrediting a service, by, say, characterizing it as “satirical” or as “a joke.”

Positive posts were more often re- lated to usefulness. For example, the stated reasons for positive sentiment toward StopFake typically concerned the service’s usefulness in countering pro-Russian propaganda and trolling and in the information war associat- ed with the ongoing Ukraine conflict.

In line with a general notion of an increasing need to interpret and act on information and misinforma- tion in social media,6,11 some users included in the study discussed fact- checking sites as important elements of an information war.

Snopes. The examples in Table

2 reflect how negative sentiment in the posts we analyzed on Snopes was rooted in issues pertaining to trust- worthiness. Integrity issues typically involved a perceived “left-leaning” political bias in the people behind the service. Pertaining to benevo- lence, users in the study said Snopes is part of a larger left-leaning or “lib- eral” conspiracy often claimed to be funded by George Soros, whereas comments on ability typically tar- geted lack of expertise in the people running the service. Some negative comments on trustworthiness may be seen as a rhetorical means of dis-

Table 3. FactCheck.org and themes we analyzed (n = 80).

Theme Sentiment Example

Usefulness

Positive (25%) […] You obviously haven’t listened to what they say. Also, I hate liars. FactCheck is a great tool.

Negative (3%) Anyway, “FactCheck” is a joke […]

Ability

Positive (6%) The media sources I use must pass a high credibility bar. FactCheck. org is just one of the resources I use to validate what I read …

Negative (16%) […] FactCheck is NOT a confidence builder; see its rider and sources, Huffpo articles … REALLY?

Benevolence

Positive (0%) No posts

Negative (25%) FactCheck studies the factual correctness of what major players in U.S. politics say in TV commercials, debates, talks, interviews, and news presentations, then tries to present the best possible fictional and propaganda-like version for its target […]

Integrity

Positive (19%) When you don’t like the message, blame the messenger. FactCheck is nonpartisan. It’s just that conservatives either lie or are mistaken more …

Negative (39%) FactCheck is left-leaning opinion. It doesn’t check facts …

Table 4. StopFake and themes we analyzed (n = 130); note * also coded as integrity/positive.

Theme Sentiment Example

Usefulness

Positive (72%) Don’t forget a strategic weapon of the Kremlin is the “web of lies” spread by its propaganda machine; see antidote http://www.stopfake. org/en/news

Negative (2%) […] StopFake! HaHaHa. You won, I give up. Next time I will quote “Saturday Night Live”; there is more truth:)) …

Ability

Positive (2%) […] by the way, the website StopFake.org is a very objective and accurate source exposing Russian propaganda and disinformation techniques. […]*

Negative (2%) […] Ha Ha … a flow of lies is constantly sent out from the Kremlin. Really. If so, StopFake needs updates every hour, but the best way it can do that is to find low-grade blog content and make it appear as if it was produced by Russian media […]

Benevolence

Positive (4%) […] StopFake is devoted to exposing Russian propaganda against the Ukraine. […]

Negative (14%) So now you acknowledge StopFake is part of Kiev’s propaganda. I guess that answers my question […]

Integrity

Positive (2%) […] by the way, the website StopFake.org is a very objective and accurate source exposing Russian propaganda and disinformation techniques. […]

Negative (11%) […] Why should I give any credence to StopFake.org? Does it ever criticize the Kiev regime, in favor of the Donbass position? […]

70 COMMUNICATIONS OF THE ACM | SEPTEMBER 2017 | VOL. 60 | NO. 9

contributed articles

tion when comparing the various services, topic-specific StopFake is perceived as more useful than Snopes and FactCheck.org. One reason might be that a service targeting a specific topic faces less criticism because it attracts a particular audience that seeks facts supporting its own view. For example, StopFake users target anti-Russian, pro-Ukrainian readers. Another, more general, reason might be that positive perceptions are mo- tivated by user needs pertaining to a perceived high load of misinforma- tion, as in the case of the Ukraine conflict, where media reports and social media are seen as overflowing with propaganda. Others highlighted the general ease information may be filtered or separated from misinfor- mation through sites like Snopes and FactCheck.org, as expressed like this:

“As you pointed out, it doesn’t take that much effort to see if something on the Internet is legit, and Snopes is a great place to start. So why not take that few seconds of extra effort to do that, rather than creating and sharing misleading items.”

This finding suggests there is in- creasing demand for fact-checking services,6 while at the same time a substantial proportion of social me- dia users who would benefit from such services do not use them suf- ficiently. The services should thus be even more active on social media sites like Facebook and Twitter, as well as in online discussion forums, where greater access to fact checking is needed.

Trustworthiness. Negative percep- tions and opinions about fact-check- ing services seem to be motivated by basic distrust rather than rational

argument. For some users in our sample, lack of trust extends beyond a particular service to encompass the entire social and political system. Us- ers with negative perceptions thus seem trapped in a perpetual state of informational disbelief.

While one’s initial response to statements reflecting a state of infor- mational disbelief may be to dismiss them as the uninformed paranoia of a minority of the public, the state- ments should instead be viewed as a source of user insight. The reason the services are often unsuccessful in re- ducing ill-founded perceptions9 and people tend to disregard fact check- ing that goes against their preexisting beliefs2,13 may be a lack of basic trust rather than a lack of fact-based argu- ments provided by the services.

We found such distrust is often highly emotional. In line with Sil- verman,11 fact-checking sites must be able to recognize how debunking and fact checking evoke emotion in their users. Hence, they may benefit from rethinking the way they design and present themselves to strengthen trust among users in a general state of informational disbelief. More- over, users of online fact-checking sites should compensate for the lack of physical evidence online by be- ing, say, demonstrably independent, impartial, and able to clearly distin- guish fact from opinion. Rogerson10 wrote that fact-checking sites exhibit varying levels of rigor and effective- ness. The fact-checking process and even what are considered “facts” may in some cases involve subjective in- terpretation, especially when actors with partial ties aim to provide the service. For example, in the 2016 U.S. presidential campaign, the organiza- tion “Donald J. Trump for President” invited Trump’s supporters to join a fact-check initiative, similar to the category “topics or controversies,” urging “fact checking” the presiden- tial debates on social media. How- ever, the initiative was criticized as mainly promoting Trump’s views and candidacy.5

Users of fact-checking sites ask: Who actually does the fact checking and how do they do it? What organi- zations are behind the process? And how does the nature of the organiza-

concern regarding the service as a contributor to propaganda or doubts about its fact-checking practices.

StopFake. As in Table 4, the results for StopFake show more posts ex- pressing positive sentiment than we found for Snopes and FactCheck.org. In particular, the posts included in the study pointed out that StopFake helps debunk rumors seen as Russian propaganda in the Ukraine conflict.

Nevertheless, the general pat- tern in the reasons users gave us for positive and negative sentiment for Snopes and FactCheck.org also held for StopFake. The positive posts were typically motivated by usefulness, whereas the negative posts reflected the sentiment that StopFake is politi- cally biased (“integrity”), a “fraud,” a “hoax,” or part of the machinery of Ukraine propaganda (“benevo- lence”).

Discussion We found users with positive percep- tions typically extoled the usefulness of fact-checking services, whereas users with negative opinions cited concerns over trustworthiness. This pattern emerged across all three ser- vices. In the following sections, we discuss how these findings provide new insight into trustworthiness as a key challenge when countering on- line rumors and misinformation2,9 and why ill-founded beliefs may have such online reach, even though the beliefs are corrected by prominent fact checkers, including Snopes, FactCheck.org, and StopFake.

Usefulness. Users in our sample with a positive view of the services mainly pointed to their usefulness. While everyone should exercise cau-

Table 5. Challenges and our related recommendations for fact-checking services.

Challenges Recommendations

Usefulness Unrealized potential in public use of fact-checking services

Increase presence in social media and discussion forums

Trustworthiness

Ability Critique of expertise and reputation

Provide nuanced but simple overview of the fact-checking process where relevant sources are included

Benevolence Suspicion of conspiracy and propaganda

Establish open policy on fact checking and open spaces for collaboration on fact checking

Integrity Perception of bias and partiality

Ensure transparency on organization and funding. and demonstrable impartiality in fact-checking process

SEPTEMBER 2017 | VOL. 60 | NO. 9 | COMMUNICATIONS OF THE ACM 71

contributed articles

610928, http://www.revealproject. eu/) but does not necessarily rep- resent the views of the European Commission. We also thank Marika Lüders of the University of Oslo and the anonymous reviewers for their in- sightful comments.

References 1. Ezzy, D. Qualitative Analysis. Routledge, London,

U.K., 2013. 2. Friesen, J.P., Campbell, T.H., and Kay, A.C. The

psychological advantage of unfalsifiability: The appeal of untestable religious and political ideologies. Journal of Personality and Social Psychology 108, 3 (Nov. 2014), 515–529.

3. Gingras, R. Labeling fact-check articles in Google News. Journalism & News (Oct. 13, 2016); https://blog. google/topics/journalism-news/labeling-fact-check- articles-google-news/

4. Hermida, A. Tweets and truth: Journalism as a discipline of collaborative verification. Journalism Practice 6, 5-6 (Mar. 2012), 659–668.

5. Jamieson, A. ‘Big League Truth Team’ pushes Trump’s talking points on social media. The Guardian (Oct. 10, 2016); https://www.theguardian.com/us-news/2016/ oct/10/donald-trump-big-league-truth-team-social- media-debate

6. Kriplean, T., Bonnar, C., Borning, A., Kinney, B., and Gill, B. Integrating on-demand fact-checking with public dialogue. In Proceedings of the 17th ACM Conference on Computer-Supported Cooperative Work & Social Computing (Baltimore, MD, Feb. 15–19). ACM Press, New York, 2014, 1188–1199.

7. Mayer, R.C., Davis, J.H., and Schoorman, F.D. An integrative model of organizational trust. Academy of Management Review 20, 3 (1995), 709–734.

8. Morejon, R. How social media is replacing traditional journalism as a news source. Social Media Today Report (June 28, 2012); http://www.socialmediatoday. com/content/how-social-media-replacing-traditional- journalism-news-source-infographic

9. Nyhan, B. and Reifler, J. When corrections fail: The persistence of political misperceptions. Political Behavior 32, 2 (June 2010), 303–330.

10. Rogerson, K.S. Fact checking the fact checkers: Verification Web sites, partisanship and sourcing. In Proceedings of the American Political Science Association (Chicago, IL, Aug. 29–Sept. 1). American Political Science Association, Washington, D.C., 2013.

11. Silverman, C. Lies, Damn Lies, and Viral Content. How News Websites Spread (and Debunk) Online Rumors, Unverified Claims, and Misinformation. Tow Center for Digital Journalism, Columbia Journalism School, New York, 2015; http://towcenter.org/wp- content/uploads/2015/02/LiesDamnLies_Silverman_ TowCenter.pdf

12. Stencel, M. International fact checking gains ground, Duke census finds. Duke Reporters’ Lab, Duke University, Durham, NC, Feb. 28, 2017; https:// reporterslab.org/international-fact-checking-gains- ground/

13. Stroud, N.J. Media use and political predispositions: Revisiting the concept of selective exposure. Political Behavior 30, 3 (Sept. 2008), 341–366.

14. Tsakonas, G. and Papatheodorou, C. Exploring usefulness and usability in the evaluation of open- access digital libraries. Information Processing & Management 44, 3 (May 2008), 1234–1250.

15. Van Mol, C. Improving web survey efficiency: The impact of an extra reminder and reminder content on Web survey response. International Journal of Social Research Methodology 20, 4 (May 2017), 317–327.

16. Xu, C., Yu, Y., and Hoi, C.K. Hidden in-game intelligence in NBA players’ tweets. Commun. ACM 58, 11 (Nov. 2015), 80–89.

Petter Bae Brandtzaeg (pbb@sintef.no) is a senior research scientist at SINTEF in Oslo, Norway.

Asbjørn Følstad (asf@sintef.no) is a senior research scientist at SINTEF in Oslo, Norway.

© 2017 ACM 0001-0782/17/09 $15.00

tion influence the results of the fact checking? Fact-checking sites must thus explicate the nuanced, detailed process leading to the presented re- sult while keeping it simple enough to be understandable and useful.11

Need for transparency. While fact- checker trustworthiness is critical, fact checkers represent but one set of voices in the information landscape and cannot be expected to be benevo- lent and unbiased just because they check facts. Rather, they must strive for transparency in their working pro- cess, as well as in their origins, orga- nization, and funding sources.

To increase transparency in its processes, a service might try to take a more horizontal, collaborative ap- proach than is typically seen in the current generation of services. Fol- lowing Hermida’s recommenda- tion4 to social media journalists, fact checkers could be set up as a plat- form for collaborative verification and genuine fact checking, relying less on centralized expertise. Form- ing an interactive relationship with users might also help build trust.6,7

Conclusion We identified a lack of perceived trustworthiness and a state of infor- mational disbelief as potential obsta- cles to fact-checking services reach- ing social media users most critical to such services. Table 5 summarizes our overall findings and discussions, outlining related key challenges and our recommendations for how to ad- dress them.

Given the exploratory nature of this study, we cannot conclude our findings are valid for all services. In addition, more research is needed to be able to make definite claims on systematic differences among the various fact checkers based on their “areas of concern.” Nevertheless, the consistent pattern in opinions we found across three prominent ser- vices suggests challenges and recom- mendations that can provide useful guidance for future development in this important area.

Acknowledgments This work was supported by the Eu- ropean Commission co-funded FP 7 project REVEAL (Project No. FP7-

Users with negative perceptions thus seem trapped in a perpetual state of informational disbelief.

Copyright of Communications of the ACM is the property of Association for Computing Machinery and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder’s express written permission. However, users may print, download, or email articles for individual use.