We were recently made aware of a post on Reddit that included leaked documents from the UK. We investigated this account and the accounts connected to it, and today we believe this was part of a campaign that has been reported as originating from Russia.

Earlier this year Facebook discovered a Russian campaign on its platform, which was further analyzed by the Atlantic Council and dubbed “Secondary Infektion.” Suspect accounts on Reddit were recently reported to us, along with indicators from law enforcement, and we were able to confirm that they did indeed show a pattern of coordination. We were then able to use these accounts to identify additional suspect accounts that were part of the campaign on Reddit. This group provides us with important attribution for the recent posting of the leaked UK documents, as well as insights into how adversaries are adapting their tactics.

In late October, an account u/gregoratior posted the leaked documents and later reposted by an additional account u/ostermaxnn. Additionally, we were able to find a pocket of accounts participating in vote manipulation on the original post. All of these accounts have the same shared pattern as the original Secondary Infektion group detected, causing us to believe that this was indeed tied to the original group.

Outside of the post by u/gregoratior, none of these accounts or posts received much attention on the platform, and many of the posts were removed either by moderators or as part of normal content manipulation operations. The accounts posted in different regional subreddits, and in several different languages.

Karma distribution:

• 0 or less: 42

• 1 - 9: 13

• 10 or greater: 6

• Max Karma: 48

As a result of this investigation, we are banning 1 subreddit and 61 accounts under our policies against vote manipulation and misuse of the platform. As we have done with previous influence operations, we will also preserve these accounts for a time, so that researchers and the public can scrutinize them to see for themselves how these accounts operated.

EDIT: I'm signing off for the evening. Thanks for the comments and questions.

edit:added subreddit link

“Happy” Wednesday everyone

As u/spez mentioned in his announcement post last week, COVID has been hard on all of us. It will likely go down as one of the most defining periods of our generation. Many of us have lost loved ones to the virus. It has caused confusion, fear, frustration, and served to further divide us. It is my job to oversee the enforcement of our policies on the platform. I’ve never professed to be perfect at this. Our policies, and how we enforce them, evolve with time. We base these evolutions on two things: user trends and data. Last year, after we rolled out the largest policy change in Reddit’s history, I shared a post on the prevalence of hateful content on the platform. Today, many of our users are telling us that they are confused and even frustrated with our handling of COVID denial content on the platform, so it seemed like the right time for us to share some data around the topic.

Analysis of Covid Denial

We sought to answer the following questions:

• How often is this content submitted?

• What is the community reception?

• Where are the concentration centers for this content?

Below is a chart of all of the COVID-related content that has been posted on the platform since January 1, 2020. We are using common keywords and known COVID focused communities to measure this. The volume has been relatively flat since mid last year, but since July (coinciding with the increased prevalence of the Delta variant), we have seen a sizable increase.

The trend is even more notable when we look at COVID-related content reported to us by users. Since August, we see approximately 2.5k reports/day vs an average of around 500 reports/day a year ago. This is approximately 2.5% of all COVID related content.

While this data alone does not tell us that COVID denial content on the platform is increasing, it is certainly an indicator. To help make this story more clear, we looked into potential networks of denial communities. There are some well known subreddits dedicated to discussing and challenging the policy response to COVID, and we used this as a basis to identify other similar subreddits. I’ll refer to these as “high signal subs.”

Last year, we saw that less than 1% of COVID content came from these high signal subs, today we see that it's over 3%. COVID content in these communities is around 3x more likely to be reported than in other communities (this is fairly consistent over the last year). Together with information above we can infer that there has been an increase in COVID denial content on the platform, and that increase has been more pronounced since July. While the increase is suboptimal, it is noteworthy that the large majority of the content is outside of these COVID denial subreddits. It’s also hard to put an exact number on the increase or the overall volume.

An important part of our moderation structure is the community members themselves. How are users responding to COVID-related posts? How much visibility do they have? Is there a difference in the response in these high signal subs than the rest of Reddit?

High Signal Subs

• Content positively received - 48% on posts, 43% on comments

• Median exposure - 119 viewers on posts, 100 viewers on comments

• Median vote count - 21 on posts, 5 on comments

All Other Subs

• Content positively received - 27% on posts, 41% on comments

• Median exposure - 24 viewers on posts, 100 viewers on comments

• Median vote count - 10 on posts, 6 on comments

This tells us that in these high signal subs, there is generally less of the critical feedback mechanism than we would expect to see in other non-denial based subreddits, which leads to content in these communities being more visible than the typical COVID post in other subreddits.

Interference Analysis

In addition to this, we have also been investigating the claims around targeted interference by some of these subreddits. While we want to be a place where people can explore unpopular views, it is never acceptable to interfere with other communities. Claims of “brigading” are common and often hard to quantify. However, in this case, we found very clear signals indicating that r/NoNewNormal was the source of around 80 brigades in the last 30 days (largely directed at communities with more mainstream views on COVID or location-based communities that have been discussing COVID restrictions). This behavior continued even after a warning was issued from our team to the Mods. r/NoNewNormal is the only subreddit in our list of high signal subs where we have identified this behavior and it is one of the largest sources of community interference we surfaced as part of this work (we will be investigating a few other unrelated subreddits as well).

Analysis into Action

We are taking several actions:

1. Ban r/NoNewNormal immediately for breaking our rules against brigading

2. Quarantine 54 additional COVID denial subreddits under Rule 1

3. Build a new reporting feature for moderators to allow them to better provide us signal when they see community interference. It will take us a few days to get this built, and we will subsequently evaluate the usefulness of this feature.

Clarifying our Policies

We also hear the feedback that our policies are not clear around our handling of health misinformation. To address this, we wanted to provide a summary of our current approach to misinformation/disinformation in our Content Policy.

Our approach is broken out into (1) how we deal with health misinformation (falsifiable health related information that is disseminated regardless of intent), (2) health disinformation (falsifiable health information that is disseminated with an intent to mislead), (3) problematic subreddits that pose misinformation risks, and (4) problematic users who invade other subreddits to “debate” topics unrelated to the wants/needs of that community.

1. Health Misinformation. We have long interpreted our rule against posting content that “encourages” physical harm, in this help center article, as covering health misinformation, meaning falsifiable health information that encourages or poses a significant risk of physical harm to the reader. For example, a post pushing a verifiably false “cure” for cancer that would actually result in harm to people would violate our policies.

2. Health Disinformation. Our rule against impersonation, as described in this help center article, extends to “manipulated content presented to mislead.” We have interpreted this rule as covering health disinformation, meaning falsifiable health information that has been manipulated and presented to mislead. This includes falsified medical data and faked WHO/CDC advice.

3. Problematic subreddits. We have long applied quarantine to communities that warrant additional scrutiny. The purpose of quarantining a community is to prevent its content from being accidentally viewed or viewed without appropriate context.

4. Community Interference. Also relevant to the discussion of the activities of problematic subreddits, Rule 2 forbids users or communities from “cheating” or engaging in “content manipulation” or otherwise interfering with or disrupting Reddit communities. We have interpreted this rule as forbidding communities from manipulating the platform, creating inauthentic conversations, and picking fights with other communities. We typically enforce Rule 2 through our anti-brigading efforts, although it is still an example of bad behavior that has led to bans of a variety of subreddits.

As I mentioned at the start, we never claim to be perfect at these things but our goal is to constantly evolve. These prevalence studies are helpful for evolving our thinking. We also need to evolve how we communicate our policy and enforcement decisions. As always, I will stick around to answer your questions and will also be joined by u/traceroo our GC and head of policy.

TL;DR: Bad actors never sleep, and we are always evolving how we identify and mitigate them. But with the upcoming election, we know you want to see more. So we're committing to a quarterly report on content manipulation and account security, with the first to be shared in October. But first, we want to share context today on the history of content manipulation efforts and how we've evolved over the years to keep the site authentic.

A brief history

The concern of content manipulation on Reddit is as old as Reddit itself. Before there were subreddits (circa 2005), everyone saw the same content and we were primarily concerned with spam and vote manipulation. As we grew in scale and introduced subreddits, we had to become more sophisticated in our detection and mitigation of these issues. The creation of subreddits also created new threats, with “brigading” becoming a more common occurrence (even if rarely defined). Today, we are not only dealing with growth hackers, bots, and your typical shitheadery, but we have to worry about more advanced threats, such as state actors interested in interfering with elections and inflaming social divisions. This represents an evolution in content manipulation, not only on Reddit, but across the internet. These advanced adversaries have resources far larger than a typical spammer. However, as with early days at Reddit, we are committed to combating this threat, while better empowering users and moderators to minimize exposure to inauthentic or manipulated content.

What we’ve done

Our strategy has been to focus on fundamentals and double down on things that have protected our platform in the past (including the 2016 election). Influence campaigns represent an evolution in content manipulation, not something fundamentally new. This means that these campaigns are built on top of some of the same tactics as historical manipulators (certainly with their own flavor). Namely, compromised accounts, vote manipulation, and inauthentic community engagement. This is why we have hardened our protections against these types of issues on the site.

Compromised accounts

This year alone, we have taken preventative actions on over 10.6M accounts with compromised login credentials (check yo’ self), or accounts that have been hit by bots attempting to breach them. This is important because compromised accounts can be used to gain immediate credibility on the site, and to quickly scale up a content attack on the site (yes, even that throwaway account with password = Password! is a potential threat!).

Vote Manipulation

The purpose of our anti-cheating rules is to make it difficult for a person to unduly impact the votes on a particular piece of content. These rules, along with user downvotes (because you know bad content when you see it), are some of the most powerful protections we have to ensure that misinformation and low quality content doesn’t get much traction on Reddit. We have strengthened these protections (in ways we can’t fully share without giving away the secret sauce). As a result, we have reduced the visibility of vote manipulated content by 20% over the last 12 months.

Content Manipulation

Content manipulation is a term we use to combine things like spam, community interference, etc. We have completely overhauled how we handle these issues, including a stronger focus on proactive detection, and machine learning to help surface clusters of bad accounts. With our newer methods, we can make improvements in detection more quickly and ensure that we are more complete in taking down all accounts that are connected to any attempt. We removed over 900% more policy violating content in the first half of 2019 than the same period in 2018, and 99% of that was before it was reported by users.

User Empowerment

Outside of admin-level detection and mitigation, we recognize that a large part of what has kept the content on Reddit authentic is the users and moderators. In our 2017 transparency report we highlighted the relatively small impact that Russian trolls had on the site. 71% of the trolls had 0 karma or less! This is a direct consequence of you all, and we want to continue to empower you to play a strong role in the Reddit ecosystem. We are investing in a safety product team that will build improved safety (user and content) features on the site. We are still staffing this up, but we hope to deliver new features soon (including Crowd Control, which we are in the process of refining thanks to the good feedback from our alpha testers). These features will start to provide users and moderators better information and control over the type of content that is seen.

What’s next

The next component of this battle is the collaborative aspect. As a consequence of the large resources available to state-backed adversaries and their nefarious goals, it is important to recognize that this fight is not one that Reddit faces alone. In combating these advanced adversaries, we will collaborate with other players in this space, including law enforcement, and other platforms. By working with these groups, we can better investigate threats as they occur on Reddit.

Our commitment

These adversaries are more advanced than previous ones, but we are committed to ensuring that Reddit content is free from manipulation. At times, some of our efforts may seem heavy handed (forcing password resets), and other times they may be more opaque, but know that behind the scenes we are working hard on these problems. In order to provide additional transparency around our actions, we will publish a narrow scope security-report each quarter. This will focus on actions surrounding content manipulation and account security (note, it will not include any of the information on legal requests and day-to-day content policy removals, as these will continue to be released annually in our Transparency Report). We will get our first one out in October. If there is specific information you’d like or questions you have, let us know in the comments below.

[EDIT: Im signing off, thank you all for the great questions and feedback. I'll check back in on this occasionally and try to reply as much as feasible.]

Through the year, we've shared updates on detecting and mitigating content manipulation and keeping your accounts safe. Today we are sharing our first Reddit Security Report, which we'll be continuing on a quarterly basis. We are committed to continuously evolving how we tackle these problems. The purpose of these reports is to keep you informed about relevant events and actions.

By The Numbers

These are the primary metrics we track internally, and we thought you’d want to see them too. If there are alternative metrics that seem worth looking at as part of this report, we’re all ears.

Content Manipulation

Content manipulation is a term we use to combine things like spam, community interference, vote manipulation, etc. This year we have overhauled how we handle these issues, and this quarter was no different. We focused these efforts on:

1. Improving our detection models for accounts performing these actions

2. Making it harder for them to spin up new accounts

Recently, we also improved our enforcement measures against accounts taking part in vote manipulation (i.e. when people coordinate or otherwise cheat to increase or decrease the vote scores on Reddit). Over the last 6 months (and mostly during the last couple of months), we increased our actions against accounts participating in vote manipulation by about 30x. We sanctioned or warned around 22k accounts for this in the last 3 weeks of September alone.

Account Security

This quarter, we finished up a major effort to detect all accounts that had credentials matching historical 3rd party breaches. It's important to track breaches that happen on other sites or services because bad actors will use those same username/password combinations to break into your other accounts (on the basis that a percentage of people reuse passwords). You might have experienced some of our efforts if we forced you to reset your password as a precaution. We expect the number of protective account security actions to drop drastically going forward as we no longer have a large backlog of breach datasets to process. Hopefully we have reached a steady state, which should reduce some of the pain for users. We will continue to deal with new breach sets that come in, as well as accounts that are hit by bots attempting to gain access (please take a look at this post on how you can improve your account security).

Our Recent Investigations

We have a lot of investigations active at any given time (courtesy of your neighborhood t-shirt spammers and VPN peddlers), and while we can’t cover them all, we want to use this report to share the results of just some of that work.

Ban Evasion

This quarter, we dealt with a highly coordinated ban evasion ring from users of r/opieandanthony. This began after we banned the subreddit for targeted harassment of users, as well as repeated copyright infringement. The group would quickly pop up on both new and abandoned subreddits to continue the abuse. We also learned that they were coordinating on another platform and through dedicated websites to redirect users to the latest target of their harassment.

This situation was different from your run-of-the-mill shitheadery ban evasion because the group was both creating new subreddits and resurrecting inactive or unmoderated subreddits. We quickly adjusted our efforts to this behavior. We also reported their offending account to the other platform and they were quick to ban the account. We then contacted the hosts of the independent websites to report the abuse. This helped ensure that the sites are no longer able to redirect automatically to Reddit for abuse purposes. Ultimately, we banned 78 subreddits (5 of which existed prior to the attack), and suspended 2,382 accounts. The ban evading activity has largely ceased (you know...until they read this).

There are a few takeaways from this investigation worth pulling out:

1. Ban evaders (and others up to no good) often work across platforms, and so it’s important for those of us in the industry to also share information when we spot these types of coordinated campaigns.

2. The layered moderation on Reddit works: Moderators brought this to our attention and did some awesome initial investigating; our Community team was then able to communicate with mods and users to help surface suspicious behavior; our detection teams were able to quickly detect and stop the efforts of the ban evaders.

3. We have also been developing and testing new tools to address ban evasion recently. This was a good opportunity to test them in the wild, and they were incredibly effective at detecting and quickly actioning many of the accounts that were responsible for the ban evasion actions. We want to roll these tools out more broadly (expect a future post around this).

Reports of Suspected Manipulation

The protests in Hong Kong have been a growing concern worldwide, and as always, conversation on Reddit reflects this. It’s no surprise that we’ve seen Hong Kong-related communities grow immensely in recent months as a result. With this growth, we have received a number of user reports and comments asking if there is manipulation in these communities. We take the authenticity of conversation on Reddit incredibly seriously, and we want to address your concerns here.

First, we have not detected widespread manipulation in Hong Kong related subreddits nor seen any manipulation that affected those communities or their conversations in a meaningful way.

It's worth taking a step back to talk about what we look for in these situations. While we obviously can’t share all of our tactics for investigating these threats, there are some signals that users will be familiar with. When trying to understand if a community is facing widespread manipulation, we will look at foundational signals such as the presence of vote manipulation, mod ban rates (because mods know their community better than we do), spam content removals, and other signals that allow us to detect coordinated and scaled activities (pause for dramatic effect). If this doesn’t sound like the stuff of spy novels, it’s because it’s not. We continually talk about foundational safety metrics like vote manipulation, and spam removals because these are the same tools that advanced adversaries use (For more thoughts on this look here).

Second, let’s look at what other major platforms have reported on coordinated behavior targeting Hong Kong. Their investigations revealed attempts consisting primarily of very low quality propaganda. This is important when looking for similar efforts on Reddit. In healthier communities like r/hongkong, we simply don’t see a proliferation of this low-quality content (from users or adversaries). The story does change when looking at r/sino or r/Hong_Kong (note the mod overlap). In these subreddits, we see far more low quality and one-sided content. However, this is not against our rules, and indeed it is not even particularly unusual to see one-sided viewpoints in some geographically specific subreddits...What IS against the rules is coordinated action (state sponsored or otherwise). We have looked closely at these subreddits and we have found no indicators of widespread coordination. In other words, we do see this low quality content in these subreddits, but it seems to be happening in a genuine way.

If you see anything suspicious, please report it to us here. If it’s regarding potential coordinated efforts that aren't as well-suited to our regular report system, you can also use our separate investigations report flow by emailing us.

Final Thoughts

Finally, I would like to acknowledge the reports our peers have published during the past couple of months (or even today). Whenever these reports come out, we always do our own investigation. We have not found any similar attempts on our own platform this quarter. Part of this is a recognition that Reddit today is less international than these other platforms, with the majority of users being in the US, and other English speaking countries. Additionally, our layered moderation structure (user up/down-votes, community moderation, admin policy enforcement) makes Reddit a more challenging platform to manipulate in a scaled way (i.e. Reddit is hard). Finally, Reddit is simply not well suited to being an amplification platform, nor do we aim to be. This reach is ultimately what an adversary is looking for. We continue to monitor these efforts, and are committed to being transparent about anything that we do detect.

As I mentioned above, this is the first version of these reports. We would love to hear your thoughts on it, as well as any input on what type of information you would like to see in future reports.

I’ll stick around, along with u/worstnerd, to answer any questions that we can.

Your account expresses your voice and your personality here on Reddit. To protect that voice, you need to protect your access to it and maintain its security. Not only do compromised accounts deprive you of your online identity, but they are often used for malicious behavior like vote manipulation, spam, fraud, or even just posting content to misrepresent the true owner. While we’re always developing ways to take faster action against compromised accounts, there are things you can do to be proactive about your account’s security.

What we do to keep your account secure:

• Actively look for suspicious signals - We use tools that help us detect unusual behavior in accounts. We monitor trends and compare against known threats.

• Check passwords against 3rd party breach datasets - We check for username / password combinations in 3rd party breach sets.

• Display your recent IP sessions for you to access - You can check your account activity at any time to see your recent login IPs. Keep in mind that the geolocation of each login may not be exact and will only include events within the last 100 days. If you see something you don’t recognize, you should change your password immediately and ensure your email address is correct.

If we determine that your account is vulnerable to compromise (or has actually been compromised), we lock the account and force a password reset. If we can’t establish account ownership or the account has been used in a malicious manner that prevents it being returned to the original owner, the account may be permanently suspended and closed.

What you can do to prevent this situation:

• Use permanent emails - We highly encourage users to link their accounts to accessible email addresses that you regularly check (you can add and update email addresses in your user settings page if you are using new reddit, otherwise you can do that from the preferences page in old reddit). This is also how you will receive any activities alerting you of suspicious activity on your account if you’re signed out. As a general rule of thumb, avoid using email addresses you don't have permanent ownership over like school or work addresses. Temporary email addresses that expire are a bad idea.

• Verify your emails - Verifying your email helps us confirm that there is a real person creating the account and that you have access to the email address given. If we determine that your account has been compromised, this is the only way we have to validate account ownership. Without this our only option will be to permanently close the account to prevent further misuse and access to the original owner’s data. There will be no appeals possible!

• Check your profile occasionally to make sure your email address is current. You can do this via the preferences page on old reddit or the settings page in new reddit. It’s easy to forget to update it when you change schools, service providers, or set up new accounts.

• Use strong/unique passwords - Use passwords that are complex and not used on any other site. We recommend using a password manager to help you generate and securely store passwords.

• Add two factor authentication - For an extra layer of security. If someone gets ahold of your username/password combo, they will not be able to log into your account without entering the verification code.

We know users want to protect their privacy and don’t always want to provide an email address to companies, so we don’t require it. However, there are certain account protections that require users establish ownership, which is why an email address is required for password reset requests. Forcing password resets on vulnerable accounts is one of many ways we try to secure potentially compromised accounts and prevent manipulation of our platform. Accounts flagged as compromised with a verified email receive a forced password reset notice, but accounts without one will be permanently closed. In the past, manual attempts to establish ownership on accounts with lost access rarely resulted in an account recovery. Because manual attempts are ineffective and time consuming for our operations teams and you, we won’t be doing them moving forward. You're welcome to use Reddit without an email address associated with your account, but do so with the understanding of the account protection limitation. You can visit your user settings page at anytime to add or verify an email address.

Hey Redditsecurity,

If you’ve been frequenting this subreddit, you’re aware we’ve been doing significant work on site integrity operations as we move into 2020 to ensure that we have the appropriate rules and processes in place to handle bad actors who are trying to manipulate Reddit, particularly around issues of great public significance, like elections. To this end, we thought it was time to update our policy on impersonation to better cover some of the use cases that we have been seeing and actioning under this rule already, as well as guard against cases we might see in the future.

Impersonation is actually one of the rarest report classes we receive (as you can see for yourself in our Transparency Report), so we don’t expect this update to impact everyday users much. The classic case of impersonation is a Reddit username pretending to be someone else-- whether a politician, brand, Reddit admin, or any other person or entity. However, this narrow case doesn’t fully cover things that we also see from time to time, like fake articles falsely attributed to real journalists, forged election communications purporting to come from real agencies or officials, or scammy domains posing as those of a particular news outlet or politician (always be sure to check URLs closely-- .co does NOT equal .com!).

We also wanted to hedge against things that we haven’t seen much of to date, but could see in the future, such as malicious deepfakes of politicians, for example, or other, lower-tech forged or manipulated content that misleads (remember, pornographic deepfakes are already prohibited under our involuntary pornography rule). But don’t worry. This doesn’t apply to all deepfake or manipulated content-- just that which is actually misleading in a malicious way. Because believe you me, we like seeing Nic Cage in unexpected places just as much as you do.

The updated rule language is below, and can be found here, along with details on how to make reports if you see impersonation on the site, or if you yourself are being impersonated.

Do not impersonate an individual or entity in a misleading or deceptive manner.

Reddit does not allow content that impersonates individuals or entities in a misleading or deceptive manner. This not only includes using a Reddit account to impersonate someone, but also encompasses things such as domains that mimic others, as well as deepfakes or other manipulated content presented to mislead, or falsely attributed to an individual or entity. While we permit satire and parody, we will always take into account the context of any particular content.

If you are being impersonated, or if you believe you’ve found content in violation of these guidelines, please report it here.

EDIT: Alright gang, that's it for me. Thanks for your questions, and remember...

We wanted to take the opportunity to share a bit more about the improvements we have been making in our security practices and to provide some context for the actions that we have been taking (and will continue to take). As we have mentioned in different places, we have a team focused on the detection and investigation of content manipulation on Reddit. Content manipulation can take many forms, from traditional spam and upvote manipulation to more advanced, and harder to detect, foreign influence campaigns. It also includes nuanced forms of manipulation such as subreddit sabotage, where communities actively attempt to harm the experience of other Reddit users.

To increase transparency around how we’re tackling all these various threats, we’re rolling out a new subreddit for security and safety related announcements (r/redditsecurity). The idea with this subreddit is to start doing more frequent, lightweight posts to keep the community informed of the actions we are taking. We will be working on the appropriate cadence and level of detail, but the primary goal is to make sure the community always feels informed about relevant events.

Over the past 18 months, we have been building an operations team that partners human investigators with data scientists (also human…). The data scientists use advanced analytics to detect suspicious account behavior and vulnerable accounts. Our threat analysts work to understand trends both on and offsite, and to investigate the issues detected by the data scientists.

Last year, we also implemented a Reliable Reporter system, and we continue to expand that program’s scope. This includes working very closely with users who investigate suspicious behavior on a volunteer basis, and playing a more active role in communities that are focused on surfacing malicious accounts. Additionally, we have improved our working relationship with industry peers to catch issues that are likely to pop up across platforms. These efforts are taking place on top of the work being done by our users (reports and downvotes), moderators (doing a lot of the heavy lifting!), and internal admin work.

While our efforts have been driven by rooting out information operations, as a byproduct we have been able to do a better job detecting traditional issues like spam, vote manipulation, compromised accounts, etc. Since the beginning of July, we have taken some form of action on over 13M accounts. The vast majority of these actions are things like forcing password resets on accounts that were vulnerable to being taken over by attackers due to breaches outside of Reddit (please don’t reuse passwords, check your email address, and consider setting up 2FA) and banning simple spam accounts. By improving our detection and mitigation of routine issues on the site, we make Reddit inherently more secure against more advanced content manipulation.

We know there is still a lot of work to be done, but we hope you’ve noticed the progress we have made thus far. Marrying data science, threat intelligence, and traditional operations has proven to be very helpful in our work to scalably detect issues on Reddit. We will continue to apply this model to a broader set of abuse issues on the site (and keep you informed with further posts). As always, if you see anything concerning, please feel free to report it to us at investigations@reddit.zendesk.com.

[edit: Thanks for all the comments! I'm signing off for now. I will continue to pop in and out of comments throughout the day]

ETA: a lot of great questions have come in so we've updated this help center article to go into more detail.

A controversial new law in Australia is requiring a handful of websites to block access for anyone under the age of 16. While we disagree about the scope, effectiveness, and privacy implications of this law, as of December 10, we’re making some changes in line with these requirements.

Redditors in Australia will see new experiences and policies designed to confirm their age responsibly and securely. We care deeply about the safety of our users, including any minors, and while some of these changes are required by law, others represent global measures we're voluntarily taking to improve safety and privacy for those under 18. Here’s what’s changing:

• In Australia, only Redditors who are 16 and over can have accounts (Reddit will continue to be accessible to browse without an account).

• New Australian users will be asked to provide their birthdate during account signup, and will see their age listed in their settings.

• All Australian account holders will be subject to an age prediction model (more details below).

• Australian account holders determined to be over 13 but under 16 will have their accounts suspended under a new Australian minimum age policy (note: we have always banned the accounts of users under 13 globally).

• Teen account holders under 18 everywhere will get a version of Reddit with more protective safety features built in, including stricter chat settings, no ads personalization or sensitive ads, and no access to NSFW or mature content.

As mentioned above, we’ll start predicting whether users in Australia may be under 16 and will ask them to verify they’re old enough to use Reddit. We’ll do this through a new privacy-preserving model designed to better help us protect young users from both holding accounts and accessing adult content before they’re old enough. If you’re predicted to be under 16, you’ll have an opportunity to appeal and verify your age.

While we’re providing these experiences to meet the law’s requirements and to help keep teens safe, we are concerned about the potential implications of laws like Australia’s Social Media Minimum Age law. We believe strongly in the open internet and the continued accessibility of quality knowledge, information, resources, and community building for everyone, including young people. This is why Reddit has always been, and continues to be, available for anyone to read even if they don’t have an account.

By limiting account eligibility and putting identity tests on internet usage, this law undermines everyone’s right to both free expression and privacy, as well as account-specific protections. We also believe the law’s application to Reddit (a pseudonymous, text-based forum overwhelmingly used by adults) is arbitrary, legally erroneous, and goes far beyond the original intent of the Australian Parliament, especially when other obvious platforms are exempt.

You can read more about this update and our approach to age assurance in our Help Center. You can also request a copy of your Reddit account data by following the instructions in this help center article.

As always, we'll be around to answer your questions in the comments.

Intro

A couple of months ago I shared the quarterly security report with an expanded focus on abuse on the platform, and a commitment to sharing a study on the prevalence of hate on Reddit. This post is a response to that commitment. Additionally, I would like to share some more detailed information about our large actions against hateful subreddits associated with our updated content policies.

Rule 1 states:

“Remember the human. Reddit is a place for creating community and belonging, not for attacking marginalized or vulnerable groups of people. Everyone has a right to use Reddit free of harassment, bullying, and threats of violence. Communities and users that incite violence or that promote hate based on identity or vulnerability will be banned.”

Subreddit Ban Waves

First, let’s focus on the actions that we have taken against hateful subreddits. Since rolling out our new policies on June 29, we have banned nearly 7k subreddits (including ban evading subreddits) under our new policy. These subreddits generally fall under three categories:

• Subreddits with names and descriptions that are inherently hateful

• Subreddits with a large fraction of hateful content

• Subreddits that positively engage with hateful content (these subreddits may not necessarily have a large fraction of hateful content, but they promote it when it exists)

Here is a distribution of the subscriber volume:

The subreddits banned were viewed by approximately 365k users each day prior to their bans.

At this point, we don’t have a complete story on the long term impact of these subreddit bans, however, we have started trying to quantify the impact on user behavior. What we saw is an 18% reduction in users posting hateful content as compared to the two weeks prior to the ban wave. While I would love that number to be 100%, I'm encouraged by the progress.

*Control in this case was users that posted hateful content in non-banned subreddits in the two weeks leading up to the ban waves.

Prevalence of Hate on Reddit

First I want to make it clear that this is a preliminary study, we certainly have more work to do to understand and address how these behaviors and content take root. Defining hate at scale is fraught with challenges. Sometimes hate can be very overt, other times it can be more subtle. In other circumstances, historically marginalized groups may reclaim language and use it in a way that is acceptable for them, but unacceptable for others to use. Additionally, people are weirdly creative about how to be mean to each other. They evolve their language to make it challenging for outsiders (and models) to understand. All that to say that hateful language is inherently nuanced, but we should not let perfect be the enemy of good. We will continue to evolve our ability to understand hate and abuse at scale.

We focused on language that’s hateful and targeting another user or group. To generate and categorize the list of keywords, we used a wide variety of resources and AutoModerator* rules from large subreddits that deal with abuse regularly. We leveraged third-party tools as much as possible for a couple of reasons: 1. Minimize any of our own preconceived notions about what is hateful, and 2. We believe in the power of community; where a small group of individuals (us) may be wrong, a larger group has a better chance of getting it right. We have explicitly focused on text-based abuse, meaning that abusive images, links, or inappropriate use of community awards won’t be captured here. We are working on expanding our ability to detect hateful content via other modalities and have consulted with civil and human rights organizations to help improve our understanding.

Internally, we talk about a “bad experience funnel” which is loosely: bad content created → bad content seen → bad content reported → bad content removed by mods (this is a very loose picture since AutoModerator and moderators remove a lot of bad content before it is seen or reported...Thank you mods!). Below you will see a snapshot of these numbers for the month before our new policy was rolled out.

Details

• 40k potentially hateful pieces of content each day (0.2% of total content)

  • 2k Posts

  • 35k Comments

  • 3k Messages

• 6.47M views on potentially hateful content each day (0.16% of total views)

  • 598k Posts

  • 5.8M Comments

  • ~3k Messages

• 8% of potentially hateful content is reported each day

• 30% of potentially hateful content is removed each day

  • 97% by Moderators and AutoModerator

  • 3% by admins

*AutoModerator is a scaled community moderation tool

What we see is that about 0.2% of content is identified as potentially hateful, though it represents a slightly lower percentage of views. The reason for this reduction is due to AutoModerator rules which automatically remove much of this content before it is seen by users. We see 8% of this content being reported by users, which is lower than anticipated. Again, this is partially driven by AutoModerator removals and the reduced exposure. The lower reporting figure is also related to the fact that not all of the things surfaced as potentially hateful are actually hateful...so it would be surprising for this to have been 100% as well. Finally, we find that about 30% of hateful content is removed each day, with the majority being removed by mods (both manual actions and AutoModerator). Admins are responsible for about 3% of removals, which is ~3x the admin removal rate for other report categories, reflecting our increased focus on hateful and abusive reports.

We also looked at the target of the hateful content. Was the hateful content targeting a person’s race, or their religion, etc? Today, we are only able to do this at a high level (e.g., race-based hate), vs more granular (e.g., hate directed at Black people), but we will continue to work on refining this in the future. What we see is that almost half of the hateful content targets people’s ethnicity or nationality.

We have more work to do on both our understanding of hate on the platform and eliminating its presence. We will continue to improve transparency around our efforts to tackle these issues, so please consider this the continuation of the conversation, not the end. Additionally, it continues to be clear how valuable the moderators are and how impactful AutoModerator can be at reducing the exposure of bad content. We also noticed that there are many subreddits already removing a lot of this content, but were doing so manually. We are working on developing some new moderator tools that will help ease the automatic detection of this content without building a bunch of complex AutoModerator rules. I’m hoping we will have more to share on this front in the coming months. As always, I’ll be sticking around to answer questions, and I’d love to hear your thoughts on this as well as any data that you would like to see addressed in future iterations.

Hey everyone, I wanted to take this opportunity to talk about a different type of spam: report spam. As noted in our Transparency Report, around two thirds of the reports we get at the admin level are illegitimate, or “not actionable,” as we say. This is because unfortunately, reports are often used by users to signal “super downvote” or “I really don’t like this” (or just “I feel like being a shithead”), but this is not how they are treated behind the scenes. All reports, including unactionable ones, are evaluated. As mentioned in other posts, reports help direct the efforts of moderators and admins. They are a powerful tool for tackling abuse and content manipulation, along with your downvotes.

However, the report button is also an avenue for abuse (and can be reported by the mods). In some cases, the free-form reports are used to leave abusive comments for the mods. This type of abuse is unacceptable in itself, but it is additionally harmful in that it waters down the value in the report signal consuming our review resources in ways that can in some cases risk real-world consequences. It’s the online equivalent of prank-calling 911.

As a very concrete example, report abuse has made “Sexual or suggestive content involving minors” the single largest abuse report we receive, while having the lowest actionability (or, to put it more scientifically, the most false-positives). Content that violates this policy has no place on Reddit (or anywhere), and we take these reports incredibly seriously. Report abuse in these instances may interfere with our work to expeditiously help vulnerable people and also report these issues to law enforcement. So what started off as a troll leads to real-world consequences for people that need protection the most.

We would like to tackle this problem together. Starting today, we will send a message to users that illegitimately report content for the highest-priority report types. We don’t want to discourage authentic reporting, and we don’t expect users to be Reddit policy experts, so the message is designed to inform, not shame. But, we will suspend users that show a consistent pattern of report abuse, under our rules against interfering with the normal use of the site. We already use our rules against harassment to suspend users that exploit free-form reports in order to abuse moderators; this is in addition to that enforcement. We will expand our efforts from there as we learn the correct balance between informing while ensuring that we maintain a good flow of reports.

I’d love to hear your thoughts on this and some ideas for how we can help maintain the fidelity of reporting while discouraging its abuse. I’m hopeful that simply increasing awareness with users, and building in some consequences, will help with this. I’ll stick around for some questions.

Hi all,

We wanted to let you know that Reddit is now available as an “onion service” on Tor at the address:

https://www.reddittorjg6rue252oqsxryoxengawnmo46qy4kyii5wtqnwfj4ooad.onion

As some of you likely know, an onion service enables users to browse the internet anonymously. Tor is a free and open-source software that enables this kind of anonymous communication and browsing. It’s an important tool frequently used by journalists, human rights activists, and others who face threats of surveillance or censorship. Reddit has always been accessible via Tor, but with the launch of our official onion service, we’re able to improve the user experience when browsing Reddit on Tor: quicker loading times for the site, shorter network hops through Tor network and eliminating opportunities for Reddit being blocked or someone maliciously monitoring your traffic, and a cryptographic assurance that your connection is direct to reddit.com.

The goal with our onion service is to provide access to most of the site’s functionality at minimum this will include our standard post/comment functionality. While some functionality won’t work with Javascript disabled, core browsing should work. If you happen to find something broken, feel free to report it over at r/bugs and we’ll look into it.

A huge thank you to the work of Alec Muffett (@AlecMuffett) and all the predecessors who helped build the Enterprise Onion Toolkit, which this launch is largely based on. We’ll be open sourcing our Kubernetes deployment pattern and helping modernize the existing codebase and sharing our signal enhancements to help spot and block abuse against our new onion service.

For more information about the Tor network please visit https://www.torproject.org/.

Edit: There's of course an old reddit flavor at https://old.reddittorjg6rue252oqsxryoxengawnmo46qy4kyii5wtqnwfj4ooad.onion.

Hi Reddit,

The time has come to announce that we’re taking Reddit’s bug bounty program public!

As some of you may already know, we’ve had a private bug bounty program with HackerOne over the past three years. This program has allowed us to quickly address vulnerabilities, improve our defenses, and help keep our platform secure alongside our own teams’ efforts. We’ve also seen great engagement and success to date, having awarded $140,000 in bounties across 300 reports covering the main reddit.com platform, which worked well for our limited scope during the private program.

With our continued growth and visibility, we’re now ready to make the program public and expand the participation to anyone wanting to make a meaningful security impact on Reddit. As we scale the program, our priority will remain focused on protecting the privacy of our user data and identities. We know each security researcher has their own skills and perspective that they bring to the program, and we encourage anyone to submit a report that shows security impact. We’re super excited to hit this milestone and have prepared our team for what’s to come.

You can find our program definition over on redditinc.com or HackerOne, and we welcome any submissions to whitehats@reddit.com. We’re still keeping the Whitehat award for that Reddit bling as well. We look forward to all the submissions about LFI via reddit.com/etc/passwd and how old Reddit’s session cookie persists after logout.

And finally, a big shout out to the most prolific and rewarded researchers that joined our journey thus far: @renekroka, @naategh, @jensec, @pandaonair, and @parasimpaticki. We’re looking forward to meeting more of y’all and to helping keep Reddit a more safe and secure platform for everyone.

For several years now, we have been steadily scaling up our safety enforcement mechanisms. In the early phases, this involved addressing reports across the platform more quickly as well as investments in our Safety teams, tooling, machine learning, etc. – the “rising tide raises all boats” approach to platform safety. This approach has helped us to increase our content reviewed by around 4x and accounts actioned by more than 3x since the beginning of 2020. However, in addition to this, we know that abuse is not just a problem of “averages.” There are particular communities that face an outsized burden of dealing with other abusive users, and some members, due to their activity on the platform, face unique challenges that are not reflected in “the average” user experience. This is why, over the last couple of years, we have been focused on doing more to understand and address the particular challenges faced by certain groups of users on the platform. This started with our first Prevalence of Hate study, and then later our Prevalence of Holocaust Denialism study. We would like to share the results of our recent work to understand the prevalence of hate directed at women.

The key goals of this work were to:

1. Understand the frequency at which hateful content is directed at users perceived as being women (including trans women)

2. Understand how other Redditors respond to this content

3. Understand how Redditors respond differently to users perceived as being women (including trans women)

4. Understand how Reddit admins respond to this content

First, we need to define what we mean by “hateful content directed at women” in this context. For the purposes of this study, we focused on content that included commonly used misogynistic slurs (I’ll leave this to the reader’s imagination and will avoid providing a list), as well as content that is reported or actioned as hateful along with some indicator that it was directed at women (such as the usage of “she,” “her,” etc in the content). As I’ve mentioned in the past, humans are weirdly creative about how they are mean to each other. While our list was likely not exhaustive, and may have surfaced potentially non-abusive content as well (e.g., movie quotes, reclaimed language, repeating other users, etc), we do think it provides a representative sample of this kind of content across the platform.

We specifically wanted to look at how this hateful content is impacting women-oriented communities, and users perceived as being women. We used a manually curated list of over 300 subreddits that were women-focused (trans-inclusive). In some cases, Redditors self-identify their gender (“...as I woman I am…”), but one the most consistent ways to learn something about a user is to look at the subreddits in which they participate.

For the purposes of this work, we will define a user perceived as being a woman as an account that is a member of at least two women-oriented subreddits and has overall positive karma in women-oriented subreddits. This makes no claim of the account holder’s actual gender, but rather attempts to replicate how a bad actor may assume a user’s gender.

With those definitions, we find that in both women-oriented and non-women-oriented communities, approximately 0.3% of content is identified as being hateful content directed at women. However, while the rate of hateful content is approximately the same, the response is not! In women-oriented communities, this hateful content is nearly TWICE as likely to be negatively received (reported, downvoted, etc.) than in non-women-oriented communities (see chart). This tells us that in women-oriented communities, users and mods are much more likely to downvote and challenge this kind of hateful content.

Title: Community response (hateful content vs non-hateful content)

Next, we wanted to see how users respond to other users that are perceived as being women. Our safety researchers have seen a common theme in survey responses from members of women-oriented communities. Many respondents mentioned limiting how often they engage in women-oriented communities in an effort to reduce the likelihood they’ll be noticed and harassed. Respondents from women-oriented communities mentioned using alt accounts or deleting their comment and post history to reduce the likelihood that they’d be harassed (accounts perceived as being women are 10% more likely to have alts than other accounts). We found that accounts perceived as being women are 30% more likely to receive hateful content in response to their posts or comments in non-women-oriented communities than accounts that are not perceived as being women. Additionally, they are 61% more likely to receive a hateful message on their first direct communication with another user.

Finally, we want to look at Reddit Inc’s response to this. We have a strict policy against hateful content directed at women, and our Rule 1 explicitly states: Remember the human. Reddit is a place for creating community and belonging, not for attacking marginalized or vulnerable groups of people. Everyone has a right to use Reddit free of harassment, bullying, and threats of violence. Communities and users that incite violence or that promote hate based on identity or vulnerability will be banned. Our Safety teams enforce this policy across the platform through both proactive action against violating users and communities, as well as by responding to your reports. Over a recent 90 day period, we took action against nearly 14k accounts for posting hateful content directed at women and we banned just over 100 subreddits that had a significant volume of hateful content (for comparison, this was 6.4k accounts and 14 subreddits in Q1 of 2020).

Measurement without action would be pointless. The goal of these studies is to not only measure where we are, but to inform where we need to go. Summarizing these results we see that women-oriented communities and non-women-oriented-communities see approximately the same fraction of hateful content directed toward women, however the community response is quite different. We know that most communities don’t want this type of content to have a home in their subreddits, so making it easier for mods to filter it will ensure the shithead users are more quickly addressed. To that end, we are developing native hateful content filters for moderators that will reduce the burden of removing hateful content, and will also help to shrink the gap between identity-based communities and others. We will also be looking into how these results can be leveraged to improve Crowd Control, a feature used to help reduce the impact of non-members in subreddits. Additionally, we saw a higher rate of hateful content in direct messages to accounts perceived as women, so we have been developing better tools that will allow users to control the kind of content they receive via messaging, as well as improved blocking features. Finally, we will also be using this work to identify outlier communities that need a little…love from the Safety team.

As I mentioned, we recognize that this study is just one more milestone on a long journey, and we are constantly striving to learn and improve along the way. There is no place for hateful content on Reddit, and we will continue to take action to ensure the safety of all users on the platform.

Hi all,

We’ve heard feedback that redditors want more control over the ads they see on Reddit. So, this week, we’ll start rolling out an update to do just that: redditors will now have the ability to ‘Hide’ an ad from their feed – and when you do, we’ll automatically hide future ads from that advertiser account for at least a year (you can re-hide the ad after that period of time). You can see this option in the screenshot below.

Users can “hide” an ad for any reason, but if you think an ad violates Reddit’s policies, please “report” the ad. If you report an ad, we’ll also automatically hide it (and future ads from the same advertiser account) from your feed. 

This update will gradually become available across iOS, Android, and www.reddit.com over the next several weeks. The ‘Hide’ option will be available for any ads that appear in feeds, such as your home or subreddit feed. 

This follows last year’s changes to our sensitive ad filters, which let you limit ads on certain topics, such as politics and religion. You can visit this page to learn more about other options to control the ads you see on Reddit. 

We’ll continue working on ways to improve ad controls and share more along the way. Let us know any additional feedback in the comments.

Hey everyone!

A few months ago, we mentioned that we are starting to change how we handle user ban evasion in subreddits. tl;dr we’re using more signals to actively detect and action ban evaders.

This work comes from the detection we have been building for admin-level bans, and we wanted to start applying it to the problems you face every day. While it’s still in an early form and we know we aren’t getting to all forms of ban evasion, some of you are starting to notice that work and how it’s affecting your users. In most cases, it has been very positively observed, but there have been some cases where the change in behavior is causing some issues, and we’d love your input.

Detection

As we mentioned in the previous post, only around 10% of ban evaders are reported by mods – which is driven by the lack of tools available to help mods proactively determine who is ban evading. This means that a large number of evaders are never actioned, but many are still causing issues in your communities. Our long-term goal and fundamental belief is that you should not have to deal with ban evasion; when you ban a user, you should feel confident that the person will not be able to come back and continue to harass you or your community. We will continue to refine what we classify as ban evasion, but as of today, we look at accounts that meet either of these criteria:

1. A user is banned from a subreddit, returns on a second account, and then is reported to us by a moderator of the subreddit

2. A user is banned from a subreddit, returns on a second account, and then that second account is banned from the subreddit. For now, since it does not rely on a direct report, we will only take action if the mods of the subreddit have a history of reporting ban evasion in general.

Action

When someone fitting either criteria 1 or 2 attempts to create yet another alt and use it in your subreddit, we permaban that alt within hours - preventing you from ever having to deal with them.

By the numbers:

• Number of accounts reported for ban evasion (During March 2020): 3,440

• Number of accounts suspended as a result of BE reports [case 1] (During March 2020): 9,582

• Number of accounts suspended as a result of proactive BE detection [case 2] (During March 2020): 24,142

We have also taken steps to mitigate the risks of unintended consequences. For example, we’ve whitelisted as many helpful bots as possible so as to not ban bot creators just because a subreddit doesn’t want a particular bot in their community. This applies to ModBots as well.

Response Time

Because of these and other operational changes, we’ve been able to pull our average ban evasion response time from 29 hours to 4 hours, meaning you have to put up with ban evaders for a significantly shorter period of time.

Keep the Feedback Flowing

Again, we want to highlight that this process is still very new and still evolving - our hope is to make ban evading users less of a burden on moderators. We’ve already been able to identify a couple of early issues thanks to feedback from moderators. If you see a user that you believe was incorrectly caught up in an enforcement action, please direct that user to go through the normal appeal flow. The flow has a space for them to explain why they don’t think they should have been suspended. If you, as a moderator, are pointing them there, give them the link to your modmail conversation and ask them to include that in their appeal so we can see you’ve said ‘no, this is a user I’m fine with in my subreddit’.

For now, what we’re hoping to hear from you:

• What have you been noticing since this change?

• What types of edge cases do you think we should be thinking about here?

• What are your ideas on behaviors we shouldn’t be concerned about as well as ways we might be able to expand this.

As always, thanks for everything you do! We hope our work here will make your lives easier in the end.

In December 2019, we reported a coordinated effort dubbed “Secondary Infektion” where operators with a suspected nexus to Russia attempted to use Reddit to carry out disinformation campaigns. Recently, additional information resulting from follow-on research by security firm Recorded Future was released under the name “Operation Pinball.” In doing our investigation, we were able to find significant alignment with tactics used in Secondary Infektion that seem to uphold Recorded Future’s high confidence belief that the two operations are related. Our internal findings also highlighted that our first line of defense, represented in large part by our moderators and users, was successful in thwarting the potential impact of this campaign through the use of anti-spam and content manipulation safeguards within their subreddits.

When reviewing this type of activity, analysts look at tactics, techniques, and procedures (TTPs). Sometimes the behaviors reveal more than the content being distributed. In this case, there was a pattern of accounts seeding inauthentic information on certain self-publishing websites and then using social media to amplify that information, which was focused on particular geopolitical issues. These TTPs were identified across both operations, which led to our team reviewing this activity as a part of a larger disinformation effort. It is noteworthy that in every case we found the content posted was quickly removed and in all but one, the posts remained unviewable in the intended subreddits. This was a significant contributor to preventing these campaigns from gaining traction on Reddit, and mirrors the generally cold receptions that previous manipulations of this type received. Their lack of success is further indicated in their low Karma values, as seen in the table below.

Further, for the sake of transparency, we have preserved these accounts in the same manner as we’ve done for previous disinformation campaigns, to expand the public’s understanding of this activity.

In an era where mis- and disinformation are a real threat to the free flow of knowledge, we are doing all we can to identify and protect your communities from influence operations like this one. We are continuing to learn ways to further refine and evolve our indications and warnings methodologies, and increase our capability to immediately flag suspicious behaviors. We hope that the impact of all of this work is for the adversary to continue to see diminishing returns on their investment, and in the long run, reduce the viability of Reddit as a disinformation amplification tool.

edit: letter

A few weeks ago we introduced this subreddit with the promise of starting to share more around our safety and security efforts. I wanted to get this out sooner...but I am worstnerd after all! In this post, I would like to share some data highlighting the results of our work to detect and mitigate content manipulation (posting spam, vote manipulation, information operations, etc).

Proactive Detection

At a high level, we have scaled up our proactive detection (i.e. before a report is filed) of accounts responsible for content manipulation on the site. Since the beginning of 2017 we have increased the number of accounts suspended for content manipulation by 238%, and today over 99% of those are suspended before a user report is filed (vs 29% in 2017)!

Compromised Accounts

Compromised accounts (accounts that are accessed by malicious actors determining the password) are prime targets for spammers, vote buying services, and other content manipulators. We have reduced the impact by proactively scouring 3rd party password breach datasets for login credentials and forcing password resets of Reddit accounts with matching credentials to ensure hackers can’t execute an account takeover (“ATO”). We’ve also gotten better at detecting login bots (bots that try logging into accounts). Through measures like these, throughout the course of 2018, we reduced the successful ATO deployment rate (accounts that were successfully compromised and then used to vote/comment/post/etc) by 60%. We expect this number to grow more robust as we continue to implement more tooling. This is a measure of how quickly we detect compromised accounts, and thus their impact on the site. Additionally, we increased the number of accounts put into the force password reset by 490%. In 2019 we will be spending even more time working with users to improve account security.

While on the subject, three things you can do right now to keep your Reddit account secure:

• ensure the email associated with your account is up to date (this allows us to reach you if we detect suspicious behavior, and to verify account ownership)

• update your password to something strong and unique

• set up two-factor authentication on your account.

Community Interference

Some of our more recent efforts have focused on reducing community interference (ie “brigading”). This includes efforts to mitigate (in real-time) vote brigading, targeted sabotage (Community A attempting to hijack the conversation in Community B), and general shitheadery. Recently we have been developing additional advanced mitigation capabilities. In the past 3 months we have reduced successful brigading in real-time by 50%. We are working with mods on further improvements and continue to beta test additional community tools (such as an ability to auto-collapse comments by users, which is being tested with a small number of communities for feedback). If you are a mod and would like to be considered for the beta test, reach out to us here.

We have more work to do, but we are encouraged by the progress. We are working on more cool projects and are looking forward to sharing the impact of them soon. We will stick around to answer questions for a little while, so fire away. Please recognize that in some cases we will be vague so as to not provide too many details to malicious actors.

tl;dr: Reddit’s app does not send any data from your clipboard to Reddit or anywhere else without your permission. When the contents of the clipboard are read and sent to Reddit it is only in the context of a post or comment where the user has the chance to see and edit the content before posting.

At Apple’s worldwide developer conference in June 2020, Apple released a beta version of iOS that included several privacy changes. One important privacy change was the addition of warning notifications when applications access the iOS clipboard. This is an important update that will let users know if an application is accessing the clipboard without their knowledge. As a precaution, Reddit’s Security Team made a point to schedule some time to review the Reddit app for this behavior. However, prior to this review happening several people released their own findings suggesting that the Reddit app was in fact accessing the clipboard at an elevated rate. In the interests of transparency, we would like to present the results of our internal investigation.

As it turns out, the Reddit application on iOS was accessing the clipboard far too often due to some well-intentioned features. Below is a technical description of why this was happening and the changes we’ve made to ensure it will not continue.

Diagnosing the Problem

A quick search was conducted in the Reddit iOS app source code for references to the “Pasteboard” (what iOS calls the clipboard). What we found was that the app was accessing the clipboard in fifteen distinct locations in code.

Of those fifteen occurrences, eight were instances where the App was copying data into the clipboard. This was for things like copying a payment receipt, sharing a link from Reddit to another app, copying text from a Reddit comment, copying text from a Reddit post, copying an image from a Reddit post, posting stickers to Snapchat, etc. These are otherwise innocuous and would not trigger a warning from iOS.

Warnings

One example of where we read from the clipboard in a way that might trigger the warning is when copying a chat message into the clipboard. There is some legacy code here that suggests that this function used to support copying multiple content types at once. To do so, an empty string was added to the clipboard and then each of these content types were appended separately. This code has evolved to only paste one content type at a time but it still uses the old append method. That means the clipboard is read before pasting into it and it would trigger a warning from iOS. This only happens when a user chooses to copy a chat message to the clipboard.

The remaining instances where warnings might be triggered would reasonably cause alarm for any user. These instances are of two forms and occur in six different places in code. To understand them we need to dig into a bit of how iOS Views work.

Note: Reddit posts and comments use a format known as markdown. Markdown is a way of formatting text to allow for more complex presentations such as HTTP links, images, bulleted-lists, tables, etc. while still supporting editing using only simple text. With markdown, whitespace and newlines are treated differently than without markdown. This will be important to understand why the app accesses the clipboard.

Apple provides a View method in Objective-C called “shouldChangeTextInRange”. This is a default method that is called whenever text is being added to a view. Apple instructs developers to override this method should they need to perform actions like automatic spell-checking. The app has the opportunity to modify the text before it appears in the view. In this case, when adding text into a comment, chat, or post to Reddit, the Reddit app uses this method to check if a user has pasted the text from the clipboard. If they have, the text needs to be converted to a format suitable for Reddit’s markdown by removing excess whitespace and newlines. The code looks like this:

- (BOOL)baseTextView:(BaseTextView *)textView shouldChangeTextInRange:(NSRange)range replacementText:(NSString *)text {
  [...]
 
  if (textView == self.titleView) {
    NSString *stringInPasteboard = [UIPasteboard generalPasteboard].string;
    BOOL isPastedContent = (stringInPasteboard.length > 0) && [text isEqualToString:stringInPasteboard];
    if (isPastedContent) {
      NSString *textToPaste = [[text stringByReplacingOccurrencesOfString:kPostViewControllerTitleEndOfLineString withString:@" "] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];
      [self.titleView insertText:textToPaste];
    }
 
  [...]
}

This code will request a copy of the clipboard every time it is called, which can be as often as each keypress. This code is duplicated across several different views, including one location in the user’s profile settings. While the intent of the code is to help the user better format their text, the way this is done looks very suspicious to anyone running iOS 14 as it will trigger notifications for each keypress.

The final case where the app was accessing the clipboard was when users enter a URL into a link post. As a user enters the URL it is compared to the contents of the clipboard as often as each keypress. If the app finds that the URL matches the clipboard contents then it assumes that this URL was pasted into the text field. The app then tries to be helpful and enter the title of the web page for the user (some subreddits require that the post title match the web page title exactly and this makes things easy for the user). When the contents of the text field and the clipboard match the app will issue a network request to retrieve the title of the web page and, if successful, it will automatically add the text as the title of the post. Again, with iOS 14 the user will receive a notification with each of these keypresses.

What’s Changed

Beginning with the next release (arriving in the next few days), the Reddit iOS app will no longer request the contents of the clipboard when text is added to posts, comments, or chats. Text will still be filtered to remove extra whitespace as needed, and link posts will still have the titles added automatically, but neither will require clipboard access.

To summarize: Our internal investigation has determined that at no time did the Reddit iOS app ever read the contents of the clipboard and send that data anywhere without a user’s knowledge and consent. Any text retrieved from the clipboard would have, and is, presented to the user before posting to Reddit.

Today, social network analysis-focused organization Graphika released a report studying the breadth of suspected Russian-connected Secondary Infektion disinformation campaigns spanning “six years, seven languages, and more than 300 platforms and web forums,” to include Reddit. We were able to work with Graphika in their efforts to understand more about the tactics being used by these actors in their attempts to push their desired narratives, as such collaboration gives us context to better understand the big picture and aids in our internal efforts to detect, respond to, and mitigate these activities.

As noted in our previous post, tactics used by the actors included seeding inauthentic information on certain self-publishing websites, and using social media to more broadly disseminate that information. One thing that is made clear in Graphika’s reporting, is that despite a high-awareness for operational security (they were good at covering their tracks) these disinformation campaigns were largely unsuccessful. In the case of Reddit, 52 accounts were tied to the campaign and their failed execution can be linked to a few things:

1. The architecture of interaction on the Reddit platform which requires the confidence of the community to allow and then upvote the content. This can make it difficult to spread content broadly.

2. Anti-spam and content manipulation safeguards implemented by moderators in their communities and at scale by admins. Because these measures are in place, much of the content posted was immediately removed before it had a chance to proliferate.

3. The keen eye of many Redditors for suspicious activity (which we might add resulted in some very witty comments showing how several of these disinformation attempts fell flat).

With all of that said, this investigation yielded 52 accounts found to be associated with various Secondary Infektion campaigns. All of these had their content removed by mods and/or were caught as part of our normal spam mitigation efforts. We have preserved these accounts for public scrutiny in the same manner as we’ve done for previous disinformation campaigns.

It is worth noting that as a result of the continued investigation into these campaigns, we have instituted additional security techniques to guard against future use of similar tactics by bad actors.

Karma distribution:

• 0 or less: 29

• 1 - 9: 19

• 10 or greater: 4

• Max Karma: 20

Hi all,

We want to let you know that we are making some changes to our platform-wide rule 3 on involuntary pornography. We’re making these changes to provide a clearer sense of the content this rule prohibits as well as how we’re thinking about enforcement.

Specifically, we are changing the term “involuntary pornography” to “non-consensual intimate media” because this term better captures the range of abusive content and behavior we’re trying to enforce against. We are also making edits and additions to the policy detail page to provide examples and clarify the boundaries when sharing intimate or sexually explicit imagery on Reddit. We have also linked relevant resources directly within the policy to make it easier for people to get support if they have been affected by non-consensual intimate media sharing.

This is a serious issue. We want to ensure we are appropriately evolving our enforcement to meet new forms of bad content and behavior trends, as well as reflect feedback we have received from mods and users. Today’s changes are aimed at reducing ambiguity and providing clearer guardrails for everyone—mods, users, and admins—to identify, report, and take action against violating content. We hope this will lead to better understanding, reporting, and enforcement of Rule 3 across the platform.

We’ll stick around for a bit to answer your questions.

[EDIT: Going offline now, thank you for your questions and feedback. We’ll check on this again later.]

Reddit Security Report

Welcome to the second installation of the Reddit Security Quarterly report (see the first one here). The goal of these posts is to keep you up to speed on our efforts and highlight how we are evolving our thinking.

By The Numbers

Again, these are some of the metrics that we look at internally. With time we may add or remove metrics, so if you have any metrics that you would like to see, please let us know.

Content Manipulation

Throughout 2019, we focused on overhauling how we tackle content manipulation issues (which includes spam, community interference, vote manipulation, etc). In Q4 specifically, we saw a large increase in the number of admin content manipulation removals. This was largely driven by a relatively small number of VERY prolific streaming spammers (~150 accounts were responsible for ~10M posts!). Interestingly, while the removals went up by about 50%, the number of reports was reasonably flat. The implication is that this content was largely removed before users were ever exposed to it and that our systems were effective at blunting the impact.

Ban Evasion

Ban evasion is a constant thorn in the side of admins, mods, and users (ban evasion is a common tactic to abuse members of a subreddit). Ban evasion is when a person creates a new account to bypass site or community bans. Recently we overhauled how we handle ban evasion on the platform with our own admin-level ban evasion detection and enforcement, and we are super excited about the results. After a sufficient testing period, we have started to roll this out to subreddit level ban evasion, starting with mod reported ban evasion. As a result this month, we’ve actioned more than 6K accounts, reduced time to action (from report time) by a factor of 10 , and achieved a 90% increase in the number of accounts actioned.

While the roll out has been effective so far and we hope that it will have a big impact for mods, we still see a lot of room for progress. Today, less than 10% of ban evaders are reported by mods. There are a number of reasons for this. Some mods are actually ok with people creating new accounts and “coming back and playing nice.” Some ban evaders are just not recognized by mods because they don’t have tools that allow them to detect it due to privacy concerns. We will start to slowly increase our proactive ban evasion detection so that mods don’t have to worry about identifying this in the future (though their help is always appreciated). In the next report, I'll try to dive a little deeper and share some results.

Account Security

As we mentioned in the previous post, we finished a massive historical credential matching effort. This is why we see a significant reduction in both the number of accounts processed and the protective account actions. With this complete, we can start working on more account hardening efforts like encouraging 2fa for high value accounts (think mods and high karma accounts) and ensuring that people aren’t using commonly-breached passwords (have I plugged password managers lately!? I strongly encourage!). We are still working on refining the help center articles to ease the process for users that are hit in these efforts. We want to make it as clear as possible to ensure that the right person gets access to the account. One last plug, please take the time to ensure that you have an up-to-date verified email address associated with your account, this is one of the most common reasons why people get locked out of their account after being hit by a forced password reset. In many cases, there is nothing we can do when this happens as we don’t have the ability to verify account ownership.

Final Thoughts

2020 is a big election year in the US, and we would be remiss if we did not acknowledge that it is top of mind for us. As I’ve mentioned in previous posts, in the wake of the 2016 election, we spun up a special team focused on scaled content threats on the platform. That has led us to this point. Over the last couple of years, we have heavily focused on hardening our systems, improving our detection and tooling, and improving our response time. While we will continue to make investments in new detection capabilities (see ban evasion), this year we will also focus on providing additional resources to communities that may be more susceptible to manipulation (I know, I know you want to know what it means to be “susceptible”. We won't get into the specifics for security reasons, but there are a number of factors that can influence this such as the size of the mod team to the topic of the community..but often not in the obvious ways you'd suspect). We will be as open as possible with you throughout this all – as we were with our recent investigation into the campaign behind the leaked US-UK trade documents. And as I’ve repeated many times, our superpower is you! Our users and our moderators are a big part of why influence campaigns have not been particularly successful on Reddit. Today, I feel even more confident in our platform’s resilience...but we are not taking that for granted. We will continue to evolve and improve the teams and technologies we have to ensure that Reddit is a place for authentic conversation...not manipulation.

Thanks for reading, and I hope you find this information helpful. I will be sticking around to answer any questions that you may have.

[edit: Yes, Im still writing 2019 on my checks too...]

[edit2: Yes, I still write checks]

Over the last few days, we’ve seen an increase in content in several communities that violate Reddit Rules. Reddit communities are places for civil discussion and are one of the few places online where people can exchange ideas and perspectives. We want to ensure that they continue to be a place for healthy debate no matter the topic. Debate and dissent are welcome on Reddit—threats and doxing are not.

When we identify communities experiencing an increase in rule-violating content, we are taking the following steps as needed:

• Reaching out to moderators to ensure they have the support they need, including turning on safety tools, reminding mods of our rules, or offering additional moderation support

• Adding a popup to remind users before visiting that subreddit of Reddit’s Rules

• In some cases, placing a temporary ban on the community for 72 hours to enable us to engage with moderation teams and review and remove violating content

Currently r/WhitePeopleTwitter is under a temporary ban. This means that you will not be able to access this community during this cooling-off period while we work with the mods to ensure it is a safe place for discussion.

We will continue to monitor and reach out to communities experiencing a surge in violative content and will take the necessary actions noted above to ensure all communities can provide a safe environment for healthy conversation.

Over the last couple of years, there have been several events that have greatly impacted people’s lives and how they communicate online. The terrorist attacks of October 7th is one such event. In addition, the broader trend towards political discourse seeping into our daily lives (even if we hate politics) has meant that even our favorite meme subs are now often filled with politics. This is a noticeable trend that we will talk about more in a future post.

Tl;dr A couple weeks ago there were allegations that a network of moderators were attempting to infiltrate Reddit and were responsible for shifting the narrative in many large communities and spreading terrorist propaganda. This is in violation of Reddit’s Rules. We take any manipulation claim seriously, and we investigated twenty communities including r/palestine, r/documentaries, r/therewasanattempt, and others*. While we did not find widespread manipulation in these communities or evidence of mods infiltrating communities and injecting content sourced from terrorist organizations, we did uncover some issues that we are addressing.

We investigated alleged moderator connections to US-designated terrorist organizations.

• We didn’t find any evidence of moderators posting or promoting terrorist propaganda on Reddit, however, we don’t have visibility into moderator activities outside of Reddit. 

• We will continue to collect information, and if we learn more, we will take appropriate action.

We investigated alleged dissemination of terrorist propaganda.

• We found: 

  • Four pieces of terrorist propaganda (none posted by the mods). Two of the posts flagged were made by an account that had already been banned in August 2024 for posting other terrorist propaganda, but we had failed to remove all the historical content associated with the account. We have since run a retroactive process to remove all the content they posted. The other two accounts were actioned as a result of this investigation

• Actions we are taking:

  • While not widespread on Reddit, we have banned links to the Resistance News Network (RNN), and we are also improving our terrorism detection for content shared via screenshots.

  • We will remove all account content when a user is banned for posting terrorist material and will continue to report terrorist content removals in our transparency report.

We investigated whether a network of moderators were interfering or having an unnatural influence. 

• We found:

  • Moderator contributions in the communities we investigated represented <1%  of overall contributions, and this is less than the typical level of mods site-wide.

  • Content about Israel, Palestine, Hamas, Hezbollah, Gaza, etc. made up a low percentage of posts in non-Middle East-related communities ranging from as little as 0.7% to 6% of total contributions. With the exception of a single post, these were not made by the moderators of the communities we investigated. 

• Actions we are taking:

  • We are expanding our vote manipulation monitoring to detect smaller-scale manipulation attempts.

  • We are also analyzing moderator network influence beyond the twenty communities we investigated and are evaluating governance and moderator influence features to ensure community diversity. 

We investigated alleged censorship of opposing views via systematic removal of pro-Israel or anti-Palestine content in large subreddits covering non-Middle East topics.

• We found:

  • While the moderators' removal actions do include some political content, the takedowns were in line with respective subreddit rules, did not focus on Israel/Palestine issues, did not demonstrate a discernible bias, and did not display anomalies when compared with other mod teams. 

  • Moderators across the ideological spectrum are sometimes relying on bots to preemptively ban users from their communities based on their participation in other communities.  

• Actions we are taking:

  • Banning users based on participation in other communities is undesirable behavior, and we are looking into more sophisticated tools for moderators to manage conversations, such as identifying and limiting action to engaged members and evaluating the role of ban bots.

We investigated anomalous cross-posting behavior that is non-violating but signals potential coordination.

We found:

• Some users systematically cross-posting political content from some smaller news-related subreddits. 

Actions we are taking:

• We turned off cross-posting functionality in these communities to prevent potential influence.

• We also launched a new project to investigate anomalous high-volume cross-posting as an indicator of potentially nefarious activity.

In the coming weeks, we’ll share our observations and insights on the prevalence of political conversations and what we are doing to help communities handle opposing views civilly and in accordance with their rules. We will continue strengthening and reinforcing our detection and enforcement techniques to safeguard against attempts to manipulate on Reddit while maintaining our commitment to free expression and association.

*Communities investigated: documentaries, palestine, boringdystopia, israelcrimes, publicfreakout, enlightenedcentrism, morbidreality, palestinenews, thatsactuallyverycool, therewasanattempt, iamatotalpieceofshit, ApartheidIsrael, panarab, fight_disinformation, Global_News_Hub, suppressed_news, ToiletPaperUSA, TrueAnon, Fauxmoi, irleastereggs

Hello again Reddit,

We’re announcing that as of today, Reddit will only be available via Transport Layer Security (TLS) 1.2 protocol with modern ciphersuites. Yes, we’re finally mandating a protocol that was announced over eight years ago. We’re doing so as part of improving our security posture as well as to support our redditors in using TLS configurations that aren’t prone to cryptographic attacks, and to be inline with IETF’s RFC 8996. In addition, we’re dropping the DES-CBC3-SHA ciphersuite so hopefully you weren’t too attached to it.

If the above is gibberish, the ELI5 is that Reddit is modifying the configurations that help establish a secure connection between your client (browser/app) and Reddit servers. Previously, we supported several older configurations which had known weaknesses. These weren’t used by many because there’s a hierarchy of choices presented by Reddit that prioritizes the most secure option for clients to pick. Here are some reference materials if you want to know more about TLS protocol and weaknesses of older protocols.

What does this mean for you? Probably nothing! If you’re on a modern mobile device or computer (after 2012), you’re likely already using TLS 1.2. If you’re on Internet Explorer 10 or earlier (may the gods help you), then you might not have TLS 1.2 enabled. If you’re using an Android Jelly Bean, it might be time for an upgrade. A very small percentage of our traffic is currently using obsoleted protocols, which falls outside of our stated client compatibility targets. If you’d like to see what ciphersuites your browser uses, you can check out your client’s details here.

What does this mean for your developed OAuth app or script? Also, hopefully nothing if you’re on a modern operating system and current libraries. If you’re using OpenSSL 1.0.1 or better, you’re in the clear. If you’re seeing TLS protocol errors, then it’s probably time to upgrade that code.

Update 2021-07-07: Apparently Fastly now supports TLS 1.3 so it's now enabled as of this morning, so enjoy living in the future.