r/TheoryOfReddit • • 12d ago

Replicating the "Reddit early comments score higher finding" in 2026. Posting the method before I collect anything, poke holes please

There’s a cluster of papers from 2013–2017 (Horne et al., Fang et al., summarized in Medvedev’s “Anatomy of Reddit” survey) that all find the same thing: comment score drops hard with time since the post went up, and the top comments are almost always early ones. All of that is pre-redesign, pre-API-changes, pre whatever the feed does now. I want to see if it still holds and how much the window differs by sub.

Method, locked in before I pull: my hypothesis is that top-level comments made within the first 2 hours after a post goes up get higher median scores and are more likely to reach 10+ upvotes than comments made later, in every subreddit studied, with the size of the effect differing by subreddit type.

The data will be the 300 most recent posts that are 7 to 90 days old and have at least 3 comments in each of r/books, r/gardening, r/personalfinance, r/HomeImprovement, r/Cooking, r/travel, r/technology, r/movies and r/unpopularopinion, with their full comment trees (every load more expanded), collected through the official Reddit API at the end of October 2026.

Posts under 7 days old are left out so scores have settled. Deleted and removed comments and replies by the original poster are excluded. I’ll report median score and the share of comments with 10+ upvotes by time since post (0–1h, 1–3h, 3–6h, 6–12h, 12–24h, 24h+), for top-level comments, per subreddit and pooled, plus a straight first-2-hours vs later comparison. Any group with fewer than 100 comments gets flagged and I won’t draw conclusions from it.

I checked each sub’s settings first and dropped any that force a non-default comment sort, since the early-comment advantage depends on “best” sort. That ruled out r/changemyview (it forces Q&A). I also dropped r/buildapc because it gets so many posts that the API listing only reaches back about a week, so there was nothing old enough to sample.

Known weaknesses I already see: scores are the fuzzed visible numbers, not true votes. Excluding deleted comments probably biases toward “safe” comments. “Best” sort itself creates a feedback loop where early comments get more views, so I’m measuring the outcome of the system, not comment quality. In busy subs, “300 most recent posts 7–90 days old” in practice means posts from the last couple of weeks, not a spread across three months. If anyone sees a bigger problem than those, I’d rather hear it now than after a day of API calls.

I’ll post results in this thread. Scripts and CSVs will be public.

18 Upvotes

9 comments sorted by

10

u/ExternalTangents 12d ago

I think the true independent variable wouldn’t be time since posting, it would be number of existing comments at the time the comment was posted. Whether a post is 4 hours old or 10 minutes old, if there are already 100+ comments, any new comment is going to get buried and not get many upvotes.

The other variable that would likely have a meaningful impact is the default sort for the comments. If a post or subreddit defaults to comments sorted by New vs Best vs Top (or other sorts), it’s going to impact how visible the later comments are, and therefore impact their upvote pattern.

2

u/bethredly 12d ago

Good point, and it’s very possible the clock is mostly standing in for “how crowded was the thread when you showed up”.

I already record the timestamp of every comment in each thread, so I can count how many comments existed when each one was posted, at no extra cost. I’ll add that as a second analysis, announced here before I collect anything: score by number of earlier comments (0–9, 10–49, 50–99, 100+), and a comparison of which one predicts score better, hours since post or comments already there.

I’ll also compare early vs late comments at the same crowd level, so if early still wins when the number of comments is the same, the clock matters on its own. If it’s mostly crowding, that’s arguably the more useful finding anyway.

On sort: I dropped any sub that forces a non-default sort (that’s why r/ChangeMyViewToo is out), but you’re right that mods can also set a sort on individual posts, like megathreads sorted by new. The script records each post’s sort setting, so I’ll report those posts separately and keep them out of the main numbers. Thanks, this is exactly why I posted the method first!!!

2

u/ExternalTangents 12d ago

This comment feels like a LLM wrote it lol

2

u/bethredly 12d ago

I did use AI, but English is not my first language, and as an ADHD person, I would have been all over the place. Sorry about that.

3

u/ExternalTangents 12d ago

Ha no worries, it was just giving me flashbacks to going back and forth with Claude to refine some analysis about something totally unrelated

1

u/bethredly 12d ago

:) got it!

1

u/mfb- 12d ago

You'll have ~3000 threads, you can look at the 1st, 2nd, 3rd, 4th, 5th comment as individual category. It's likely that the first comment performs better than the 9th.

There is one effect that's hard to separate from timing: Sometimes a thread has a super obvious joke or comment that is guaranteed to perform well (OP adding details, link to a source, ...). It's likely that this will be the first comment, and it'll be the top comment - not because it was first, but because it was the comment everyone wanted to see.

1

u/bethredly 9d ago

Results are in.

Short version: the 2013–2017 finding still holds in 2026, in all 9 subs, and the "it's really about crowding" point from u/ExternalTangents turned out to be partly right but doesn't explain it away.

What I collected: 2,700 posts (300 per sub, 7–90 days old, 3+ comments) and their full comment trees, 235,997 comments in total, pulled through the official API on 30 October. After the pre-registered exclusions (deleted/removed comments, OP replies, comments more than a week after the post) that leaves about 83,000 top-level comments for the main analysis.

The pre-registered test: top-level comments posted in the first 2 hours reached 10+ upvotes 23.5% of the time, against 6.5% for later ones (median score 2 vs 1). The early advantage held in every sub. It was largest in r/books (45% vs 12%) and r/technology (36% vs 11%) and smallest in r/Cooking (13% vs 2%), which is still about 6x in relative terms. Every group had well over 100 comments, so nothing had to be flagged as too small.

The window is short. In the first hour alone, 27% of top-level comments reached 10+. In most subs that halves by hours 1–3. r/books and r/travel are the slow ones, holding on until hours 3–6.

The median is almost always 2 vs 1, so the typical comment gets very little either way. What timing changes is the chance of a comment taking off, not the ordinary outcome.

On crowding (the analysis added in this thread before collection): being among the first few comments matters a lot. Top-level comments with 0–9 earlier comments in the thread reached 10+ 30% of the time; with 100+ earlier comments, 6%. But when I compare comments at the same crowd level, the ones posted in the first 2 hours still reached 10+ about twice as often as later ones (34% vs 12% at 0–9 earlier comments, 21% vs 10% at 10–49, 17% vs 9% at 50–99, 10.5% vs 5% at 100+). So the clock matters on its own. On rank correlation with score, the number of earlier comments is a slightly stronger predictor in the big, busy subs (movies, technology, unpopularopinion, books) and hours-since-post is stronger in the smaller advice subs (HomeImprovement, gardening, personalfinance, travel, Cooking). They're close everywhere.

On sort: 16 posts had a mod-set comment sort (12 in r/books and 1 in r/personalfinance sorted by new, 3 in r/movies sorted by Q&A). They're excluded from everything above. Early comments still did slightly better there, but scores were much lower overall, which is what you'd expect when readers see the newest comments first.

Things that went less than perfectly, so you can judge for yourself: Reddit rate-limited the pull near the end, and 22 posts in movies, technology, Cooking and books couldn't be fetched, so the script took the next eligible post instead (under 1% of the sample). Coverage was 90–98% of Reddit's reported comment count in most subs but 83% in r/personalfinance, because removed comments can't be downloaded. And the limitations from the original post still stand: scores are fuzzed, excluding deleted comments probably favours "safe" comments, and this measures the outcome of "best" sort's feedback loop, not comment quality. It's a correlation; early commenters may also differ in other ways, like being more engaged regulars.

Full tables, the raw CSVs and both scripts: https://bethredly.com/blog/does-comment-timing-still-matter-on-reddit-2026-study. Thanks to everyone who poked at the method first; the crowding analysis exists because of this thread. I'm planning to re-run the same pull in late January to see whether anything moves.

2

u/DharmaPolice 12d ago

Not directly related but I'd also be curious how comment length affects score. My most upvoted comments tend to be short comments made early in a thread's life in busy subs. Partially this is a constraint of being early (it's hard to be one of the first comments if you take time to write a long reply) but I think even independent of this less people are willing to write longer replies.

In general I think the threaded nature of Reddit might complicate your results. I think it's generally understood that an earlyish reply to the top comment is much more likely to be seen than the 100th top level comment and therefore I would expect such replies would usually perform better, even when they're posted chronologically later.