We Survived an Influx of Roughly 187,000 Reddit Users Caused By an Old Post on Secret Coca-Cola Millionaires
As tends to happen once every year or two, the unexpected “make the front page of Reddit” event occurred last night, in the midst of the site renovations; an inopportune time but I’ll take it.
Aaron and I were watching Jonathan Strange and Mr Norrell on iTunes at around 2 a.m. when I saw the MySQL server straining under an incredible amount of requests. It was the old post about the secret Coca-Cola millionaires in Quincy, Florida. Something like 90 requests a second were slamming the data center up in Michigan. Traffic source: Reddit. Short investigation later, turns out it became the number one post of the “Today I Learned” subreddit and made the overall front page of the site.
I got in touch with the webhosting engineers, one of whom began to work her magic. The team that put time in on the project today upgraded some PHP things and switched me over to an alternative setup they explained that should allow exponentially more connections whenever it happens again. Concurrently – and I have no idea how or why but for which I am extremely grateful – the co-founder of Cloudflare, which we use as the CDN, sent me a tweet telling me to set a page cache for the specific page that was receiving the huge traffic influx, which I was able to do so it made no MySQL requests at all. Within a minute or two, the live visitors had doubled. Shortly thereafter, tripled. It was crazy. I was watching 1,400+ people from around the world in the middle of the night at one point browsing the site (alas, I missed the 7-9 a.m. spike when it increased by several fold because I finally had to catch a few hours of sleep … I’ll need to check the stats to see how many people were on actively reading at one time).
The good news is that this diversion resulted in permanent improvements on the site I probably wouldn’t have gotten to otherwise, especially as I work to rationalize the javascript and css so it results in a mere fraction of the server requests going forward. The regular community is going to enjoy much faster load times than we otherwise would have; hopefully, fully visible by Autumn.
The behavior patterns were odd, though as these folks were not like you all:
- They all swooped in, sustained for nearly twelve hours, spending a couple of minutes on the page that was getting the attention, then left. Practically nobody looked at additional pages or clicked around the blog itself. That is not how I read the Internet at all.
- Almost no one left a public comment despite 937 comments (at this moment) on the Reddit page itself discussing the post.
- Several people in the discussion thread on Reddit were making some derivation of the inquiry: “What’s the next Coca-Cola?” I broke my usual rule and logged in to the official joshuakennon Reddit user, which is not the account I normally use, to leave a response to one of them, which you can read here.
Look at how beautifully Cloudflare handled the whole thing … of the 303.63+ Gigabytes in bandwidth served over the past twenty-four hours, it took care of 95.4% of it, leaving only 4.6% for the server itself to handle. When the site went down a few times for a minute or two, the “always on” feature served a static, non-live version of whatever each visitor was reading so the user experience wasn’t interrupted terribly despite 187,000 or so individual people showing up.
For all of the trouble (and my exhaustion today given I barely closed my eyes before having to get up to go to work), we blew the previous single-day advertising record out of the water as those page views turned into tasty tasty profits. The Reddit folks like to click ads. I feel as if I should use it to buy more Coke stock. It would be appropriate, if not entirely rational.
I have to say, I found myself feeling … uneasy … about the whole thing. It was as if our little (or not so little, anymore, as most of you regulars are lurkers) community was overrun. I didn’t like it. It could have been, too, having the most personal post I’ve ever written on the sidebar and front page, along with another post that had a picture of my nephew (update: I have subsequently removed many family pictures as the site has grown while trying to salvage the majority of the old posts). It made me feel uncomfortably exposed. We’re back to normal. They’re mostly gone (though any of you who stuck around permanently, welcome!). All is well. They got their “Today I Learned”, I got their money. It was a fair exchange.
Reader Comments (16)
Comments are presented chronologically, with replies indented beneath the comments to which they respond.
Liquid_Independence
July 11, 2015
Congrats on the publicity. It was smart of you to act quickly to add that page cache. People run into opportunities all the time, but recognizing one and leveraging resources to maximize its potential require someone to have the right temperament and a certain set of skills.
stegner
July 11, 2015
'That is not how I read the Internet at all.'
How do you read the internet? I tend to click everything that looks like it could lead anywhere, and then prune back from 10-20 open tabs. Anything that looks like it could yield interesting stuff going forward gets RSS'd. Rinse and repeat.
Bo
July 11, 2015
It's funny how people are always looking for "the next Coca-Cola", while the next coca-cola is actually, Coca-Cola.
Badkins
July 11, 2015
I must admit that I got a little selfishly worried when I read this post. This has become my favorite place on the internet. Everyone here is so much more educated and knowledgeable than I am, and I learn a great deal both from your articles and the wonderful, rational, level-headed commenters. I do tell people to come here, trying to share the knowledge, particularly when discussing investing or (more likely) explaining why gold isn't an investment. I suspect most of them don't follow my advice.
Scott McCarthy
July 11, 2015
I find it kind of funny that the most popular post on this is, basically, "How ordinary people made millions buying stocks on margin!" I mean, not that that was the message (or even the tone) of the article, but I am absolutely certain that was all some people took away.
Connelly Barnes
July 11, 2015
Cool! These viral events are always fun even regardless of the financial considerations.
My personal website gets per year maybe 100k visits, which I thought wasn't worth monetizing because my back of the envelope calculations indicated it's not much money. Occasionally I will have some research video that gets hundreds of thousands or millions of views on YouTube, but so far other people (typically, random journalists or University media relations) posted those and I didn't get any profit. *Shrug*.
I think those kind of things could be monetized but for me the computer science is more enjoyable and profitable. On the other hand, by having some research widely publicized, I have gotten better jobs, so "mind space" is definitely an alternative form of capital, even when it has not been directly monetized.
Engineer7006
July 11, 2015
I don't think the level of conversation on Reddit is quite the same as here, actually I think Joshua had a post along those lines a while back.
ChocoTaco369
July 11, 2015
Replying to Engineer7006
Ya think? Joshua's Reddit post was awesome, too. This is the response:
-----------------
RatCapers -2 points 1 day ago
Hand to God, our Christmas tree is made up of [strikethrough: Coca-Cola bottles] the blood of third world union leaders.
FTFY
-----------------
This is how the general public feels. Most people are bitter and living paycheck to paycheck due to their own choice to be ignorant. Sigh.
Mr.owenr
July 12, 2015
Replying to ChocoTaco369
Why do you think that people choose to remain ignorant? Besides the blood of third world union leaders, what else are you supposed to put inside coke bottles after drinking them?
ChocoTaco369
July 13, 2015
Replying to Mr.owenr
I think the human condition is to take the easy way out. We survived as a species by conserving energy and grouping together. As a result, the human condition is to be lazy and fall in line. The successful 1% are the group leaders, the rare breeds who like making decisions. Most people in the world have no interest in working hard and taking risks. It is far easier to blame the successes of others on "the man" and "the system" than adapt to the system and work your tail off within the system and conquer it. There are a lot of angry folks out there that have no one to blame but themselves, but spend all their time blaming others.
Derek
July 11, 2015
Replying to Engineer7006
No, it certainly is not at the same level. Still, I felt the need to read through it and find those few interesting comments buried in the pile of nonsense. A few asked genuine questions or tried to make reasonable arguments. I tip my proverbial cap to them.
Heath
July 11, 2015
Reminds me of the attention a typical small town recieves the three days the carnival is in town. A few brush sweeps after the mob has left and the community is back to basics!
innerscorecard
July 11, 2015
It's ironic that the visitors from reddit clicked on a lot of ads here, given how anti-commercial reddit is, even towards monetization by reddit itself.
innerscorecard
July 12, 2015
Hmm - and another paradox - why would reddit users click on ads relatively aggressively, but then not click the links to the content on the rest of this site? Baffling.
Any thoughts, Joshua?
Gilvus
July 12, 2015
Replying to innerscorecard
My hypothesis is that the attributes that actually make a piece of content viral (novelty, hilarity, sex, nostalgia, and any other trigger that naturally make humans go "oooooh" inside) are emotional in nature, and short-lived. In contrast, people who are deliberately hunting for certain types of content are the ones who tend to stick around, explore, and devour everything that the content creator has made provided that the content is consistent or closely interrelated within a certain channel/blog/site/series. Another thing I've noticed (and am annoyed by) on Josh's blog is that the ads tend to be very click-baity (probably based on prior browsing history and keywords found in Josh's writings), which could help explain why reddit users click on the ads: most of them are on "emotional autopilot" or "oooh, a piece of candy" mode instead of "I'm on a mission to better my life and accumulate knowledge capital" mindset. This isn't an affront to reddit users, because when I take a break I tend to engage the emotional autopilot as well.
I learned this when I created a viral video an in attempt to draw attention to my old YouTube channel where I filmed nifty science demonstrations on plate tectonics and other stuff. I found that for every ~50 people who watched the video, only 1 would explore my channel and click on the other stuff that was wholly unrelated to the hook. The viral video analytics were also interesting - my geographic location was where the vast majority of the views came from, presumably because people shared via social networking with their friends in the same area, and this was corroborated later when random people from around campus I'd never met before would recognize me. In sharp contrast, my other non-viral videos are geographically distributed according to where the population centers in the English-speaking world are, and the viewers tend to either quit watching halfway through the first one, or remain engaged and watch each episode in sequence.
If Joshua is willing to forward his own hypotheses and/or post some of the analytics data for us to pore over, I'd be super stoked.
Adam
July 14, 2015
I'm pretty sure that's actually the same post that brought me to this site, though I found it through twitter, not reddit.