Presenter: Adam Hitchcock
Track: II
Description:
What does it take to add realtime functionality to a truly web scale app. The result is the DISQUS realtime system, a highly concurrent system for allowing web clients to subscribe to arbitrary events in the DISQUS infrastructure.
People are more engaged when things happen in real time.
They get a lot of traffic, over a billion hits per month.
They call it RealERtime.
They use:
Old system was memcache pulling every 5 seconds.
New system is redis pub/sub and a Flask proxy cluster.
They quickly ran out of CPU on the Flask servers, so they moved things to a backend server to handle formatting.
This was better, but the Flask servers were still growing too quickly.
Replaced Flask servers and HA Proxy servers with an Nginx pub endpoint.
The Thoonk queue takes post_save and post_delete hooks, sits on tops of Redis.
Provides job semantics (what’s claimed, what’s not)
Gevent is the best thing ever, it lets you process things really fast.
Works with pipelines and mixins to compartmentalize logic.
Data pipelines using mixings (each mixin manipulates the data and passes it on).
Replaced webservers and redis pub/sub
EventSource Lets the browser handle the async calls instead of making javascript do it.
Darktime - Use existing network to load test, pull out a certain percentage of your user base at the new code.
Darkesttime - Testing a single instance with a ton of traffic.
Measure everything - Especially when numers don’t match up, even if it’s hardin a distributed system, and try to express things as +1 and -1 if you can.
When the pope was announced they saw over 6TB of traffic that day.
Check out HttpPushStreamModule!!!