Twemproxy: A fast, light-weight proxy for memcached

Today, we’re open sourcing Twemproxy, a fast, light-weight proxy for the memcached protocol. Twemproxy was primarily developed to reduce open connections to our cache servers. At Twitter, we have thousands of front-end application servers, and we run multiple Unicorn instances on each one. Each instance establishes at least one connection to every cache server. As we add front-ends and increase the number of Unicorns per front-end, this connection count continues to increase. As a result the performance of the cache server degrades due to per-connection overhead.

We use Twemproxy to bound the number of connections on the cache servers by deploying it as a local proxy on each of the front-ends. The Unicorns on each front-end connect to the cache servers through the local proxy, which multiplexes and pipelines the front-end requests over a single connection to each cache server. This reduces the number of connections to each cache server by a significant factor.

We have been running Twemproxy in production for six months, during which time we’ve increased the number of front-ends by 30%. Twemproxy has been instrumental in allowing front-end and cache server pools to scale independently as capacity requirements change.

Twemproxy can:

• Maintain persistent server connections
• Keep connection count on cache servers low
• Pipeline requests and responses
• Use multiple server pools simultaneously
• Proxy the complete memcached ASCII protocol
• Easily configure server pools using a YAML file
• Use multiple hashing modes, including consistent hashing
• Increase observability using stats exposed on a monitoring port

Feel free to use, fork and contribute back to Twemproxy if you can. If you have any questions, feel free to file any issues on GitHub.