Heroku: http://heroku.com/how/architecture

Platform Architecture

It's fast, it's easy, and it just works.

Multi-Tenant

Heroku is a multi-tenant platform and hosting environment. Your app doesn't need its own servers, slices, or clusters, and you'll benefit from continuous platform improvements.

Fully Managed

Each element in the diagram represents a large pool of servers invisibly managed by Heroku. Because we control all the infrastructure and software layers, we can present a clean interface for deploying your code.

Curation

We actively curate each layer, ensuring you have the latest in software and best practices, and that it's all stable and integrated smoothly.

Everything You Need

Together, these elements provide everything you need to run a modern, high-performance, scalable web app. Click around to understand the path of a request through the system.

HTTP Reverse Proxy

Nginx (open source, C)

This is the entry point for all requests coming into the platform. We maintain many of these front-end servers, and manage DNS, load balancing, and fail-over across them.

This layer is tuned for high performance, and only handles HTTP-level processing, such as SSL and gzip compression, before passing connections immediately into the stack.

HTTP Cache

Varnish (open source, C)

All requests, big or small, flow through this ultra-high-performance cache.

If the requested content is available in the cache (a "hit"), the cache responds immediately and the request never reaches the dynamic application.

Modern web apps get a huge performance boost from proper caching, and most frameworks (including Rails) have cache support built in.

Routing Mesh

(custom written, Erlang)

The routing mesh balances requests across your app's dynos, tracking load and intelligently routing traffic to available resources.

This makes your app more scalable and fault tolerant, as it can route traffic around misbehaving or overloaded app servers, as well as issue requests for new ones.

A distributed pool of dynamic HTTP routers, the mesh easily handles frequent and instantaneous registration and de-registration of dynos.

Dyno Grid

This is where the action happens. Your actual code runs inside a dyno process. You can run as many dynos as needed, and they're distributed across the grid.

The number of dynos running for your app can be increased or decreased instantly – it takes less than 2 seconds to start a dyno for most apps. The routing mesh can even hold connections while waiting for a dyno to start.

Memory Cache

Memcached (open source, C)

A high-performance in-memory cache is a standard part of the modern web app, and is built-in to every Heroku app – big or small.

Memcached is perfect for storing page fragments, the results of expensive database queries, or anything else you want blazingly fast access to.