CDN Introduction
CDNs are a good chunk of internet traffic today. % numbers correspond to the tuple (2017, 2022)
Content delivery networks (CDNs) improve websites, streaming, and download performance of Internet content consumption by end-users while reducing the cost to serve for content providers. A CDN is a collaborative collection of network elements spanning the Internet, where content is replicated over mirrored Web servers (i.e., point of presence (PoP), edge or replica servers), located at the edge of the Internet service providers' (ISPs') networks to which end-users are connected.
Traditional CDN: content is served into the CDN once, then content is delivered to end-users from the edge servers rather than serving each individual request directly from the content provider’s origin server.
AWS CDN (Cloudfront) Locations
There are many types of content supported by CDNs but we focus here on three that currently dominate internet traffic:
| App | How CDNs Helps |
|---|---|
| Web apps | Improve the delivery of website content (static and dynamic) so that websites perform better, load faster, and generate more revenue for content providers. CDNs not only cache static content at the edge but also handle dynamic, transactional content from e-commerce providers, online auction sites, by accelerating the data transfer to provide an improved experience to end-users. |
| Large-scale Video on Demand Streaming | For a Netflix use case, the CDN allows both technical and non-technical advantages. On top of the traditional CDN advantages, Netflix’s CDN called Open Connect, enables ISPs to provide a great Netflix experience for their mutual customers. The CDN localizes Netflix traffic as close as possible to their members, limiting the network and geographical distances that video must travel during playback. This of course benefits Netflix members, but it also benefits ISPs and internet users in general. |
| NOTE: You may notice that this course provided the background needed for understanding many elements of the Netflix software stack. | |
| Large-scale real time streaming (e.g. live TV) | In this use case e.g. the Hulu architecture shown below, approximately 800 streams are encoded and |
Hulu’s AWS-based architecture - from here
How it works
-
A user (Alice) requests a file (also called an asset) by using a URL with a special domain name, such as
.domain.net. The DNS routes the request to the best performing POP location, which is usually the POP that is geographically closest to the user. -
If no edge servers in the POP have the file in their cache, the POP requests the file from the origin server. The origin server any publicly accessible web server.
-
The origin server returns the file to an edge server in the POP.
-
An edge server in the POP caches the file and returns the file to the original requestor (Alice). The file remains cached on the edge server in the POP until the time-to-live (TTL) specified by its HTTP headers expires. If the origin server didn’t specify a TTL, the default TTL is seven days.
-
Additional users can then request the same file by using the same URL that Alice used, and can also be directed to the same POP.
-
If the TTL for the file hasn’t expired, the POP edge server returns the file directly from the cache. This process results in a faster, more responsive user experience.
Architectures
Note that most CDNs are deployed in a hierarchical fashion, with multiple levels of caching. In the lecture we will go through the Akamai CDN Architecture, slides 13-27 only.
From the description above we can distinguish many algorithmic problems that we need to solve as CDN engineers:
| Problem | Description |
|---|---|
| Assign content to edge caches | As Netflix quotes, “The Video catalog is too large to store everything at all locations, we need to pre-position the most popular video files at the locations where they are most likely to serve nearby user requests. These techniques are known as edge caching.” |
 |
|
| Cache expiration time | Each piece of content in a Cloud CDN cache has an associated expiration time, and it’s important to set an expiration that is appropriate for your use case. Because origin servers must resend content that expires on cache servers, you need to choose the expiration carefully. https://cloud.google.com/cdn/docs/best-practices |
| Cache key | Each cache entry in a Cloud CDN cache is identified by a cache key. When a request comes into the cache, the cache converts the URI of the request into a cache key, and then compares it with keys of cached entries. If it finds a match, the cache returns the object associated with that key. |
Cache assignment optimization
To achieve the optimal assignment Netflix forecasts what will be popular tomorrow and only uses disk and network resources for loading (filling) the caches during quiet, off-peak hours.
The prioritization objective is to simultaneously cache the most popular content but also minimize the number of file replacements to reduce fill traffic. For content placement, we do not need to predict popularity all the way to the user level, so we can take advantage of local or regional demand aggregation to increase accuracy. However, we need to predict at a highly granular level in another dimension: there can be hundreds of different files associated with each episode of a show so that we can provide all the encoding profiles and quality levels (bitrates) to support a wide range of devices and network conditions. We need separate predictions for each file because their size and popularity, therefore their cache efficiency, can vary by orders of magnitude.
This is a good overview of how ML helps Netflix operate their CDN.
Expiration time setting
Expiration times are specified by the max-age and s-maxage values in the Cache-Control header, defined by the HTTP specification. For example, the following Cache-Control header makes the associated content publicly readable and cache-able with a cache expiration of 72 hours (259200 seconds):