KPI to be measured
Ask a question
Was this article helpful? 4 out of 4 found this helpful
There are things which can not wait, and one of those is watching videos: in fact, whenever a user wants to watch a video, he surely does not want to wait for loading time. How can we understand how long do our customers need to wait before they can start watching a video? We want to understand THRON delivery performance for real users around the world.
How THRON delivers content
THRON is designed to create, store, manage and use digital content. In order to achieve optimal performance and reliability, it leverages distributed architecture over a cloud computing infrastructure, providing several “nodes”. Each one of these nodes is capable of managing user requests such as watching a multimedia content.
A Content Delivery Network (CDN) is a network providing nodes, often referred to as "Points of Presence" (POP), which cache content remotely and allow content to be delivered by those nodes which are closer to the end user, avoiding packets to travel all the way to the origin servers.
THRON delivery is enhanced using an integrated CDN, its acceleration is being used both for multimedia content (like video/audio) and static content (like documents/images).
Let's consider the toughest scenario in terms of amount of data to be transferred, hence potential waiting time for end-users: this happens when video content are delivered via progressive download in HD quality. In this scenario video quality (bit rate) doesn't follow user bandwidth capacity and the risk of long buffering time is at its highest.
This article explains how measurements have been made and provides analysis results.
Performance measurements can be achieved in many ways, most of them focus on analyzing maximum server throughput by measuring performance within the same node/cluster/network or by using powerful internet data centers. Those measurements are usually applied to identify maximum theoretical performance, ignoring real user infrastructure because it creates a bottleneck.
A different method to measure performance is to focus on what real users will experience on their devices when visiting websites/projects, obtaining a Real User Monitoring (RUM). In order to obtain RUM, we use a standard browser (Google Chrome) that runs in a Catchpoint (www.catchpoint.com) node. A Catchpoint node is a standard PC connected to a standard connection, opening a specific web page.
- when the page has been loaded,
- when the player has started playback,
- when the player has ended playback.
For this test we are going to use the "Big Buck Bunny" video that can be used without restriction (https://peach.blender.org/about/) with a duration of 26.8 seconds. THRON player uses the HD version of the content which has a bitrate of 2 Mbps and the Progressive Download HTML5 Technology.
Each test has been grouped by geographical region and city.
To provide a readable result set, only average values are provided. Values variance (deviation from average) was very low anyway.
Results are provided in the following table after a brief description of table’s columns.
City: the city from which the measurement has been made.
TTC: Time To Complete, defines the delay time from the "playback started" event to the "playback ends" event. The video duration is 26.837 ms. The closer this value is to the video duration, more the playback of the content is smooth.
% Delay: The ratio between time spent for watching the video and the effective content duration. Empiric tests show that delay lower than 4% are not perceptible by human viewing the content so it means no interruptions.
|City||TTPB [ms]||TTC [ms]||% Delay|
|Beijing - China||2,500||27,988||2.33|
|Berlin - Germany||830||27,149||0.00|
|Bloomfield - New Jersey||1,436||27,521||0.79|
|Buenos Aires - Argentina||1,719||27,022||0.00|
|Hangzhou - China||2,019||28,039||2.47|
|London - UK||1,014||27,607||0.98|
|Los Angeles - California||1,492||27,543||0.92|
|Madrid - Spain||1,015||27,030||0.00|
|Melbourne - Australia||2,921||27,108||0.00|
|Moscow - Russia||1,142||27,238||0.11|
|Paris - France||969||27,312||0.14|
|Quingdao - China||2,084||28,273||3.31|
|Redmond - Washington||2,120||27,148||0.05|
|Rio De Janeiro - Brasil||2,563||27,550||0.77|
|Riyadh - Saudi Arabia||1,676||27,411||0.29|
|Rome - Italy||956||27,231||0.29|
|Seattle - Australia||2,113||27,418||0.74|
|Stockholm - Sweden||973||27,472||0.54|
|Tokio - Japan||1,630||27,076||0.00|
|Vancouver - Canada||1,791||27,033||0.00|
HD video quality in progressive download is used on all tests.
Video playback interruptions
No video playback interruptions have been recorded during all tests, not even from China. Progressive download with HD quality represents the toughest scenario for the end-user, because bit rate is not adaptive. We will create another test scenario for a better understanding of the quality used by end-users with adaptive bitrate.
Using THRON CDN provides best performances for worldwide content distribution with HD quality.
Videos playback is the most complex delivery case, so we can assume every content delivery (documents, images, audios) will achieve same or better results.