Optimised localisation (OpLoc) for community networks.
Optimised Localisation (Oploc)
For Community Networks
Wahidah Md. Shah
August 2013
Lancaster University
Submitted in fulfilment of the requirements for the degree of
Doctor of Philosophy in Computer Science
Declaration
I declare that the work in this thesis has not been submitted for a
degree at any other university, and that the work is entirely my own.
Signature
Wahidah Md. Shah
Abstract
In recent years, there has been a tremendous increase in the amount of video-based
content distributed across the Internet; with this growth forecast to continue. A
range of distribution approaches have been developed in response to this demand,
including Peer-to-Peer (P2P) and Content Distribution Network (CDN). These
technologies are now vital to deliver content across the Internet, in order to keep
pace with user expectations and the ever growing number of video-on-demand and
video streaming services now on offer.
In spite of this growth in video-based services; Internet access in many parts
of the world remains a challenge. This phenomenon is known as the 'digital divide', which expresses the connectivity gap between those with and without high
quality access to the Internet. In response to this issue, community groups have
built their own infrastructures (which we refer to as 'Community Networks' within
this thesis), using technologies such as 'Wireless Mesh Networks', which involves
combining wireless networking technology and ad-hoc routing protocols in order to
create self-managing networks that can deliver Internet access to residents. Ordinarily these networks will have good network connectivity within the community,
but may share a limited connection to the wider Internet.
The use of video delivery technologies, such as P2P, within these Community
Networks introduces some key challenges. The shared connection to the wider
Internet may reduce the probability of successfully connecting clients with remote
clients and potentially creates network bottlenecks at the backhaul connection.
The use of P2P localisation strategies, which would encourage clients to connect
with local clients, could also result in issues with the availability of content.
This thesis proposes OpLoc, a system designed to strike a balance between saving bandwidth and preserving the live playback of P2P content within resource-
ii
ABSTRACT
iii
constrained networks. OpLoc achieves this using two key mechanisms; dynamic
localisation, which helps network providers determine the optimal level of localisation, depending on the physical network limitations and a local hook-in technique that is designed to preserve the continuity of live playback within resourceconstrained networks, whilst also balancing the bandwidth requirements associated
with P2P distribution.
OpLoc is evaluated through a variety of experiments using a testing framework.
This testing framework allows for experimentation within controlled conditions, in
order to provide reproducible and consistent results within larger networks (that
would otherwise be difficult to study using real implementations). This evaluation
shows how OpLoc is able to adapt to the network conditions; reducing bandwidth
whilst preserving live playback in a resource-constrained network environment.
Preface
I declare that the work presented in this thesis is, to the best of my knowledge and
belief, original and my own work. The material has not been submitted, either in
whole or part, for a degree at this or any other university.
Excepts of this thesis have been published in conference proceedings.
(Wahidah Md. Shah)
,
IV
Acknowledgment
Alhamdulillah, thank you Allah s.w.t, who gave me the opportunity, strength and
faith in completing this thesis. Apart from that, this thesis also would not have
been possible without the support of many people. I would like to take this opportunity to thank a number of these people.
Firstly, I would like to express gratitude and a million thanks to my supervisor,
Dr. Nicholas Race, for giving me continuous support, guidance, and encouragement throughout my postgraduate study at Lancaster University. Next, dedication
is for Dr. Johnathan Ishmael for his great support and help especially during the
experimentation and implementation phases. Without these two important people
giving me hope, belief and countless support since my first day, none of this would
be possible.
I would like to express a special dedication for my husband, Nor Rizal, daughters, Nor Wajahah and Nor Nabihah, son, Imam Solehin for being patience and
being by my side all the time. Also to mak and abah, and my siblings. Thank you
for your prayers and support.
Finally, not forgetting all the Malaysian friends in Lancaster for their friendships,
advise and sharing.
This research was funded by the Universiti Teknikal Malaysia Melaka, Malaysia.
v
Contents
Abstract
ii
Preface
iv
Acknowledgment
v
List of Figures
xi
List of Tables
xiv
List of Abbreviations
XV
1 Introduction
1.1
1
Growth in Video Distribution . . . . . . .
1
1.2 Supporting Growth in Video Distribution .
2
1.3 Community Networks .
3
1.4 Motivations . . . . . .
4
1.5 Thesis Aims and Contributions
5
1.6 Thesis Structure .
7
2 Background
9
2.1
Multimedia Distribution History .
10
2.2
Video Growth Distribution Architectures
11
2.2.1
Client-Server
11
2.2.2
Proxy Caching
13
2.2.3
Content Distribution Network (CDN) .
16
2.2.4
IP Multicast . . . . . . . . . . . . . .
17
2.2.5
Application Layer Multicast (ALM) .
19
.
vi
vii
CONTENTS
2.3
The Motivation of P2P Streaming .
20
2.4
Peer to Peer (P2P) . . . . . .
22
2.4.1
Types of P2P Overlays
25
2.4.2
Types of Overlay Topologies
30
2.4.2.1
Tree-based Overlay .
31
2.4.2.2
Mesh-based Overlay
35
2.4.3
2.5
3
Popular P2P Protocols
37
2.4.3.1
Gnutella .
37
2.4.3.2
Fast Track
38
2.4.3.3
BitTorrent .
38
Community Networks Challenges
40
2.5.1
Backhaul Connection . .
41
2.5.2
Upload Capacity of Peer
43
2.6
Locality Awareness
43
2.7
Conclusion .
45
Related Work
47
3.1
Overview of Issues with P2P .
47
3.2
Research in Best-Effort P2P
49
3.2.1
Peer Selection . . . .
50
3.2.1.1
Locality driven by ISP information
51
3.2.1.2
Locality driven by external services .
52
3.2.1.3
Locality based on client side detection
52
3.2.2
Piece Selection
......
54
3.2.3
Bandwidth Consumption .
55
3.2.4
Conclusion . . . . . . . . .
56
3.3
Gaps in Providing Live Streaming via P2P
56
3.4
Research in P2P Streaming
58
3.5
VoD Streaming via P2P
59
3.5.1
Peer Selection .
59
3.5.2
Piece Selection
60
3.5.3
Bandwidth Consumption .
62
3.5.4
Conclusion . . . . . . . . .
62
viii
CONTENTS
3.6
3.7
4
P2P Live Streaming Implementation
63
3.6.1
Peer Selection .
63
3.6.2
Piece Selection
65
3.6.3
Bandwidth Consumption .
68
Conclusion . . . . . . . . . . . .
68
Design of the OpLoc Framework
70
4.1
Overview . . . . . . . .
70
4.2
Motivations and Aims
71
4.3
Requirements of the OpLoc Framework
73
4.3.1
Peer Selection .
74
4.3.2
Piece Selection
77
4.3.3
Cost and Contention Issues in P2P Live Streaming
78
4.4
Design of OpLoc . . . . . . . . . . .
79
4.4.1
The Design of Peer Selection .
80
4.4.2
The Design of Network Model
83
4.4.3
The Design of Piece Selection
84
4.4.3.1
Bootstrapping Scenario
84
4.4.3.2
Reproduction of New Peer List
85
4.4.4
4.5
OpLoc Framework
Conclusion . .
•
0
0
••
0
86
•
5 Implementation
5.1
88
91
Testing Framework Platform .
91
5.1.1
Testing Framework Design
93
5.1.2
Testing Framework Entity Design
96
5.1.2.1
Seed ..
96
5.1.2.2
Tracker
97
5.1.2.3
Peer ..
97
5.1.3
Testing Framework Locality Design
101
5.1.4
Local Hook-in Point
102
5.1.5
Dynamic Localisation .
103
5.1.6
Testing Framework Environment Design
105
ix
CONTENTS
5.1.7
5.2
6
7
Testing Framework Validation
Conclusion .
• • • • • • • • • • • •
0
•
106
106
Evaluation
108
6.1
Evaluation Environment
109
6.1.1
Peer Selection .
110
6.1.2
Piece Selection
111
6.1.3
Experiment Design
111
6.1.3.1
Assumptions
112
6.1.3.2
Evaluation Parameters
112
6.2
Experiment Scenario . . . . . . .
114
6.3
Experiment Results and Analysis
117
6.3.1
Playback Consistency .
117
6.3.1.1
Random . . .
117
6.3.1.2
Half Locality
118
6.3.1.3
Strict Locality
119
6.3.1.4
Random Public Strict Local
120
6.3.2
Bandwidth Utilisation . . . . . . . .
120
6.3.3
Dynamic Localisation and Local Hook-in
121
6.3.4
Discussion . . . . . . . . . . . .
122
6.3.5
Optimised Localisation (OpLoc)
123
6.3.5.1
Playback Point Consistency
123
6.3.5.2
Bandwidth Utilisation
124
6.4 Discussion .
125
6.5
127
Conclusion .
Conclusion
145
7.1
Thesis Contributions
147
7.2
Thesis Impacts
148
7.3 Future Work .
149
7.4
151
Summary
Bibliography
152
CONTENTS
X
A Initial Experiment Results for the BitTorrent
167
Ao1 Introduction 0 0 0 0
0 167
Ao2 Experiment Setup 0
167
Ao3 Results from the Virtual Server Experiment
169
A0301
TCP Packet 0 0 0 0 0 0 0 0 0
0 170
Ao302
Downloading vs Uploading 0
174
A.4 Result Summary
175
Ao5 Initial Simulations
177
A0501
NS2 Simulator 0
0 177
A0502
Initial NS2 Setup
0 177
Ao6 Conclusion 0 0 0 0 0 0 0 0
0 179
List of Figures
2.1
Client-Server Architecture . . .
12
2.2
Distribution Path in Unicasting
13
2.3
Distribution Path in Proxy-assisted Client-Server network .
15
2.4
Distribution Path in CDN . . . .
17
2.5
Distribution Path in IP Multicast
19
2.6
P2P Overlay vs Physical Underlay Network
26
2. 7 Video Distribution Architectures
. .
27
2.8
General P2P Network Architectures .
28
2.9
FastTrack Hybrid P2P Approach
30
2.10 Tree-based P2P Overlay Network
31
2.11 Mesh-based P2P Overlay Network .
35
2.12 BitTorrent Network . . . . . . . . .
40
2.13 WMNs with limited backhaul connection
42
2.14 Peers with limited upload capability.
43
4.1
General P2P Streaming Components
73
4.2
Peer and Piece Selection in Existing P2P Systems
74
4.3
Possible Delivery Path With and Without Location-Awareness
75
4.4
General P2P Components with OpLoc Framework .
80
4.5
Peer and Piece Selection with OpLoc . . . . . . . .
81
4.6
The steps involved in Locality Policy component in WMNs Scenario 81
4.7
A Resource-Constrained Network with Localisation
87
4.8
Local Hook-in Point Design . . . . . . . . . . . . .
88
5.1
Testing Framework Design
95
5.2
Locality Design . . . . . .
96
Xl
LIST OF FIGURES
xii
5.3
Peer Connection Diagram . . . . . . . . .
104
5.4
Testing Framework Environment Topology
105
5.5
Validation Result Comparison between P2P-Next Real Code and
P2P-Next Testing Framework Experiment . . .
107
6.1
Ideal Performance of Liveness in Peer Playback
109
6.2
Scenario 1 - Wireless Mesh Networks
115
6.3
Scenario 2- ADSL Network . . . . .
115
6.4
Scenario 1 - No Locality or Piece Selection
129
6.5
Scenario 2 - No Locality or Piece Selection
130
6.6
Scenario 1 : 50% of the peers support locality, no piece selection. .
131
6. 7
Scenario 2 : 50% of the peers support locality, no piece selection. .
132
6.8
Scenario 1 : 100% of the peers support locality .
133
6.9
Scenario 2 : 100% of the peers support locality .
134
6.10 Scenario 1 : Average Playback Point- Random Public, Strict Local 135
6.11 Scenario 2 : Average Playback Point- Random Public, Strict Local 136
6.12 Bandwidth Usage (Scenario 1 - AS Capacity of 220)
137
6.13 Bandwidth Usage (Scenario 2- AS Capacity of 220)
138
6.14 Scenario 1 vs Scenario 2- Piece Selection (Piece Picker) .
139
6.15 Scenario 1 vs Scenario 2- Piece Selection (Piece Picker) Cont.
140
6.16 Scenario 1 : Average Playback Point- OpLoc
141
6.17 Scenario 2 : Average Playback Point - OpLoc
142
6.18 Scenario 1 : Bandwidth- OpLoc.
143
6.19 Scenario 2 : Bandwidth- OpLoc.
144
A.1 Network Topology of Virtual Client and Seeder
168
A.2 Total TCP Packet Sent from and to VM1
172
A.3 Total TCP Bytes Sent from and to VM1 .
172
A.4 Total TCP Packet Sent from and to VM2
173
A.5 Total TCP Bytes Sent from and to VM2 .
173
A.6 The Number of TCP Uploading, Downloading and BitTorrent message packet in BBC1 Streaming At VM1 . . . . . . . . . . . . . . . 174
LIST OF FIGURES
xiii
A.7 Graph of TCP uploading and downloading with BitTorrent message
in File Download At VM1 . . . . . . . . . . . .
175
A.8 The simulation results on rarest pieces selection
178
A.9 Simulation result on sequential order pieces selection
178
List of Tables
2.1
Centralised and Decentralised P2P architecture . . . . . . . . . . . 29
5.1
Parameters of Validation Experiment . . .
106
6.1
Testing Framework Configuration Details .
116
6.2
Reason for Piece Failure: No Locality .
118
6.3
Reason for Piece Failure: 50% Locality
119
6.4
Reason for Piece Failure: OpLoc . . .
125
A.1 BitTorrent Message and its Function
169
A.2 Total TCP Packets at VM Client
170
A.3 Total TCP Bytes VM Client
171
A.4 Result Summary . . . . . .
175
A.5 Summary of Packets Received and Sent at Each VM Client .
176
xiv
List of Abbreviations
ADSL
Asymmetric Digital Subscriber Line
ALM
Application Level Multicast
AP
Access Point
ARPANET Advanced Research Project Agency Network
AS
Autonomous System
AODV
Ad hoc On-Demand Distance Vector
BASS
BitTorrent Assisted Streaming System
BEST
BitTorrent Extensions for Streaming
BiToS
BitTorrent Streaming
CBN
Community Broadband Network
CON
Content Distribution Network
CN
Community Network
DAW
Distance-Availability Weighted
DHT
Distributed Hash Table
DNS
Dynamic Resolution Protocol
DPI
Deep Packet Inspection
DSL
Digital Subscriber Line
FCFS
First Come First Serve
XV
LIST OF ABBREVIATIONS
IGMP
Internet Group Message Protocol
IP
Internet Protocol
ISP
Internet Service Provider
LAN
Local Area Network
LRSWRX
Least Recently Shared With Requesting Expired Chunk
LTM
Location-aware Topology Matching
LRF
Local Rarest-First
MAC
Media Access Control
NICE
Internet Cooperative Environment
NLI
Network Localisation Index
OpLoc
Optimised Locality
P2P
Peer-to-Peer
PIM
Protocol Independent Multicast
Prism
Portal Infrastructure for Streaming Media
QoE
Quality of Service
RPSL
Random Public and Strict Local
RTT
Round Trip Time
TCP
Transport Control Protocol
TTL
Time-To-Live
TFT
Tit-for-Tat
UTAPS
Underlying Topology-aware Peer Selection
VM
Virtual Machine
VoD
Video on Demand
xvi
LIST OF ABBREVIATIONS
VRNS
Variable Random Neighbour-Selection
WMN
Wireless Mesh Networks
www
World Wide Web
xvii
Chapter 1
Introduction
1.1
Growth in Video Distribution
Over the last 10 years the Internet has had to support a significant growth in the
distribution of video content, with this trend forecast to continue. Cisco, in its
recent Visual Networking Index (VNI) 2012-2017 [21], highlights Internet video as
representing 57% of consumer Internet traffic in 2012 and forecasts that this will
reach 69% by 2017. The delivery of Internet video to TV also doubled in 2012,
and is expected to increase 5-fold by 2017. The sum of all forms of video including
P2P is predicted to be 80-90% of global consumer traffic by 2017.
The origins of video distribution via the Internet date back to 1993, with the
introduction of the Mosaic web browser that was capable of rendering various types
of media including text, audio, images and video. As computing hardware increased in capability this opened up the possibility of an increasingly time-dependent
set of media being presented, but this also highlighted the limitations of the Internet
as
a best-effort distribution infrastructure. This meant that video was typically
delivered as a static, non-streaming digital asset downloaded from a media server
and stored locally on a user's computer for playback.
Enhancements in last 20 years in network capacity and mechanisms to support
video distribution across the Internet have resulted in the widespread availability
of live-streaming video services. Services such as YouTube [112], online TV offered
1
1.2. SUPPORTING GROWTH IN VIDEO DISTRIBUTION
by PPLive [79], the BBC iPlayer [6] and video communication by Skype [93] all
highlight the new possibilities of the Internet. Multimedia video over the Internet
has become a dominant application and combined with ubiquitous network access
and mobile devices now makes the potential for anywhere, anytime video services.
Traffic from wireless devices is forecast to exceed the traffic from wired devices
by 2017 [21]. These statistics all point towards an increasing demand that will be
placed on the Internet as a platform for the reliable and cost effective distribution
of video content.
1.2
Supporting Growth in Video Distribution
The growth in video distribution has been partly driven by networking advancements, multimedia compression and encoding optimisations but clearly through
an increasing demand from users. The content available continues to improve in
terms of resolution and overall video quality, and the signs are that this trend will
continue as we move from Standard Definition (SD) to High Definition (HD) video
through to 3D and Super High Vision content. These step-changes in video quality
all place additional burden on a underlying infrastructure to support their distribution. As the Internet is fundamentally a best-effort delivery platform, one of the
established approaches for delivering video is to use a Content Distribution Network (known as a CDN). A CDN consists of a number of dedicated CDN servers
that are deployed at strategic locations within the Internet, which are responsible
for storing copies of popular content from the original source. When users request
content from the original source they are transparently re-directed to a geographically closer CDN server that has a copy of the media asset. This approach reduces
the burden on the original server and also shortens the network delivery path. However, the cost of implementing a CDN is high due to the number of edge servers
required. Moreover, as CDNs are based on a client-server architecture there is an
inherent limit in the serving capability of the system.
Peer-to-Peer (known as P2P) has emerged as an alternative mechanism for
2
1.3. COMMUNITY NETWORKS
delivering video. During the last 10 years it has been shown to be capable of distributing large amounts of content, including video material. Studies have shown
that a significant proportion of Internet traffic has been found to be P2P-based,
with P2P representing more than 50% of Europe traffic in 2008 [88]. In 2005, [75]
CacheLogic analysed the data streams of Internet backbones and ISPs, and found
that 61.4% of P2P traffic was video-based. P2P is particularly attractive as it
represents a scalable solution without the need for additional server support, as
is in the case with CDNs - with a P2P solution each device (known as a 'peer')
involved in the communication is capable of acting as both a server and client;
thus receiving content and sharing it with others.
1.3
Community Networks
Whilst modern technology has brought us closer to the notion of ubiquitous access
to the Internet, there is still a significant gap between the access levels available in
different parts of the world. Within the UK, this gap is particularly evident when
comparing the access levels available in urban areas against those found in rural
communities, who have very limited or often no access at all to the Internet. This
phenomenon is commonly referred to as the 'digital divide', which expresses the
connectivity gap betwe.en these groups of users.
In order to try and bridge this divide, many rural communities have been empowered to build, install and then operate their own networks. Community Broadband Network (CBN) is one example of an independent broadband operator that
provides telecommunication infrastructures to rural sites in the UK. CBN launched
in January 2004 [25] with the aim of encouraging and supporting communities in
establishing their own networks. These community-based networks (henceforth
abbreviated as CNs within this thesis) provide villages, neighbourhoods and local
businesses access to the Internet in addition to providing local networking services.
CNs benefit the community and neighbourhood in several ways. Users in a community network are able to share and communicate locally at lower cost, providing
3
1.4. MOTIVATIONS
faster access and consuming less network traffic. In addition, the community may
also host their own forums, servers and manage their networks based on their own
particular needs.
Community Networks are commonly built using wireless technology- this can
be achieved at relatively low cost without requiring significant investments in infrastructure. One particular technology used is Wireless Mesh Networks (WMNs)
which allow users to communicate with each other locally through a variety of
access points, or mesh nodes. A more detailed description of CNs and WMNs is
presented in section 2.5.
1.4
Motivations
Whilst the demand for high quality content continues apace, the ability to deliver
this across parts of the Internet remain a particular challenge. Those on the fringe
of the network- such as users of a Community Network, are often unable to reliably access streamed media simply because of the resource limitations and external
network capacity that is available to them. Technologies such as P2P should, in
theory, be able to help; the concept of P2P would mean that users requesting
media items could have them provided by other users in the same community without relying on external network access and hence reducing the costs of distribution. The fundamental design of P2P allows clients that view video streaming
to upload the same video to other clients at the same time. Thus, distributing
video streaming with P2P can reduce the cost of having media servers in place.
This also enables clients to actively contribute their resources such as storage space
and bandwidth to the overall network. Allowing clients to re-distribute video to
other clients makes the whole system scalable to support a large number of clients.
In addition, if the existing download source fails or on when a nearer source is
located, clients are able to change to a different source of video content.
Despite the numerous benefits of P2P, there are some key challenges. Firstly,
the concept of locality is still not common in P2P deployments. Most P2P clients
4
1.5. THESIS AIMS AND CONTRIBUTIONS
select remote peers without consideration of the underlying network, which increases the amount of cross-over traffic among different countries. Based on a
research work published in 2012 [5], they claim that there is still no work on the
locality in live video streaming over P2P networks. Moreover, when considering
P2P within CNs for live streaming the problem is exacerbated. Such bandwidth
restrictions reduce the probability of successful connections of local clients to other
clients that are located outside of the local network. Frequent communication with
the remote clients challenges the sustainability of the network and creates a network bottleneck at the backhaul connection. A simple localisation however does
not well performed if peer has low capability as this limits number of successful
connection. Thus, instead simple localisation, dynamic locality is introduces in
order to counter a network with low peer capability.
Whilst the use of localisation can help there is a secondary issue that relates to
the 'liveness' of streamed content; that happen when requested live contents are
not available. Apart from localisation, an efficient mechanism is needed in order
to ensure that the required live contents are available at the localised clients.
The combination of localisation and the mechanism needs to balance the aims
of bandwidth saving and the requirement of preserving the 'liveness' of streamed
content.
1.5
Thesis Aims and Contributions
This thesis is concerned with the enhancements necessary for P2P systems to be
capable of delivering live streaming within Community Network environments that
have limited external connectivity. The thesis proposes a new dynamic localisation
scheme and a mechanism to preserve the playback of live content whilst balancing
the bandwidth requirements.
The specific aims and contributions of the thesis are as follows:
• To study the properties of P2P systems and the impact of loca-
5
1.5. THESIS AIMS AND CONTRIBUTIONS
lisation when delivering live streams. Existing studies have reported
significant findings which show the research gap in providing efficient streaming delivery. These findings highlight a need to provide better peer selection
algorithms, and also the introduction of localisation to reduce remote communication and bandwidth costs. Experiments were carried out through a
series of simulations and an implementation in order to study the sharing behaviour among peers in P2P (presented in the Appendix). Discussion on the
issues of the standard P2P and P2P streaming are also reported in Chapter
3.
• To design a framework for efficient localisation within P2P technologies.
An optimised localisation mechanism encourages peers to favour local clients
and at the same time, preserves the live playback. In live streaming, content
is lively produce at the server and thus efficient mechanism must available in
order to ensure live content availability and accessibility. With localisation
in place, some pre-cautionary actions need to be taken to ensure required live
content is available at the local clients. Otherwise, the live content cannot be
downloaded and peers are unable to continue to play the streaming content.
By having localisation that ideally favours the local clients, a mechanism in
piece selection has to ensure live contents are available in the local clients.
Thus, localisation needs to balance the aims - localising the clients and also
in providing the required live content locally in order to preserve the live
playback.
• To engineer the optimised localisation framework onto CNs. The
design of the framework is built in a python testing framework, which run
P2P live streaming within the CNs network. The localisation framework
is targeting to select a sufficient amount of local clients and preserve the
live playback by ensuring the live contents are available at the clients. The
6
For Community Networks
Wahidah Md. Shah
August 2013
Lancaster University
Submitted in fulfilment of the requirements for the degree of
Doctor of Philosophy in Computer Science
Declaration
I declare that the work in this thesis has not been submitted for a
degree at any other university, and that the work is entirely my own.
Signature
Wahidah Md. Shah
Abstract
In recent years, there has been a tremendous increase in the amount of video-based
content distributed across the Internet; with this growth forecast to continue. A
range of distribution approaches have been developed in response to this demand,
including Peer-to-Peer (P2P) and Content Distribution Network (CDN). These
technologies are now vital to deliver content across the Internet, in order to keep
pace with user expectations and the ever growing number of video-on-demand and
video streaming services now on offer.
In spite of this growth in video-based services; Internet access in many parts
of the world remains a challenge. This phenomenon is known as the 'digital divide', which expresses the connectivity gap between those with and without high
quality access to the Internet. In response to this issue, community groups have
built their own infrastructures (which we refer to as 'Community Networks' within
this thesis), using technologies such as 'Wireless Mesh Networks', which involves
combining wireless networking technology and ad-hoc routing protocols in order to
create self-managing networks that can deliver Internet access to residents. Ordinarily these networks will have good network connectivity within the community,
but may share a limited connection to the wider Internet.
The use of video delivery technologies, such as P2P, within these Community
Networks introduces some key challenges. The shared connection to the wider
Internet may reduce the probability of successfully connecting clients with remote
clients and potentially creates network bottlenecks at the backhaul connection.
The use of P2P localisation strategies, which would encourage clients to connect
with local clients, could also result in issues with the availability of content.
This thesis proposes OpLoc, a system designed to strike a balance between saving bandwidth and preserving the live playback of P2P content within resource-
ii
ABSTRACT
iii
constrained networks. OpLoc achieves this using two key mechanisms; dynamic
localisation, which helps network providers determine the optimal level of localisation, depending on the physical network limitations and a local hook-in technique that is designed to preserve the continuity of live playback within resourceconstrained networks, whilst also balancing the bandwidth requirements associated
with P2P distribution.
OpLoc is evaluated through a variety of experiments using a testing framework.
This testing framework allows for experimentation within controlled conditions, in
order to provide reproducible and consistent results within larger networks (that
would otherwise be difficult to study using real implementations). This evaluation
shows how OpLoc is able to adapt to the network conditions; reducing bandwidth
whilst preserving live playback in a resource-constrained network environment.
Preface
I declare that the work presented in this thesis is, to the best of my knowledge and
belief, original and my own work. The material has not been submitted, either in
whole or part, for a degree at this or any other university.
Excepts of this thesis have been published in conference proceedings.
(Wahidah Md. Shah)
,
IV
Acknowledgment
Alhamdulillah, thank you Allah s.w.t, who gave me the opportunity, strength and
faith in completing this thesis. Apart from that, this thesis also would not have
been possible without the support of many people. I would like to take this opportunity to thank a number of these people.
Firstly, I would like to express gratitude and a million thanks to my supervisor,
Dr. Nicholas Race, for giving me continuous support, guidance, and encouragement throughout my postgraduate study at Lancaster University. Next, dedication
is for Dr. Johnathan Ishmael for his great support and help especially during the
experimentation and implementation phases. Without these two important people
giving me hope, belief and countless support since my first day, none of this would
be possible.
I would like to express a special dedication for my husband, Nor Rizal, daughters, Nor Wajahah and Nor Nabihah, son, Imam Solehin for being patience and
being by my side all the time. Also to mak and abah, and my siblings. Thank you
for your prayers and support.
Finally, not forgetting all the Malaysian friends in Lancaster for their friendships,
advise and sharing.
This research was funded by the Universiti Teknikal Malaysia Melaka, Malaysia.
v
Contents
Abstract
ii
Preface
iv
Acknowledgment
v
List of Figures
xi
List of Tables
xiv
List of Abbreviations
XV
1 Introduction
1.1
1
Growth in Video Distribution . . . . . . .
1
1.2 Supporting Growth in Video Distribution .
2
1.3 Community Networks .
3
1.4 Motivations . . . . . .
4
1.5 Thesis Aims and Contributions
5
1.6 Thesis Structure .
7
2 Background
9
2.1
Multimedia Distribution History .
10
2.2
Video Growth Distribution Architectures
11
2.2.1
Client-Server
11
2.2.2
Proxy Caching
13
2.2.3
Content Distribution Network (CDN) .
16
2.2.4
IP Multicast . . . . . . . . . . . . . .
17
2.2.5
Application Layer Multicast (ALM) .
19
.
vi
vii
CONTENTS
2.3
The Motivation of P2P Streaming .
20
2.4
Peer to Peer (P2P) . . . . . .
22
2.4.1
Types of P2P Overlays
25
2.4.2
Types of Overlay Topologies
30
2.4.2.1
Tree-based Overlay .
31
2.4.2.2
Mesh-based Overlay
35
2.4.3
2.5
3
Popular P2P Protocols
37
2.4.3.1
Gnutella .
37
2.4.3.2
Fast Track
38
2.4.3.3
BitTorrent .
38
Community Networks Challenges
40
2.5.1
Backhaul Connection . .
41
2.5.2
Upload Capacity of Peer
43
2.6
Locality Awareness
43
2.7
Conclusion .
45
Related Work
47
3.1
Overview of Issues with P2P .
47
3.2
Research in Best-Effort P2P
49
3.2.1
Peer Selection . . . .
50
3.2.1.1
Locality driven by ISP information
51
3.2.1.2
Locality driven by external services .
52
3.2.1.3
Locality based on client side detection
52
3.2.2
Piece Selection
......
54
3.2.3
Bandwidth Consumption .
55
3.2.4
Conclusion . . . . . . . . .
56
3.3
Gaps in Providing Live Streaming via P2P
56
3.4
Research in P2P Streaming
58
3.5
VoD Streaming via P2P
59
3.5.1
Peer Selection .
59
3.5.2
Piece Selection
60
3.5.3
Bandwidth Consumption .
62
3.5.4
Conclusion . . . . . . . . .
62
viii
CONTENTS
3.6
3.7
4
P2P Live Streaming Implementation
63
3.6.1
Peer Selection .
63
3.6.2
Piece Selection
65
3.6.3
Bandwidth Consumption .
68
Conclusion . . . . . . . . . . . .
68
Design of the OpLoc Framework
70
4.1
Overview . . . . . . . .
70
4.2
Motivations and Aims
71
4.3
Requirements of the OpLoc Framework
73
4.3.1
Peer Selection .
74
4.3.2
Piece Selection
77
4.3.3
Cost and Contention Issues in P2P Live Streaming
78
4.4
Design of OpLoc . . . . . . . . . . .
79
4.4.1
The Design of Peer Selection .
80
4.4.2
The Design of Network Model
83
4.4.3
The Design of Piece Selection
84
4.4.3.1
Bootstrapping Scenario
84
4.4.3.2
Reproduction of New Peer List
85
4.4.4
4.5
OpLoc Framework
Conclusion . .
•
0
0
••
0
86
•
5 Implementation
5.1
88
91
Testing Framework Platform .
91
5.1.1
Testing Framework Design
93
5.1.2
Testing Framework Entity Design
96
5.1.2.1
Seed ..
96
5.1.2.2
Tracker
97
5.1.2.3
Peer ..
97
5.1.3
Testing Framework Locality Design
101
5.1.4
Local Hook-in Point
102
5.1.5
Dynamic Localisation .
103
5.1.6
Testing Framework Environment Design
105
ix
CONTENTS
5.1.7
5.2
6
7
Testing Framework Validation
Conclusion .
• • • • • • • • • • • •
0
•
106
106
Evaluation
108
6.1
Evaluation Environment
109
6.1.1
Peer Selection .
110
6.1.2
Piece Selection
111
6.1.3
Experiment Design
111
6.1.3.1
Assumptions
112
6.1.3.2
Evaluation Parameters
112
6.2
Experiment Scenario . . . . . . .
114
6.3
Experiment Results and Analysis
117
6.3.1
Playback Consistency .
117
6.3.1.1
Random . . .
117
6.3.1.2
Half Locality
118
6.3.1.3
Strict Locality
119
6.3.1.4
Random Public Strict Local
120
6.3.2
Bandwidth Utilisation . . . . . . . .
120
6.3.3
Dynamic Localisation and Local Hook-in
121
6.3.4
Discussion . . . . . . . . . . . .
122
6.3.5
Optimised Localisation (OpLoc)
123
6.3.5.1
Playback Point Consistency
123
6.3.5.2
Bandwidth Utilisation
124
6.4 Discussion .
125
6.5
127
Conclusion .
Conclusion
145
7.1
Thesis Contributions
147
7.2
Thesis Impacts
148
7.3 Future Work .
149
7.4
151
Summary
Bibliography
152
CONTENTS
X
A Initial Experiment Results for the BitTorrent
167
Ao1 Introduction 0 0 0 0
0 167
Ao2 Experiment Setup 0
167
Ao3 Results from the Virtual Server Experiment
169
A0301
TCP Packet 0 0 0 0 0 0 0 0 0
0 170
Ao302
Downloading vs Uploading 0
174
A.4 Result Summary
175
Ao5 Initial Simulations
177
A0501
NS2 Simulator 0
0 177
A0502
Initial NS2 Setup
0 177
Ao6 Conclusion 0 0 0 0 0 0 0 0
0 179
List of Figures
2.1
Client-Server Architecture . . .
12
2.2
Distribution Path in Unicasting
13
2.3
Distribution Path in Proxy-assisted Client-Server network .
15
2.4
Distribution Path in CDN . . . .
17
2.5
Distribution Path in IP Multicast
19
2.6
P2P Overlay vs Physical Underlay Network
26
2. 7 Video Distribution Architectures
. .
27
2.8
General P2P Network Architectures .
28
2.9
FastTrack Hybrid P2P Approach
30
2.10 Tree-based P2P Overlay Network
31
2.11 Mesh-based P2P Overlay Network .
35
2.12 BitTorrent Network . . . . . . . . .
40
2.13 WMNs with limited backhaul connection
42
2.14 Peers with limited upload capability.
43
4.1
General P2P Streaming Components
73
4.2
Peer and Piece Selection in Existing P2P Systems
74
4.3
Possible Delivery Path With and Without Location-Awareness
75
4.4
General P2P Components with OpLoc Framework .
80
4.5
Peer and Piece Selection with OpLoc . . . . . . . .
81
4.6
The steps involved in Locality Policy component in WMNs Scenario 81
4.7
A Resource-Constrained Network with Localisation
87
4.8
Local Hook-in Point Design . . . . . . . . . . . . .
88
5.1
Testing Framework Design
95
5.2
Locality Design . . . . . .
96
Xl
LIST OF FIGURES
xii
5.3
Peer Connection Diagram . . . . . . . . .
104
5.4
Testing Framework Environment Topology
105
5.5
Validation Result Comparison between P2P-Next Real Code and
P2P-Next Testing Framework Experiment . . .
107
6.1
Ideal Performance of Liveness in Peer Playback
109
6.2
Scenario 1 - Wireless Mesh Networks
115
6.3
Scenario 2- ADSL Network . . . . .
115
6.4
Scenario 1 - No Locality or Piece Selection
129
6.5
Scenario 2 - No Locality or Piece Selection
130
6.6
Scenario 1 : 50% of the peers support locality, no piece selection. .
131
6. 7
Scenario 2 : 50% of the peers support locality, no piece selection. .
132
6.8
Scenario 1 : 100% of the peers support locality .
133
6.9
Scenario 2 : 100% of the peers support locality .
134
6.10 Scenario 1 : Average Playback Point- Random Public, Strict Local 135
6.11 Scenario 2 : Average Playback Point- Random Public, Strict Local 136
6.12 Bandwidth Usage (Scenario 1 - AS Capacity of 220)
137
6.13 Bandwidth Usage (Scenario 2- AS Capacity of 220)
138
6.14 Scenario 1 vs Scenario 2- Piece Selection (Piece Picker) .
139
6.15 Scenario 1 vs Scenario 2- Piece Selection (Piece Picker) Cont.
140
6.16 Scenario 1 : Average Playback Point- OpLoc
141
6.17 Scenario 2 : Average Playback Point - OpLoc
142
6.18 Scenario 1 : Bandwidth- OpLoc.
143
6.19 Scenario 2 : Bandwidth- OpLoc.
144
A.1 Network Topology of Virtual Client and Seeder
168
A.2 Total TCP Packet Sent from and to VM1
172
A.3 Total TCP Bytes Sent from and to VM1 .
172
A.4 Total TCP Packet Sent from and to VM2
173
A.5 Total TCP Bytes Sent from and to VM2 .
173
A.6 The Number of TCP Uploading, Downloading and BitTorrent message packet in BBC1 Streaming At VM1 . . . . . . . . . . . . . . . 174
LIST OF FIGURES
xiii
A.7 Graph of TCP uploading and downloading with BitTorrent message
in File Download At VM1 . . . . . . . . . . . .
175
A.8 The simulation results on rarest pieces selection
178
A.9 Simulation result on sequential order pieces selection
178
List of Tables
2.1
Centralised and Decentralised P2P architecture . . . . . . . . . . . 29
5.1
Parameters of Validation Experiment . . .
106
6.1
Testing Framework Configuration Details .
116
6.2
Reason for Piece Failure: No Locality .
118
6.3
Reason for Piece Failure: 50% Locality
119
6.4
Reason for Piece Failure: OpLoc . . .
125
A.1 BitTorrent Message and its Function
169
A.2 Total TCP Packets at VM Client
170
A.3 Total TCP Bytes VM Client
171
A.4 Result Summary . . . . . .
175
A.5 Summary of Packets Received and Sent at Each VM Client .
176
xiv
List of Abbreviations
ADSL
Asymmetric Digital Subscriber Line
ALM
Application Level Multicast
AP
Access Point
ARPANET Advanced Research Project Agency Network
AS
Autonomous System
AODV
Ad hoc On-Demand Distance Vector
BASS
BitTorrent Assisted Streaming System
BEST
BitTorrent Extensions for Streaming
BiToS
BitTorrent Streaming
CBN
Community Broadband Network
CON
Content Distribution Network
CN
Community Network
DAW
Distance-Availability Weighted
DHT
Distributed Hash Table
DNS
Dynamic Resolution Protocol
DPI
Deep Packet Inspection
DSL
Digital Subscriber Line
FCFS
First Come First Serve
XV
LIST OF ABBREVIATIONS
IGMP
Internet Group Message Protocol
IP
Internet Protocol
ISP
Internet Service Provider
LAN
Local Area Network
LRSWRX
Least Recently Shared With Requesting Expired Chunk
LTM
Location-aware Topology Matching
LRF
Local Rarest-First
MAC
Media Access Control
NICE
Internet Cooperative Environment
NLI
Network Localisation Index
OpLoc
Optimised Locality
P2P
Peer-to-Peer
PIM
Protocol Independent Multicast
Prism
Portal Infrastructure for Streaming Media
QoE
Quality of Service
RPSL
Random Public and Strict Local
RTT
Round Trip Time
TCP
Transport Control Protocol
TTL
Time-To-Live
TFT
Tit-for-Tat
UTAPS
Underlying Topology-aware Peer Selection
VM
Virtual Machine
VoD
Video on Demand
xvi
LIST OF ABBREVIATIONS
VRNS
Variable Random Neighbour-Selection
WMN
Wireless Mesh Networks
www
World Wide Web
xvii
Chapter 1
Introduction
1.1
Growth in Video Distribution
Over the last 10 years the Internet has had to support a significant growth in the
distribution of video content, with this trend forecast to continue. Cisco, in its
recent Visual Networking Index (VNI) 2012-2017 [21], highlights Internet video as
representing 57% of consumer Internet traffic in 2012 and forecasts that this will
reach 69% by 2017. The delivery of Internet video to TV also doubled in 2012,
and is expected to increase 5-fold by 2017. The sum of all forms of video including
P2P is predicted to be 80-90% of global consumer traffic by 2017.
The origins of video distribution via the Internet date back to 1993, with the
introduction of the Mosaic web browser that was capable of rendering various types
of media including text, audio, images and video. As computing hardware increased in capability this opened up the possibility of an increasingly time-dependent
set of media being presented, but this also highlighted the limitations of the Internet
as
a best-effort distribution infrastructure. This meant that video was typically
delivered as a static, non-streaming digital asset downloaded from a media server
and stored locally on a user's computer for playback.
Enhancements in last 20 years in network capacity and mechanisms to support
video distribution across the Internet have resulted in the widespread availability
of live-streaming video services. Services such as YouTube [112], online TV offered
1
1.2. SUPPORTING GROWTH IN VIDEO DISTRIBUTION
by PPLive [79], the BBC iPlayer [6] and video communication by Skype [93] all
highlight the new possibilities of the Internet. Multimedia video over the Internet
has become a dominant application and combined with ubiquitous network access
and mobile devices now makes the potential for anywhere, anytime video services.
Traffic from wireless devices is forecast to exceed the traffic from wired devices
by 2017 [21]. These statistics all point towards an increasing demand that will be
placed on the Internet as a platform for the reliable and cost effective distribution
of video content.
1.2
Supporting Growth in Video Distribution
The growth in video distribution has been partly driven by networking advancements, multimedia compression and encoding optimisations but clearly through
an increasing demand from users. The content available continues to improve in
terms of resolution and overall video quality, and the signs are that this trend will
continue as we move from Standard Definition (SD) to High Definition (HD) video
through to 3D and Super High Vision content. These step-changes in video quality
all place additional burden on a underlying infrastructure to support their distribution. As the Internet is fundamentally a best-effort delivery platform, one of the
established approaches for delivering video is to use a Content Distribution Network (known as a CDN). A CDN consists of a number of dedicated CDN servers
that are deployed at strategic locations within the Internet, which are responsible
for storing copies of popular content from the original source. When users request
content from the original source they are transparently re-directed to a geographically closer CDN server that has a copy of the media asset. This approach reduces
the burden on the original server and also shortens the network delivery path. However, the cost of implementing a CDN is high due to the number of edge servers
required. Moreover, as CDNs are based on a client-server architecture there is an
inherent limit in the serving capability of the system.
Peer-to-Peer (known as P2P) has emerged as an alternative mechanism for
2
1.3. COMMUNITY NETWORKS
delivering video. During the last 10 years it has been shown to be capable of distributing large amounts of content, including video material. Studies have shown
that a significant proportion of Internet traffic has been found to be P2P-based,
with P2P representing more than 50% of Europe traffic in 2008 [88]. In 2005, [75]
CacheLogic analysed the data streams of Internet backbones and ISPs, and found
that 61.4% of P2P traffic was video-based. P2P is particularly attractive as it
represents a scalable solution without the need for additional server support, as
is in the case with CDNs - with a P2P solution each device (known as a 'peer')
involved in the communication is capable of acting as both a server and client;
thus receiving content and sharing it with others.
1.3
Community Networks
Whilst modern technology has brought us closer to the notion of ubiquitous access
to the Internet, there is still a significant gap between the access levels available in
different parts of the world. Within the UK, this gap is particularly evident when
comparing the access levels available in urban areas against those found in rural
communities, who have very limited or often no access at all to the Internet. This
phenomenon is commonly referred to as the 'digital divide', which expresses the
connectivity gap betwe.en these groups of users.
In order to try and bridge this divide, many rural communities have been empowered to build, install and then operate their own networks. Community Broadband Network (CBN) is one example of an independent broadband operator that
provides telecommunication infrastructures to rural sites in the UK. CBN launched
in January 2004 [25] with the aim of encouraging and supporting communities in
establishing their own networks. These community-based networks (henceforth
abbreviated as CNs within this thesis) provide villages, neighbourhoods and local
businesses access to the Internet in addition to providing local networking services.
CNs benefit the community and neighbourhood in several ways. Users in a community network are able to share and communicate locally at lower cost, providing
3
1.4. MOTIVATIONS
faster access and consuming less network traffic. In addition, the community may
also host their own forums, servers and manage their networks based on their own
particular needs.
Community Networks are commonly built using wireless technology- this can
be achieved at relatively low cost without requiring significant investments in infrastructure. One particular technology used is Wireless Mesh Networks (WMNs)
which allow users to communicate with each other locally through a variety of
access points, or mesh nodes. A more detailed description of CNs and WMNs is
presented in section 2.5.
1.4
Motivations
Whilst the demand for high quality content continues apace, the ability to deliver
this across parts of the Internet remain a particular challenge. Those on the fringe
of the network- such as users of a Community Network, are often unable to reliably access streamed media simply because of the resource limitations and external
network capacity that is available to them. Technologies such as P2P should, in
theory, be able to help; the concept of P2P would mean that users requesting
media items could have them provided by other users in the same community without relying on external network access and hence reducing the costs of distribution. The fundamental design of P2P allows clients that view video streaming
to upload the same video to other clients at the same time. Thus, distributing
video streaming with P2P can reduce the cost of having media servers in place.
This also enables clients to actively contribute their resources such as storage space
and bandwidth to the overall network. Allowing clients to re-distribute video to
other clients makes the whole system scalable to support a large number of clients.
In addition, if the existing download source fails or on when a nearer source is
located, clients are able to change to a different source of video content.
Despite the numerous benefits of P2P, there are some key challenges. Firstly,
the concept of locality is still not common in P2P deployments. Most P2P clients
4
1.5. THESIS AIMS AND CONTRIBUTIONS
select remote peers without consideration of the underlying network, which increases the amount of cross-over traffic among different countries. Based on a
research work published in 2012 [5], they claim that there is still no work on the
locality in live video streaming over P2P networks. Moreover, when considering
P2P within CNs for live streaming the problem is exacerbated. Such bandwidth
restrictions reduce the probability of successful connections of local clients to other
clients that are located outside of the local network. Frequent communication with
the remote clients challenges the sustainability of the network and creates a network bottleneck at the backhaul connection. A simple localisation however does
not well performed if peer has low capability as this limits number of successful
connection. Thus, instead simple localisation, dynamic locality is introduces in
order to counter a network with low peer capability.
Whilst the use of localisation can help there is a secondary issue that relates to
the 'liveness' of streamed content; that happen when requested live contents are
not available. Apart from localisation, an efficient mechanism is needed in order
to ensure that the required live contents are available at the localised clients.
The combination of localisation and the mechanism needs to balance the aims
of bandwidth saving and the requirement of preserving the 'liveness' of streamed
content.
1.5
Thesis Aims and Contributions
This thesis is concerned with the enhancements necessary for P2P systems to be
capable of delivering live streaming within Community Network environments that
have limited external connectivity. The thesis proposes a new dynamic localisation
scheme and a mechanism to preserve the playback of live content whilst balancing
the bandwidth requirements.
The specific aims and contributions of the thesis are as follows:
• To study the properties of P2P systems and the impact of loca-
5
1.5. THESIS AIMS AND CONTRIBUTIONS
lisation when delivering live streams. Existing studies have reported
significant findings which show the research gap in providing efficient streaming delivery. These findings highlight a need to provide better peer selection
algorithms, and also the introduction of localisation to reduce remote communication and bandwidth costs. Experiments were carried out through a
series of simulations and an implementation in order to study the sharing behaviour among peers in P2P (presented in the Appendix). Discussion on the
issues of the standard P2P and P2P streaming are also reported in Chapter
3.
• To design a framework for efficient localisation within P2P technologies.
An optimised localisation mechanism encourages peers to favour local clients
and at the same time, preserves the live playback. In live streaming, content
is lively produce at the server and thus efficient mechanism must available in
order to ensure live content availability and accessibility. With localisation
in place, some pre-cautionary actions need to be taken to ensure required live
content is available at the local clients. Otherwise, the live content cannot be
downloaded and peers are unable to continue to play the streaming content.
By having localisation that ideally favours the local clients, a mechanism in
piece selection has to ensure live contents are available in the local clients.
Thus, localisation needs to balance the aims - localising the clients and also
in providing the required live content locally in order to preserve the live
playback.
• To engineer the optimised localisation framework onto CNs. The
design of the framework is built in a python testing framework, which run
P2P live streaming within the CNs network. The localisation framework
is targeting to select a sufficient amount of local clients and preserve the
live playback by ensuring the live contents are available at the clients. The
6