Zen of Cloud Learning Cloud Computing by Examples on Microsoft Azure pdf pdf
types and technologies with different levels of difficulties.
Zen of Cloud Learning Cloud Computing by Examples on Microsoft Azure
Ze n o f C
- lo u d
Zen of Cloud
Learning Cloud Computing by Examples
on Microsoft Azure
Zen of Cloud
Learning Cloud Computing by Examples
on Microsoft Azure
Haishi Bai CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2015 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Version Date: 20140624 International Standard Book Number-13: 978-1-4822-1581-6 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid-
ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright
holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may
rectify in any future reprint.Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti-
lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy-
ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site atContents
Foreword ........................................................................................................................... xiii
SeCtion i CLoUD CoMPUtinG FUnDAMentALS
1 Overview of Cloud Computing ....................................................................................3
1.1 History .................................................................................................................... 3
1.2 Essence of Cloud Computing .................................................................................. 5
1.2.1 Elasticity ..................................................................................................... 6
1.2.1.1 On and Off Mode ...................................................................... 7
1.2.1.2 Rapid-Growth Mode .................................................................. 8
1.2.2 Availability ................................................................................................. 8
1.2.2.1 Fault Domain ............................................................................. 9
1.2.2.2 Update Domain .........................................................................10
1.2.3 Scalability ..................................................................................................10
1.3 Microsoft Azure Overview .....................................................................................11
1.3.1 IaaS (Infrastructure as a Service) ...............................................................11
1.3.2 PaaS (Platform as a Service) ...................................................................... 12
1.3.3 SaaS (Software as a Service) .......................................................................13
1.3.4 Cost Calculation ........................................................................................15
1.4 Preparing the Development Environment for Microsoft Azure...............................16
1.4.1 Subscribe to Microsoft Azure .....................................................................16
1.4.2 Install Software Development Kit .............................................................16
1.5 Introduction of Microsoft Azure Management Portal .............................................16
1.5.1 Sign In .......................................................................................................17
1.5.2 Page Layout ...............................................................................................17
1.6 Summary ................................................................................................................19
2 Building Websites on the Cloud .................................................................................21
2.1 Microsoft Azure Websites .......................................................................................21
2.2 Website Deployment and Upgrade .........................................................................25
2.3 Integration with Source Control Systems............................................................... 32
2.4 Scaling of Websites ................................................................................................ 37
vi ◾ Contents
2.4.1 Vertical Scaling ......................................................................................... 37
2.4.2 Horizontal Scaling .................................................................................... 39
2.4.3 Autoscaling ............................................................................................... 39
2.5 Migrating Existing ASP.NET Websites ................................................................. 40
2.5.1 Azure Websites Runtime Environment ..................................................... 40
2.5.2 Data Storage ..............................................................................................41
2.5.3 Session States .............................................................................................41
2.6 Website Gallery ......................................................................................................41
2.7 Website Configuration........................................................................................... 42
2.8 Website Diagnostics and Monitoring ..................................................................... 49
2.8.1 Website Diagnostics .................................................................................. 49
2.8.2 Website Monitoring ...................................................................................52
2.8.3 Custom Domain Names ........................................................................... 54
2.9 Summary ................................................................................................................55
3 Cloud Service Fundamentals ......................................................................................57
3.1 Microsoft Azure Cloud Services .............................................................................57
3.2 Cloud Services and Roles ........................................................................................61
3.2.1 Role .......................................................................................................... 63
3.2.2 Cloud Service ........................................................................................... 63
3.3 Basic Steps of Cloud Service Deployment .............................................................. 64
3.4 Cloud Service Deployments and Upgrades ............................................................ 69
3.4.1 Incremental Updates (Update Domain Walk) .......................................... 70
3.4.2 Simultaneous Updates .............................................................................. 70
3.4.3 Multiple Deployment Environments ........................................................ 70
3.5 Instances and Load Balancing ................................................................................74
3.5.1 Instances ................................................................................................... 75
3.5.2 Load Balancing ..........................................................................................76
3.6 Configuration File and Definition File .................................................................. 79
3.6.1 Cloud Service Definition File (.csdef) ....................................................... 80
3.6.2 Cloud Service Configuration File (.cscfg) ..................................................81
3.7 Summary ............................................................................................................... 84
4 Advanced Cloud Service .............................................................................................85
4.1 Endpoint Types ......................................................................................................85
4.1.1 Input Endpoint ..........................................................................................85
4.1.2 Internal Endpoint ......................................................................................85
4.1.3 InstanceInput Endpoint ............................................................................ 86
4.2 Worker Role .......................................................................................................... 88
4.2.1 Worker Role Application Scenarios ........................................................... 90
4.3 Inter-Role Communications .................................................................................. 96
4.3.1 Options for Inter-Role Communication ................................................... 96
4.4 Role Lifecycle ...................................................................................................... 100
4.4.1 Process of Deploying and Launching a Role Instance ............................. 100
4.4.2 Role Instance Statuses .............................................................................101
4.5 Startup Tasks ........................................................................................................102
Contents ◾ vii
4.5.1 Defining Startup Tasks ............................................................................102
4.5.2 Startup Task Properties ...........................................................................103
4.6 Diagnostics and Debug ........................................................................................109
4.6.1 Debugging Locally ..................................................................................109
4.6.2 Microsoft Azure Diagnostics ...................................................................109
4.6.3 IntelliTrace ..............................................................................................114
4.6.4 Monitoring Cloud Service ....................................................................... 119
4.7 Developer Community ........................................................................................ 123
4.8 Summary ..............................................................................................................125
5 Data Storage: Relational Database ...........................................................................127
5.1 Microsoft Azure Data Storage Solutions .............................................................. 127
5.2 SQL Database Overview ......................................................................................129
5.2.1 Differences between an SQL Database and an SQL Server ......................129
5.3 SQL Database Management and Optimization ....................................................139
5.3.1 SQL Server Management Studio .............................................................139
5.3.2 Microsoft SQL Server Data Tools ............................................................144
5.3.3 Dynamic Management Views ..................................................................145
5.3.4 Query Optimization ................................................................................146
5.4 Data Sync and Migration .....................................................................................149
5.4.1 Data-Tier Application ..............................................................................149
5.4.2 Data Sync ................................................................................................152
5.5 Periodically Backup Your SQL Databases .............................................................157
5.6 Use MySQL Database ..........................................................................................159
5.6.1 Microsoft Azure Store ..............................................................................159
5.6.2 Purchasing MySQL Service .....................................................................160
5.6.3 Other Means to Run MySQL ..................................................................161
5.7 Summary ..............................................................................................................161
6 Data Storage: Storage Services .................................................................................163
6.1 Local Storage ........................................................................................................163
6.2 Overview of Microsoft Azure Storage Services .....................................................164
6.2.1 Microsoft Azure Storage Account ............................................................165
6.2.2 Provisioning a Windows Storage Account ...............................................167
6.2.3 Storage Account Access Keys ...................................................................168
6.3 Using BLOB Storage ............................................................................................170
6.3.1 BLOB Storage Overview .........................................................................170
6.3.2 Block BLOB and Page BLOB ..................................................................184
6.3.3 ETag and Snapshots .................................................................................187
6.3.4 REST API ...............................................................................................187
6.3.5 Shared Access Signature and Stored Access Policies .................................188
6.3.6 BLOB Update, Copy, and Lease ..............................................................190
6.3.7 Error Handling ........................................................................................190
6.4 Using Table Storage ..............................................................................................190
6.4.1 Table Storage Overview ...........................................................................190
6.4.2 Optimizing Data Partition ......................................................................191
viii ◾ Contents
6.4.3 Query Table Data ................................................................................... 204
6.4.4 Other Operations ................................................................................... 204
6.4.5 Batch Operations .................................................................................... 205
6.4.6 Dynamic Table Entities .......................................................................... 205
6.4.7 Shared Access Signatures ........................................................................ 206
6.5 Use Queue Storage .............................................................................................. 206
6.5.1 Queue Storage Overview ........................................................................ 206
6.5.2 Programmatically Operate Queues ......................................................... 207
6.6 Monitor Storage Accounts ................................................................................... 209
6.6.1 Configure Storage Service Monitoring.....................................................210
6.6.2 Cost of Service Monitoring ......................................................................211
6.7 Summary ..............................................................................................................211
7 Virtual Machines and Virtual Networks ..................................................................213
7.1 Microsoft Azure IaaS ............................................................................................213
7.2 Disk Images and Virtual Disks ............................................................................ 220
7.3 Virtual Machine Communications ...................................................................... 228
7.3.1 Virtual Machine Endpoints .................................................................... 228
7.3.2 Virtual Machines under the Same Cloud Service ................................... 230
7.4 Virtual Networks ................................................................................................. 234
7.4.1 Virtual Networks Overview ....................................................................235
7.4.2 Point-to-Site Virtual Network ................................................................ 237
7.4.3 Site-to-Site Virtual Network ................................................................... 243
7.4.4 ExpressRoute .......................................................................................... 243
7.5 Summary ............................................................................................................. 243
SeCtion ii CLoUD SoLUtionS
8 Cloud Solution Architecture ....................................................................................247
8.1 Client/Server ........................................................................................................247
8.1.1 Characteristics of Client/Server Architecture ...........................................247
8.1.1.1 Benefits ....................................................................................247
8.1.1.2 Shortcomings ...........................................................................249
8.1.2 Client/Server Architecture on Cloud .......................................................249
8.1.3 Multitenant System Design .....................................................................251
8.1.4 Migrating Client/Server Systems to Cloud ..............................................253
8.1.5 Client/Server Systems on Microsoft Azure ...............................................253
8.1.6 Mobile Clients ........................................................................................ 254
8.2 Browser/Server ..................................................................................................... 254
8.2.1 Characteristics of Browser/Server Architecture ........................................255
8.2.2 Browser/Server Architecture on Cloud.....................................................256
8.2.3 Difficulties of Adapting an Existing Single-Tenant Browser/Server Application for Multitenancy.................................................................. 264
8.2.4 Host Single-Tenant Systems on Microsoft Azure for Multiple Tenants ......267 8.3 n-Tiered Architecture .......................................................................................... 269
8.3.1 Characteristics of n-Tiered Architecture ................................................. 269
Contents ◾ ix 8.3.2 n-Tier, MVC, and MVVM .....................................................................270
8.3.3 Microsoft Azure Service Bus Queue ....................................................... 273
8.3.4 Implementing n-Tiered Services on Microsoft Azure .............................. 277
8.4 Distributed System .............................................................................................. 284
8.4.1 Message-Based Connections ................................................................... 287
8.4.2 Relayed Connections ...............................................................................291
8.5 Summary ............................................................................................................. 299
9 High-Availability Design .........................................................................................301
9.1 Availability ...........................................................................................................301
9.2 High-Availability Techniques .............................................................................. 302
9.2.1 Redundancy ........................................................................................... 303
9.2.2 Load Balancing ....................................................................................... 303
9.2.3 Failover ................................................................................................... 303
9.3 Load Balancing and Health Probe ....................................................................... 308
9.4 Competing Consumers .........................................................................................310
9.4.1 Loose Coupling .......................................................................................310
9.4.2 Dynamic Load Balancing ........................................................................311
9.4.3 Dynamic Scaling .....................................................................................311
9.4.4 Failover ....................................................................................................311
9.5 Case Study: High-Availability Service Bus Entities ..............................................312
9.5.1 Background ............................................................................................. 315
9.5.2 Segmented Message Pipelines ..................................................................316
9.5.3 Paired Namespaces ..................................................................................317
9.5.4 Conclusion ..............................................................................................317
9.6 Summary ..............................................................................................................317
10 High-Reliability Design ...........................................................................................319
10.1 Reliability, Availability, and Maintainability ........................................................319
10.1.1 Reliability ................................................................................................319
10.1.2 Maintainability ....................................................................................... 320
10.1.3 Relationships between Availability, Reliability, and Maintainability....... 320
10.2 Embracing Failures ...............................................................................................321
10.2.1 Failures in Operation ...............................................................................321
10.2.2 Failures in State Management..................................................................321
10.2.3 Failures in System Design and Implementation ...................................... 322
10.3 Transient Errors ................................................................................................... 322
10.3.1 Transient Fault Handling Application Block .......................................... 323
10.4 Design for Reliability........................................................................................... 326
10.4.1 Single Point of Failure .............................................................................327
10.4.2 Writing Reliable Code ............................................................................ 328
10.5 Summary ..............................................................................................................331
11 High-Performance Design ........................................................................................333
11.1 Microsoft Azure In-Role Cache ............................................................................333
11.1.1 Overview ................................................................................................ 334
11.1.2 Deployment Options .............................................................................. 334
x ◾ Contents
13.4 Summary ............................................................................................................. 396
12.3.2 Microsoft Azure Active Directory Premium ............................................372
12.4 Summary ..............................................................................................................373
SeCtion iii DeViCeS AnD CLoUD
13 Mobile Service ..........................................................................................................377
13.1 Mobile Service Overview ..................................................................................... 377
13.2 Push Notifications ............................................................................................... 386
13.2.1 Push Notification Overview ................................................................... 386
13.3 Scheduler and API ................................................................................................393
14 Internet of Things .....................................................................................................397
12.3 Microsoft Azure AD New Features ......................................................................372
14.1 IoT Overview ...................................................................................................... 397
14.1.1 Radio Frequency Identification ............................................................... 398
14.1.2 Artificial Intelligence Equipment ............................................................ 398
14.1.3 Wearable Devices ................................................................................... 398
14.1.4 Wireless Sensor Network ........................................................................ 399
14.2 Devices and Cloud .............................................................................................. 399
14.2.1 Importance of Devices for Cloud ............................................................ 399
14.2.2 Importance of Cloud for Devices ............................................................ 400
12.3.1 Azure Authentication Library ..................................................................372
12.2.2 Graph API ...............................................................................................367
11.1.3 Cache Features ........................................................................................338
11.2.5 Future of Azure Cache ............................................................................ 348
11.1.4 Concurrency Modes ................................................................................339
11.1.5 Local Cache ............................................................................................ 341
11.1.6 Session State ........................................................................................... 341
11.2 Microsoft Azure Cache Service .............................................................................345
11.2.1 Overview ................................................................................................ 346
11.2.2 Cache Service versus In-Role Cache ....................................................... 346
11.2.3 Managing Cache Clusters on Microsoft Azure Management Portal ....... 346
11.2.4 Memcache Support ................................................................................. 347
11.3 Microsoft Azure CDN ......................................................................................... 348
12.2.1 Managing Microsoft Azure Tenants and Users ........................................357
11.4 Asynchronous Operations and Parallel Operations .............................................. 349
11.5 Summary ..............................................................................................................350
12 Claim-Based Architecture ........................................................................................351
12.1 Claim-Based Authentication and Authorization ...................................................352
12.1.1 Basic Authentication and Authorization Process ......................................353
12.1.2 Authentication and WIF .........................................................................354
12.1.3 Authentication Broker .............................................................................354
12.2 Introduction to Microsoft Azure AD ....................................................................356
14.3 Challenges of IoT .................................................................................................401 14.4 .NET Micro Framework ...................................................................................... 402
Contents ◾ xi 14.4.1 .NET Micro Framework Overview......................................................... 402 14.4.2 .NET Gadgeteer Overview ..................................................................... 405
14.4.3 Device Integration Sample Scenario ....................................................... 409
14.5 Summary ..............................................................................................................416
SeCtion iV SYSteM inteGRAtion AnD PRoJeCt MAnAGeMent
15 Message-Based System Integration ..........................................................................419
15.1 System Integration ................................................................................................419
15.1.1 Integration by Data ................................................................................ 420
15.1.2 Shared Business Functions ...................................................................... 420
15.1.3 Enterprise Service Bus ............................................................................ 420
15.2 Message-Based System Integration ...................................................................... 422
15.2.1 Content-Based Routing .......................................................................... 422
15.2.2 Priority Queue ........................................................................................ 423
15.2.3 Request/Response ................................................................................... 426
15.2.4 Dead Letter Queue ................................................................................. 427
15.2.5 Event-Driven Consumer ......................................................................... 430
15.3 Advanced Message Queuing Protocol .................................................................. 433
15.3.1 AMQP Overview ................................................................................... 434
15.3.2 AMQP Adoption .................................................................................... 436
15.4 Advantages of Message-Based Integration ........................................................... 438
15.4.1 Loose Coupling ...................................................................................... 438
15.4.2 Dynamic Extension ................................................................................ 441
15.4.3 Asynchronous Communication .............................................................. 441
15.4.4 Centralized Management ....................................................................... 441
15.5 Summary ............................................................................................................. 444
16 Source Control and Tests with Visual Studio Online.............................................. 445
16.1 Create a Visual Studio Online Account ............................................................... 446
16.2 Source Control with Visual Studio Online .......................................................... 446
16.3 Create and Use Unit Tests ....................................................................................452
16.4 Create and Use Load Tests ...................................................................................459
16.5 Summary ..............................................................................................................465
17 Scripting and Automation ........................................................................................467
17.1 Microsoft Azure PowerShell Cmdlets .................................................................. 467
17.1.1 Preparing a Microsoft Azure PowerShell Cmdlets Environment ............. 467
17.1.2 Managing Virtual Machines ................................................................... 469
17.1.3 Managing Cloud Services ........................................................................472
17.1.4 Managing Microsoft Azure Websites .......................................................472
17.1.5 Other Cmdlets ........................................................................................472
17.2 Microsoft Azure Cross-Platform Command Line Tools........................................472
17.2.1 Installing the Command Line Tools ........................................................474
17.2.2 Getting Started with the Command Line Tools ......................................476
17.3 Microsoft Azure Management API ...................................................................... 477
17.4 Summary ............................................................................................................. 480
xii ◾ Contents
18 Azure and DevOps ...................................................................................................481
18.1 DevOps Overview ................................................................................................481
18.1.1 Everything Is Code ..................................................................................481
18.1.2 Everyone Is a Developer .......................................................................... 482
18.1.3 Every Day Is Release Day ....................................................................... 482
18.2 VM Agent and VM Extensions ........................................................................... 483
18.2.1 VM Agent .............................................................................................. 483
18.2.2 VM Extensions ....................................................................................... 483
18.2.3 Custom Script Extension ........................................................................ 484
18.2.4 DSC, Puppet, and Chef .......................................................................... 485
18.3 New Portal .......................................................................................................... 486
18.4 Zen of Cloud ....................................................................................................... 487
Bibliography ......................................................................................................................489
Foreword
This book by Haishi Bai provides a detailed introduction to cloud computing, in general, and to Microsoft Azure, in particular.
Haishi Bai has had an extensive and successful career in information systems, which brought him from the Northeast of China to Beijing and then finally to Silicon Valley and to the Microsoft Head Office in Redmond, Washington as a Microsoft Azure evangelist. His experience in the rise and fall of e-business, boom of social networks, and rise of cloud computing provides him a deep understanding of the pros and cons in cloud computing.
This book explains the various concepts of Azure in a logical and clear manner. It is divided into four sections: cloud computing fundamentals, cloud solutions, devices and cloud, and sys- tem integration and project management. Beginners can use this book as a guide in their jour- ney through cloud computing. Experienced cloud developers can benefit from it by studying specific scenarios. The book consists of 69 complete end-to-end examples that provide step-by- step guidance on implementing typical cloud-based scenarios. The examples cover a wide range of application types and technologies with different levels of difficulties. The book also provides practical knowledge, tips and tricks that you can apply to your own work.
This book should provide invaluable help to IT managers who want to stay up to date, develop- ers who want to implement applications using Microsoft Azure, as well as system engineers who are looking to gain in-depth knowledge on cloud computing.
Enjoy the reading as I did and looking forward to your great contributions to this new world!
Pierre Masai CIO of Toyota Motor Europe
CLoUD CoMPUtinG i FUnDAMentALS
Cloud computing, as its name suggests, is to leverage cloud for computing workloads. These work- loads are not new—we have been running applications and services on our workstations and data centers for tens of years. Then why do we need cloud? What benefits are we getting by using it? What are the new scenarios it enables? In this first section of the book, we first look back at the his- tory of cloud computing, and then discuss some unique and exciting capabilities cloud computing brings us. We also study the basics of getting your workloads on cloud using Microsoft developer tools and services. From a developer’s perspective, you can view cloud as a huge resource pool where you can pull out resources to support your application needs, and return the resources when you are done with them. This resource usage model is the foundation of key cloud characteristics such as elasticity, availability, and agility.
overview of Cloud Computing
1.1 History
In August 1962, Joseph Carl Robnett Licklider published the famous paper “On-Line Man- Computer Communication,” one of the first conceptions of the future Internet. In the paper, he elaborated on how man would interact with the computer to accomplish various tasks. Not only did he mention the widespread keyboard drag-and-drop operations but also predicted the future of man–computer communication via natural languages. However, the main idea in this paper is how to share the reliable storage and the superb computational capabilities of computers among users in order to accomplish complicated tasks with an online collaboration. The conceptions of resource sharing, dynamic interaction, and remote collaboration laid the foundation for the Internet theory and depicted the preliminary blueprint for cloud computing.
In the past 50 years, although software architecture has evolved through several different stages, from Mainframe to Client–Server, to Browser–Server, to Distributed System, and finally to Cloud Computing (Figure 1.1), providing value-added services with optimized resource utiliza- tion remains the theme of software system design.
In the time of Licklider, people concentrated on optimizing the utilization of limited resources on the host systems because terminals had almost no processing power and all computing tasks were carried out on the hosts. With the emerging wide spread use of personal computers, stor- age capacities and processing power of client machines have been constantly increasing. As a result, more computing and storage requirements can be satisfied directly on client machines. Distributed systems pushed this idea to the extreme and eliminated the need of centralized servers. Many phenomenal distributed systems and applications have emerged and prospered. However, on the flip side, development, maintenance, and management of distributed systems have also shown unprecedented complicity. Then, as browsers and the Internet mature, computing and storage are again pushed to the server side. Browsers replace desktop programs and become the mainstream user front end. Is this a return of mainframe mode? In order to answer this question, we cannot miss mentioning Salesforce.
In 1999, 37-year-old Marc Benioff resigned from his senior vice presidential position at Oracle to found Salesforce. He put forward, for the first time in history, the concept of Software- as-a-Service (SaaS). The SaaS mode shifted the paradigm of software applications in enterprises.
4 ◾ Zen of Cloud Distributed Devices systems and cloud computing Client/ server
Client evolution Browser/ server
Mainframe
Server evolution