release2 issue12 Ebook free download pdf pdf
Release 2.0
12 Issue 2.0.12, April 2009 http://r2.oreilly.com
Ben Lorica, from Mobile Banks in Developing Countries, page 02
“The mobile phone is fast becoming the ‘PC of the developing
world.’ For the vast majority of the world’s poor the mobile phone
has made it convenient to send and receive money, pay bills, and
enjoy other services once limited to people with bank and debit
card accounts.”Release 2.0 Issue 12, April 2009
Contents Published six times a year by O’Reilly Media, Inc., 1005 Gravenstein Highway North,
01: What’s on our Radar Sebastopol, CA 95472 http://r2.oreilly.com
02: Mobiles and Money in the Developing World This newsletter covers the world By Ben Lorica of information technology and the Internet — and the business and societal issues they raise.
11: Our Search for Intelligent Software Agents By Ben Lorica executive editor Tim O’Reilly
tim@oreilly.com 15: Analytics in Action: A Conversation with LinkedIn’s
Analytic Team editor and publisher Sara Winge By Ben Lorica & Roger Magoulas sara@oreilly.com art director
19: Advanced Meter Infrastructure (AMI)—Smart Grid Mark Paglietti and Smart Meters markp@oreilly.com
By Roger Magoulas copy editor Karen Shaner
23: Acknowledgments contributing writers Brady Forrest Ben Lorica
24: Calendar Roger Magoulas Sarah Milstein Nathan Torkington © 2009, O’Reilly Media, Inc.
All rights reserved. No material in this publication may be reproduced without prior written permission; however, we gladly arrange for reprints, bulk orders, or site licenses. Individual subscriptions cost $495 per year. 90257 subscription information Release 2.0 PO Box 17046 North Hollywood, CA 91615-9588 http://r2service.oreilly.com customer service 1.800.889.8969 1.707.827.7019 r2@oreilly.com
Sara Winge is VP of the Radar group at O’Reilly.
What’s on our Radar
The “Radar” analogy is a pretty good fit for what we aim to do at O’Reilly—scan for faint signals about new disruptive technologies, and amplify those that Radar group, explore several blips on the Radar that herald interesting and important changes in our world.
First, Ben looks at how mobile phone adoption is driving economic advancement among the millions of “unbanked” people in developing coun- tries where mobiles are the primary Internet access device. In “Mobiles and Money in the Developing World,” he concludes that the new banking services developed for this huge new market not only broaden opportunities for first- time customers to enter the economic mainstream, but they also point to innovative approaches that may help revive the global financial system.
In “Our Search for Intelligent Software Agents,” we evaluate a range of tech- nologies that can enrich customer experience and enhance customer service initiatives, including recommendation engines, rule-based systems, machine- learning tools, and virtual worlds. While we found that many of these promising new technologies were not quite ready for prime time, we did tease out useful principles and best practices for implementing them now and in the future.
We continue to track Big Data, the theme of our last issue. In “Analytics in Action,” we ask the data analytics team at LinkedIn about collecting, analyzing, and getting new product ideas from the data generated by their more than 40 million users. Their responses offer insight and advice for anyone who believes, as we do, that data is a core strategic asset in our Web 2.0 world.
Energy conservation technology and sensors converge in the Advanced Meter Infrastructure (AMI), a system that has the potential to change the way we buy and sell power. Roger provides a short survey of how the AMI works and what we can expect as it gets implemented. Expect more coverage of green technology in future issues.
What’s on your Radar? Send your ideas and requests for topics we should cover to r2@oreilly.com, and keep an eye on the Radar blog at radar.oreilly.com n n for more on faint signals that are growing stronger. Release 2.0.12 April 2009 Mobiles and Money in the Developing World Ben Lorica Mobiles and Money in the Developing World by Ben Lorica
Ben Lorica is a Senior Analyst in O’Reilly’s Research group.
Close to 60% of all mobile phone subscribers reside in the developing world. With the fastest growth in new mobile phone subscribers coming from those countries, the mobile phone is fast becoming the “PC of the developing world”.
In developing countries, the mobile network infrastructure is superior to exist- ing fixed-line networks, and users are going straight to mobile to get online. The numbers are impressive: at the end of 2007 there were 280 million mobile phone subscribers in Africa; India had just over 360 million at the end of January 2009. Vodafone-controlled mobile provider Safaricom, is the most prof- itable company in East and Central Africa. It’s no surprise that companies and cations and business models—they serve the majority of users, and they’re starting from scratch, not building on top of legacy products.
And far from being localized, the mobile applications and enterprises pio- neered in developing countries will be mimicked elsewhere. As American tech- nologist and frequent traveler to Africa, Erik Hersman, is fond of saying: “If it works in Africa, it will work anywhere.”
Only about a third of the three to four billion mobile phone subscribers worldwide have a banking relationship with a financial services company. The race to reach the large numbers of “unbanked” through the mobile platform is well underway—the industry buzzword is “financial inclusion.” An important goal within developing countries, financial inclusion has in recent years led to thriving mobile banks that aren’t (yet) well-known in the West. For the vast majority of the world’s poor, the mobile phone has made it convenient to send and receive money, pay bills, and enjoy other services once limited to people
Something as simple as a bank account that allows direct deposit is critical to workers seeking employment with established companies.
with bank and debit card accounts. Something as simple as a bank account that allows direct deposit is critical to workers seeking employment with established companies. Large employers reluctant to disburse salaries in cash can deposit mobile airtime or currency directly into the virtual accounts of mobile bank users. A case in point is a recent startup founded by American researcher Nathan Eagle: txteagle matches multinational companies with African workers who have mobile phone access. txteagle workers perform simple tasks and get paid with airtime or deposits into a mobile wallet. In recognition of the poten- tial impact of mobile banking, the Bill and Melinda Gates foundation recently announced $12.5 million in grants for market and regulatory research projects that help expand mobile banking services. The long-term goal of the founda- tion is to encourage savings by providing the unbanked simple mechanisms to start saving. Savings lead to risk protection (“emergency fund”), asset accumu- lation, and funds for education and other basic needs.
Mobile phone users in developing countries
Mobile banking is loosely defined as conducting formal financial transactions with a mobile phone. It’s an extremely competitive space pitting mobile opera- tors against each other and financial services companies. It’s useful to distin- guish between mobile banking—mobile phone access to existing bank customers—and mobile banks—financial institutions that arose with mobile phones. Before describing mobile banks in developing countries, a few factors about mobile phone users in those countries are worth considering. As noted earlier, large numbers of users do not have a bank account and use “pay as you
Release 2.0.12 April 2009 Mobiles and Money in the Developing World Ben Lorica
go” plans to prepay for mobile minutes. Some estimates place the number of prepaid mobile subscribers at more than half of all subscribers in the world. While they pose less of a credit risk to operators, prepaid subscribers also tend to spend less on high-end mobile (data) services. Because of its low cost com- pared to phone minutes, text messaging (particularly SMS) is extremely popular and many mobile banking features introduced in developing countries use SMS to verify transactions. Hannes van Rensburg, CEO of mobile banking technol- ogy provider Fundamo, believes that its instant feedback mechanism and wide- spread use make SMS an ideal entry point for mobile banking. Finally, the econ- overseas and domestic workers’ transfer of funds to relatives in their home countries). Estimates of the share of workers’ remittance range from a few per- centage points to as much as 10% of the nominal GDP of some developing countries, with an increasing portion flowing through mobile banks. As of mid 2008, researchers from the TowerGroup estimated total global mobile-based, cross-border remittance at $320 billion annually. With these factors in mind, we grouped the services offered by mobile banks into a few categories (see Sidebar: Popular Mobile Banking Services).
Popular Mobile Banking Services Transferring mobile phone air time
The goal of mobile operators is to make it as convenient as possible for prepaid subscribers to “recharge” or “top-up” the minutes
on their phone. Users who want to top-up their own phone typically either send an SMS message or “press a button”. In develop-
ing countries, sending mobile phone minutes to friends and family is also popular. Smart Telecom of the Philippines letsSmartMoney account holders send mobile air minutes to other Smart mobile phone subscribers. Available at designated outlets,
SmartMoney is a reloadable payment card accessible with a Smart mobile phone that can be used at any outlet that accepts MasterCard. Working with mobile operators in multiple countries, Singapore-based Transfer To takes it a step further by inter-connecting the prepaid services of carriers. Transfer To works with mobile operators in developing countries to tailor services to
particular local markets and specifically targets expatriate workers from developing countries who wish to reload minutes on phones back in their home country, a process known as cross-border top-ups.
Financial Inclusion: For the unbanked, the ability to send minutes back and forth turns the mobile phone into an informal
savings account and fund transfer service.Money transfers using mobile phones
The most famous money transfer service is operated by Vodafone subsidiary Safaricom in Kenya: the M-PESA service is installed on SIM cards and is available to all Safaricom users. To enable an M-PESA account, a customer gets credit towards his virtual
account by paying cash to a registered M-PESA agent. To transfer funds ranging from a few cents to a few hundred dollars, cus- tomers access a menu-driven application, built into their SIM card, that allows them to send funds to other mobile phone users. If the receiver is another M-PESA user, the funds get added to his account. Otherwise, the virtual funds can be cashed in with any registered agent using a secret code and an I.D.
At the end of 2008, M-PESA had over 5,000 registered agents in Kenya and 5 million registered users. (In contrast, four major
local banks in Kenya had 750 branches and 3 million accounts combined.) In October 2008, over a hundred million dollars was transferred through M-PESA.Other examples target remittances from expatriate workers. The leading mobile operators in the Philippines have money- transfer services designed to appeal to workers who remit money on a regular basis. Globe Telecom’s GCASH charges a nominal fee to the expatriate money sender, but charges no fee to recipients in the Philippines. GCASH senders go to registered agents and apply for GCASH remit transactions. GCASH recipients are notified via SMS and present an I.D. and a reference number at registered GCASH outlets. Globe Telecom has registered outlets in countries with large numbers of overseas Filipino workers. About one hundred million dollars flows through the GCASH system daily.
Financial Inclusion: Being able to cheaply send, receive, and retain money turns the mobile phone into a zero-interest sav- ings account, although it’s important to note that many services have strict limits on maximum account size (e.g., M-PESA, GCASH, SmartMoney, WIZZIT all have limits). Popular services like M-PESA expand access to money-transfer services by having registered agents in small villages. With traditional money transfer services (and banks) limited to larger towns, users have flocked to M-PESA.
Mobile Wallets and Payment Systems Using technology called Near Field Communication, many European and East Asian consumers can now pay for parking services and purchase items by “swiping” their mobile phones across a reader. Equivalent payment systems in developing countries tend to be less sophisticated, with some systems involving nothing more than SMS notification and prepaid debit cards or minutes. A recent example involving one of Nathan Eagle’s students further illustrates how quickly mobile payment systems emerge: 30% of Rwandans now pay for their electricity using a new mobile phone payment system that mimics a system used to sell mobile minutes via scratch cards. In most developing countries, M-PESA and other popular money transfer services also function as pay- ment systems—buyers simply send M-PESA to sellers through their mobile phones. Release 2.0.12 April 2009 Mobiles and Money in the Developing World Ben Lorica Regardless of what technology is used, electronic payment systems give the unbanked access to financial transactions nor-
mally reserved for bank/debit cardholders. In the Philippines, Smart Telecom’s SmartMoney can be used anywhere MasterCard is
accepted. SmartMoney users purchase items by sending an SMS message containing the seller’s merchant number and payment amount. A non-trivial portion of the Zambian GNP flows through the Celpay mobile payment system. After activating their mobile wallets by visiting participating banks, users of eTranzact in Nigeria and Ghana conduct cashless purchases either through SMS or a mobile browser.Realizing that payment systems can help cut costs and enhance customer satisfaction, companies in developing countries
are turning to technology providers who specialize in mobile banking and payment systems. Ten-year old South African com-
pany Fundamo is a leading supplier of technology that enables mobile operators to offer financial service products including
mobile wallets and mobile banking. Fundamo has deployed their technology over forty times in locations including Africa, the
Middle East, and Pakistan. The Celpay system in Zambia uses Fundamo’s technology, as does South African mobile operator
MTN, to offer mobile banking services in several African countries. Florida-based Yellowpepper’s platform includes mobile wallets and payment systems aimed at banks, mobile carriers, and companies in Latin America.Financial Inclusion: The unbanked enjoy benefits usually limited to bank/debit cardholders, including direct deposit, auto-
matic bill payment, and cashless purchases. Rather than spend hours in line to pay their bills, consumers pay them instantly.
For small merchants, this means not having to carry large amounts of cash when they go to the nearest bank to deposit their earnings.
Banking and Brokerage Services Mobile phones can provide access to financial information lookups (stocks/bonds, exchange rates) and account and broker-
age services. Hello Money, Barclays Bank’s mobile banking service in India, allows customers to perform a wide range of banking
transactions: check account balances, transfer funds between accounts, list recent transactions, activate and automate bill pay-
ments, and file service requests (e.g., new checks, PIN). In addition, customers can top-up their prepaid mobile phone and pay
specific charges on their credit cards. Enabling transactions through mobile phones can result in substantial cost savings for the
financial institutions, freeing up funds to target the unbanked. ICICI bank in India needs just five employees to manage 250,000
transactions per day on its online trading platform. Cost savings reaped through the use of mobile technology have allowed
ICICI to target previously unbanked customers, resulting in four million new customers in 2008. According to Hannes van
Rensburg, some banks have started agile new business divisions charged with establishing mobile banks. Instead of merely pro-
viding mobile access to existing customers, these are entirely new entities, created to be more accommodating to unbanked
consumers and, because they are “branchless banks,” they are free from some of the business rules that hamper their parent bank.
Financial Inclusion: Financial information lookups need not require an account, but other conventional banking functions
do. On the other hand, the savings accrued by enabling mobile access to bank customers can be used to target the unbanked. Challenges We often hear people in the
Broadly speaking, the main challenges facing mobile banks are interoperability,
mobile banking industry
cultural issues, regulatory environments, and security and privacy. Mobile usage patterns and behavior vary across countries, and the most successful
declare that they shouldn’t
mobile banking initiatives tend to establish a strong foothold in one country or region before expanding to others. Safaricom’s M-PESA was well-established
be subject to banking rules.
in Kenya before they announced plans to roll out services in other countries. To ease their entry into new markets, mobile banking technology providers Fundamo and Yellowpepper work with local partners in specific regions. of the industry. For example, mobile banking in India is subject to stricter con- trols compared to other developing countries. Mobile banking in India is tied to KYC/AML-compliant (see below) bank accounts and credit card accounts. Moreover, since transactions are required to be based in Rupees, overseas remittance services aren’t permitted. In other countries, successful mobile banking services are belatedly subjected to regulatory scrutiny. In late Decem- ber 2008, the Executive Director of the Kenya Bankers Association complained that Safaricom had an unfair advantage in that it behaved like a bank but did not declare itself as one. Around the same period, the Kenyan Finance Minister ordered an audit of M-PESA, which some in the local media attributed to lobby- ists for the banking sector. We often hear people in the mobile banking indus- try declare that they shouldn’t be subject to banking rules “since they don’t award interest on deposits.” What they fail to mention is that banks are closely regulated for another important reason: customers assume that their funds can be withdrawn at a moment’s notice. For this reason, we predict the large amounts deposited in M-PESA and other successful mobile banks will be sub- ject to strict capital controls that ensure adequate reserves. Even when regula- tions governing mobile banks are in place, the enforcement of laws in resource-strapped countries is another matter.
As with other systems that permit money transfers, regulators need assur- ance that mobile banks can’t be readily used for money laundering (among bankers this is referred to as AML or “anti-money laundering” controls). To dis- courage money laundering, banking regulators in each country have specific rules regarding the identity of customers (bankers call this KYC or “know your customer” controls). Critics of mobile banks point out that mobile users who opt to prepay minutes tend to be subjected to fewer identity checks, increasing the possibility that such services will be used for money laundering. In practice, mobile banking services that work closely with bank regulators have KYC con- Release 2.0.12 April 2009 Mobiles and Money in the Developing World Ben Lorica
trols that are comparable to traditional banks. In addition, compared to Internet systems than can easily hide behind proxies, mobile money transfers are easier to audit and trace making mobile banking less attractive for money laundering.
In both mobile and online commerce, security requires the right mix of technology and business processes: when properly designed, mobile banking security and privacy is at least comparable to online banking. Mobile phones receive and send information either through SMS, USSD (“SMS with sessions”), or WAP/GPRS. Mobile banking applications are either installed on the device or SIM card, accessed through a mobile web browser, or based on SMS/USSD. by mobile operators and contains the user menus and security keys used to encrypt network traffic (STK or SIM toolkit). Aside from PINs, some banking services rely on a secure digital ID stored in either the device, the SIM card, a memory card inside the device, or on an external device. Regulations governing security also vary across countries. Among the most stringent are regulations in India that require mobile banking services have “end-to-end application level encryption” (eliminating the possible use of SMS and USSD). Regardless of the security systems in place, users need to be educated about fraudulent practices as popular scams employ old-fashioned social engineering: smishing, the SMS equivalent of email phishing, is a popular method for stealing credentials and assets.
In its simplest form, interoperability means that users of one mobile pay- ment system will be able to send money into the accounts of users of another system. Taken a step further, interoperability would additionally mean that users of a mobile bank can withdraw or remit cash at any ATM or POS-enabled agent. At the moment interoperability is extremely limited: Western Union works with a few mobile money transfer services and there is talk of a few pay- ment systems working towards interoperability. According to members of the GCASH team, “…the current trend is not to allow interconnection with local competitors (other local telcos) as competition tends to be stiff domestically.” There are other reasons why a single framework unifying mobile banking services across the world probably won’t be materializing soon. Hannes van Rensburg cites the absence of a quick and accessible financial clearing system. For international money-transfers, GCASH insiders believe that the key challenge lies in satisfying complex regulatory requirements governing cross- border remittances. Mobile Banks and Financial Reform The success of mobile banking
The potential market is enormous: only about a third of the three billion
in the developing world is
mobile phone subscribers have a banking relationship. Many researchers expect growth to continue at a steady pace for years to come. Juniper research
“…teaching the rest of the
predicts that mobile banking users will grow tenfold from 2007 to 2011. Berg Insight forecasts that by 2014, five to twenty percent of all international money
world how financial services
transfers will be routed through mobile banking services. While we think the
should work.”
global economic crisis may slow down the growth of mobile banks, the trajec- tory and long-term trend is clear: mobile banks will continue to grow steadily in other financial services providers. Popular mobile banking services in the devel- oping world work on simple and cheap handsets, making the cost of entry more manageable. To the unbanked, the convenience and benefits that come with mobile banks are too compelling to ignore. Finally, Hannes van Rensburg believes that, especially in a period of economic crisis, consumers will continue to want easy, real-time access to financial information including current exchange rates, prices, or the balance on their virtual wallets.
Van Rensburg also believes that the success of mobile banking in the devel- oping world is “…teaching the rest of the world how financial services should work.” The global economic crisis has led to numerous calls for reforming the financial services sector in major industrialized countries. The idea that policy makers in the industrialized world should draw inspiration from developing countries may sound audacious, and while financial experts have yet to con- sider it, other thought leaders have. When asked what advice he would give to the CEOs of the world’s largest banks, Harvard University Management Professor Clayton Christensen recently suggested they “…go to the developing world and buy a phone company!” Mobile payment and banking systems have been so disruptive in the developing world, Christensen believes that Western banks would benefit immensely from studying them closely.
At a time when troubled banks are lobbying for minor changes and as little regulation as possible, mobile banks have much to teach our own struggling financial sector. The technology and business processes that power the best mobile payment and savings systems are impressive. Unencumbered by legacy systems and business rules, companies in the developing world have built tech- nology and business systems that leapfrog those in the developed world. While it may be easy to start a small-scale mobile payment system, the most success- ful mobile banks use complex software systems designed to securely handle many more near real-time transactions than traditional banking systems. For Release 2.0.12 April 2009 Mobiles and Money in the Developing World Ben Lorica Recommended Reading & Viewing
financial services companies seeking growth opportunities, mobile banks are
Banking on Mobiles: Why, How, for Whom?
skilled at developing mass-market products for the expanding population of
(from CGAP.org http://bit.ly/tULM2)
mobile subscribers. Large Philippine banks initially viewed GCASH with suspi-
Mobile Banking Blog (http://bit.ly/ogWtb)
cion, but some eventually realized that partnering with mobile banks gave
Reserve Bank of India: Mobile Payment in
them access to popular technologies and to consumers in remote areas of the
India—Operative Guidelines for Banks
country. The third largest bank in the country developed a service with Globe
(http://bit.ly.gXg5L)
Telecom, which allows customers to transfer money between their bank and
Mobile Phone Banking and Low-Income
GCASH accounts. More recently, the Rural Bank Association of the Philippines
Customers: Evidence from South Africa
has partnered with GCASH to introduce services for customers of their network
(from CGAP.org http://bit.ly/WlY6) GCASH and Rural Banking in the Philippines
Mobile operators designed their impressive new financial institutions to
(http://bit.ly/bPyh4, http://bit.ly/ArAk)
accommodate the unbanked. Reviving the global financial system may require
Clayton Christensen’s Advice for Jamie Dimon
a similar emphasis on institutions that cater to the needs of the working class
(http://bit.ly/1zzf92)
majority. From what we’ve seen in the short history of the industry, products
Nathan Eagle, “Crowd-Sourcing on Mobile
designed to appeal to the unbanked can quickly influence the banking indus-
Phones in the Developing World”
try. The success of GCASH coupled with Globe Telecom’s sustained business
(http://bit.ly/18KNcX)
development have convinced Philippine banks that partnerships can benefit both banks and telecom companies. With their control of the device (via SIM card) and the network, mobile operators in the developing world are poised to play a prominent role in reinventing global finance. In the meantime, the mobile banking industry remains committed to expanding access and to the development of new products. A recent example involves mobile banks and microfinance outfits in developing countries who are joining forces to give users access to low-interest consumer loans and mobile credit cards. Savvy operators are also starting to look at usage data: Van Rensburg tells the story of how a mobile banking money remittance service within an African country n n used data mining techniques to further optimize the location of its hubs. Release 2.0.12 April 2009 Our Search for Intelligent Software Agents Ben Lorica Our Search for Intelligent Software Agents by Ben Lorica
The O’Reilly Research team recently looked at how software agents could help improve the experience of online customers and increase the productivity of customer care employees. We took a broad view and examined specific technologies such as recommendation engines, rule-based systems, popular machine-learning tools, and emerging platforms including virtual worlds. After talking to a range of experts, including academics, early-stage startups, and established companies, we came to the conclusion that software agents (and related technologies), while intriguing, are still of limited use in this early stage of development. From the interviews and our experience with many related agents. We’ve listed these principles below and included a few stories of com- panies and projects that exemplify them.
Domain expertise is important: Effective systems tend to combine domain
expertise with clever algorithms. When we looked at customer care systems, those developed in close coordination with frontline employees and users worked best. Training and instructional software developed without consulting pedagogy experts in the specific field covered fared poorly. I’m reminded of a recent story of a financial consultant (working in catastrophe insurance) charged with modeling losses due to hurricanes. Lacking knowledge in basic weather modeling or fluid mechanics, the consultant devised risk models that drew only on techniques from quantitative finance. Using the recent disaster in credit derivatives as a guide, his (hurricane) risk model may work over the short term, but chances are it will eventually lead to large losses.
Limit the scope of the system: Even narrowly focused systems can result in
compute-intensive tasks. In addition, it’s surprisingly difficult to create intelli- gent systems that excel in more than a few well-defined tasks. For example, the recommendation engines available on many web sites generate lists of related products (e-commerce) or content (publications), typically using technology called collaborative filtering, which mines user data to predict that “people who liked x will also like y. New recommendation engines like RichRelevance and Baynote go beyond simple collaborative filtering. They use ensembles or collec- tions of models and classifiers, and make adaptive, personalized adjustments in near real-time. Sites are enlisting these newer systems because they substan- tially increase conversion rates—in other words, their recommendations are hitting the mark. While these new recommendation systems remain narrowly focused on identifying interesting items, they rely on complex software and I.T. systems. RichRelevance has state-of-the-art data centers and is available only as a service (SaaS).
Release 2.0.12 April 2009 Our Search for Intelligent Software Agents Ben Lorica Integrate into workflow and pay attention to the UI: Currently being used
Simply put, many algorithmic
by a third of all U.S. physicians, Epocrates is a medical decision support system
systems improve as more data
available on many handheld platforms. Recognizing that doctors have sporadic computer access because they move around constantly, and that doctor-
becomes available.
patient interaction would be adversely affected if a doctor was staring at a computer screen, Epocrates targeted handheld devices. Epocrates software designers also produce intuitive and easy-to-navigate user interfaces for each of the major handheld platforms.
Don’t overlook rule-based systems and checklists: Simple business rules
Epocrates relies on its staff of MDs and pharmacists to constantly improve its purely rule-based decision support software for physicians. Physicians use Epocrates software to help diagnose and research ailments as well as to research prescription drugs and alternative medicines.
When a rule-based decision support system is too costly to develop, a sim- ple checklist can still make a difference. Peter Pronovost, a critical-care specialist at Johns Hopkins hospital, assembled a checklist to help prevent line infections, a common occurrence in many intensive care units. Pronovost made two key decisions: (1) he decided to create a paper checklist rather than try to cobble together a software system, (2) his checklist was narrowly focused to line infec- tions. After convincing his hospital’s administrators to adopt his checklist, the 10-day line infection rate went from 11% to zero.
Data, data, and more data: Simply put, many algorithmic systems improve
as more data becomes available. Recommendation engines require a minimum amount of data to get going, and only after accumulating more data do the sys- tems start generating great recommendations. In the course of developing tools for users (e.g., language translation, image processing), Google research- ers allude to “…a threshold of sufficient data” beyond which their algorithms start producing great results.
Text-mining algorithms are useful, but have known limitations: We are
avid users of text-mining tools (e.g., classifiers, topic models, clustering, summa- rization, sentiment analysis) and frequently recommend them to others. Unfortunately, many users don’t appreciate the limitations and metrics used to evaluate popular text mining algorithms (e.g., how much does an algorithm depend on a particular domain, or on linguistic rules and syntax?). We don’t think having a resident NLP expert is necessary, but a basic understanding of their assumptions and weaknesses is important.
Humans are the “last mile:” For best results, combine machine-learning and
human editors. Human judgment can nicely complement an automated sys- tem. Algorithms and rules can produce acceptable performance levels, but humans can help turn good systems into great ones. Even when you intend to use pure machine-learning tools, you’ll likely still need humans to help build large training sets. We categorize technical books along several dimensions and use machine-learning and rule-based classifiers to suggest categories. The results are important enough that we use domain experts (O’Reilly editors) to make the final call. Google News is a machine-generated news aggregation ser- monitor and occasionally overrule algorithms. Image tagging (especially porn vs. not porn) is an area where humans are still widely used: in contrast to search engines that rely on algorithms, the more cautious social networks vet every image that gets uploaded onto their sites.
Turks to the rescue: Pioneered by Amazon, mechanical turks are Amazon
account holders who agree to perform simple, repetitive tasks for a very small fee. Many sites use turks for image tagging (porn vs. not porn), or text tagging and categorization. Aimed at harnessing East African mobile phone users, txteagle untethers its mechanical turk service from the PC and delivers tasks and payments through the mobile phone. Dolores Labs is a San Francisco- based mechanical turk company that rates their contributing turks and pro- vides companies with an intuitive user interface for defining the tasks they want performed. To improve quality of results, both Dolores Labs and txteagle make it easy to specify the number of turks for each individual task. It turns out that for the simple labeling tasks most associated with mechanical turks, research indicates that accuracy increases rapidly with the number of turks employed: the aggregate work of 4 non-experts closely approximates the work of an expert. Turks from Dolores Labs have helped build training sets for some of our classifiers.
Game psychology as an untapped resource: Game-like elements can make
hybrid (machine + human) systems more effective. When IBM experimented with customer self-service technologies in a virtual world, they were inspired by how users of massively multiplayer online games came together to perform complex tasks. Incentives/bounties and collaborative problem-solving became important elements of IBM’s system. Dolores Labs uses scoreboards and leader boards to foster friendly competition and increase the productivity of their mechanical turks. By converting routine tasks into games, Games With A
Release 2.0.12 April 2009 Our Search for Intelligent Software Agents Ben Lorica
Purpose turns players into mechanical turks who help improve machine-learn- ing algorithms.
Virtual Worlds are nascent, getting easier, and worth paying attention to:
For certain applications, (e.g., customer care) immersive 3D environments will eventually be common. Besides verbal and written communication, customer care workers can use software agents, gestures, and visual cues to assist cus- tomers with complex tasks. Located inside a virtual world environment and borrowing heavily from massively multiplayer online games, IBM’s experimental environment for self-service technologies relied primarily on users to resolve solving fell short did the system connect users to IBM technicians. Complexity remains an issue: realistic virtual worlds are still not easy to use, but there has been recent progress. Tools that let designers import files from graphics appli- cations have made the task of building spaces in virtual worlds easier. Tools for creating intelligent software agents vary by platform, with some platforms rely- ing on proprietary languages (e.g., Second Life’s Linden scripting language) and n n others using standard tools (Multiverse users can code scripts in Python). Release 2.0.12 April 2009 Analytics in Action Ben Lorica & Roger Magoulas Analytics in Action: A Conversation with LinkedIn’s Analytics Team by Ben Lorica & Roger Magoulas
Many social networks have been early adopters of the Big Data management tools we described previously (see Issue 2.0.11, February, 2009). Some have begun analyzing their massive data sets. We recently spent a few hours with LinkedIn’s analytics team and were impressed by their willingness to learn and experiment with new tools and techniques, and their increasing role in product development within the company. In a period of relatively high unemployment, sites like LinkedIn can draw even more users if they can facilitate job searches. Examples of how analytic tools have helped job seekers and other LinkedIn users are mentioned in the following interview. Another interesting direction analytics team is starting to use their data sets to detect macroeconomic trends, an effort that may lead to tools for policy makers.
Unlike many analytic organizations that tend to separate analysts and quants from computer scientists, LinkedIn’s team includes statisticians, text miners, physicists, mathematicians, and computer scientists, who work side- by-side on numerous projects. The size of their data means the team works routinely with Big Data tools including Hadoop, MPP & key-value databases, statistics packages, and visualization toolkits. When suitable tools aren’t avail- able, the team is comfortable building their own. We saw an impressive (but unreleased) interactive visualization tool (complete with transition probabili- ties) designed to help users assess optimal paths to specific job roles. A mem- ber of LinkedIn’s analytics team heads Project Voldemort, a popular open source distributed key-value storage system.
R2: What types of people do you currently have on the analytics team? LinkedIn: We look for the rare combination of technical capability and cre-
ativity. Most importantly we look for the ability to analyze a problem and so far we have found these people from all sorts of backgrounds. Our team includes individuals with backgrounds in physics, statistics, math, and computer science.
R2: How did you recruit for this group? LinkedIn: Luckily at LinkedIn we have a pretty good working knowledge of
our product which allows us to reach exactly the people we want to target for job opportunities in analytics. We use the tool just like other users and compa- nies. We post jobs and InMail individuals who appear to have the right back- ground for the areas we want to grow in. Also, we occasionally receive messages from users who inquire about positions in our group. We’ve also started up an internship program for both undergraduate and graduate stu- dents as well as occasional collaborations with academic research groups.
Release 2.0.12 April 2009 Analytics in Action Ben Lorica & Roger Magoulas
These research projects include individuals from computer science depart-
LinkedIn is getting closer to
ments at the forefront of social network analysis and groups from business
becoming a good gauge of schools trying to better understand labor flows. the world economy.
R2: What surprising trends have you uncovered in your data? LinkedIn: One of the interesting things we have looked at is what new job
titles are appearing. We have job history data for many users going back to early in their careers (people list jobs from many years back). Recently we have seen a surge in titles related to the gaming industry (e.g., ‘3D artist’). Using our (and the evolution of software engineering titles). Other interesting trends include the recent decline of the real estate industry. We can also use our tools to highlight the rise and decline of the Internet and telecommunications indus- try from 1996-2002. In many ways LinkedIn is getting closer to becoming a good gauge of the world economy, and by looking at the data we can help identify which industries are underserved. In the future, we envision using this data to help guide young people on career paths they should consider pursuing.
R2: What is LinkedIn’s philosophy towards analytics? LinkedIn: LinkedIn has built up a considerable user base (40 million and
growing) and with that, a tremendous amount of data about how users interact with the site. We are using this data to become more user-centric in our design approach and product ideation. For example, by observing the session-level behavior of millions of users, we can identify how to better improve the search experience on LinkedIn and subsequently measure the impact of our efforts. In addition to iterating and improving basic site-wide metrics (primarily visits and engagement), we invest heavily in trying to find insights from our data that can point to new products to develop or areas to focus on. For example, we have extensively studied variations in user behavior based on location and have tried to identify what, if any, approaches we need to rethink as we expand interna- tionally. We can begin to answer questions like: Are users in country X not as viral as users elsewhere because of cultural reasons, features on the site, com- petition, or site speed?
Given the structured profile data we have for each member, we are using analytics to deliver relevant and targeted content to each user. In some cases, this means building upon existing features and finding relevant content to push to users. We also try to steer the company towards trying new ideas inspired by analysis. In that way, analytics plays an R&D role within the product
Identifying what is relevant group. Our goal is to build new, highly targeted, and data-rich projects quickly. to a user at the right time
Those that don’t move the needle, we table. Those we see as successful we then graduate to be more heavily invested in by the rest of the organization. We
is critical to ensuring a click,
have recently established and built up an engineering team that is focused on delivering data-driven products.
a purchase, and a good user experience.
R2: What user-facing features has the analytics team built? LinkedIn: Our team conceived and built the “people you may know” feature has reconnected millions of members and very often led to better engagement.
We’ve also been working on a talent match tool for automatically showing job posters the individuals who best match the openings they’ve posted. We are trying to reverse the results as well, and show the most relevant jobs to our users when they login. We have promoted groups to users based on content in their profile and have built “browse maps,” where users can find similar people or jobs based on the aggregate behavior of millions of users. All of these prod- ucts have demonstrated high engagement rates and have led to longer ses- sions. In addition, there is a viral component in many of these products. For example, if a user joins a group, his or her connections can see that action and take action themselves. So in total, the right targeted content to one individual gets a “multiplier” effect due to the viral nature of our professional network.