Contract Delivery Date Actual Delivery Date Deliverable Type
FP7-2011-7-287661
Document Title
D5.4 Intent-aware User Interface Software - Social Interfaces
Project Number
FP7-2011-7-287661
Contract Delivery Date
M28
Project Acronym
GAMBAS
Document
Version
0.8
Project Title
Generic Adaptive Middleware for Behavior-driven
Autonomous Services
Actual Delivery Date
Deliverable Type
P (Prototype)
Document Author or Reviewer
Stefan Foell
Gerd Kortuem
Marcus Handte
Organization
OU
OU
UDE
Deliverable Access
CO (Confidential)
Work Package
WP5
WP5
WP5
Abstract
This document describes the second prototype of the intent-aware user interface software
developed by the GAMBAS project and completes the implementation of the user interface stack.
For each layer of the intent-aware user interface model, a set of user interfaces is presented which
facilitates information access and user interaction on this layer. Finally, the interface stack is
complemented with social user interfaces to foster insights into and knowledge of social behaviours.
It is important to stress that this document itself only accompanies the prototype and it is not an
actual deliverable. The actual deliverable, i.e. the code of the prototype, can be accessed via the
shared code repository hosted by UDE.
GAMBAS STREP
1
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Revision Organization Description
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
OU
OU
OU
OU
OU
OU
UDE
UDE
GAMBAS STREP
Created initial template
Added document structure
Added description of prototype iterations
Added description and screenshots of user interfaces
Added introduction and conclusion
Added requirements coverage
Internal review.
Prepared submission.
2
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Table of Contents
1
2
3
Introduction..................................................................................................................................... 5
1.1
Purpose.................................................................................................................................... 5
1.2
Scope ....................................................................................................................................... 5
1.3
Structure .................................................................................................................................. 6
Prototype Coverage......................................................................................................................... 7
2.1
Prototype 1 .............................................................................................................................. 7
2.2
Prototype 2 .............................................................................................................................. 7
Intent-Aware User Interface Layers ................................................................................................ 9
3.1
Transport Layer ....................................................................................................................... 9
3.2
Quality-of-Transport Layer .................................................................................................... 11
3.3
Personal Behaviour Layer ...................................................................................................... 12
3.4
Social Behaviour Layer .......................................................................................................... 14
4
Requirement Coverage.................................................................................................................. 17
5
Conclusion ..................................................................................................................................... 20
6
Bibliography................................................................................................................................... 21
GAMBAS STREP
3
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
List of Figures
Figure 1 Intent-aware User Interface Model........................................................................................... 7
Figure 2 User interface menu .................................................................................................................. 9
Figure 3 Route plannig interface ........................................................................................................... 10
Figure 4 Indication of crowd levels on routes ....................................................................................... 12
Figure 5 Navgation Service .................................................................................................................... 13
Figure 6 Personal user interfaces .......................................................................................................... 13
Figure 7 User interface for sharing of trip goals.................................................................................... 15
Figure 8 User interfaces for social travel Information .......................................................................... 16
GAMBAS STREP
4
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
1 Introduction
This document reports on the development of the second prototype of intent-aware user interface
software developed by the GAMBAS project. The document accompanies the actual deliverable, i.e.
the code of the prototype, which can be accessed via the shared code repository hosted by UDE. In
the following, we describe the key features of the software and present the four user interface layers
which make up the GAMBAS user interface. The key innovation added in the final design iteration is
the social user interface layer which supports social information exchange of public transport users.
1.1 Purpose
The aim of WP5 is the design, development and evaluation of a novel intent-aware user interface
concept, with a particular focus on mobile guidance in public transport scenarios which denotes one
key application domain in GAMBAS. This deliverable D5.4 presents the GAMBAS user interface
framework, i.e. the concrete visual user interfaces and user interface software that has been
developed as part of the GAMBAS project.
Early work in WP5 developed an intent-aware user interfaces model which maps out the user
experience of an innovative mobile guide system (W5.1). The model is inherently data-driven, making
use of context data, social information and public transport data use to create information-rich and
effective user interfaces for mobile users. Altogether, the model consists of four layers with different
features and services. (1) On the transport layer, public transport data is crawled and incorporated
into mobile interface software. (2) On the quality-of-transport layer, qualitative information about
conditions of transport usage and crowd levels on public transport routes is made accessible to
users. (3) On the personal behaviour level, transport behaviour recommendations are delivered to
travellers to improve navigation and decision-making in public transport networks. (4) On the social
behavioural level, transport usage becomes a social experience as fellow travellers and information
about their transport usage is visible. This deliverable D5.4 describes the complete GAMBAS user
interface framework, i.e. the concrete realisation of this model. The framework encompasses user
interfaces and software that for all four layers of the model. While previous reports have focused on
personal interfaces, this Deliverable D5.4 includes a description of the social layer which extends the
previous software release with features for shared, social transport experiences..
1.2 Scope
Deliverable 5.4 is the fourth deliverable in WP3: Deliverable 5.1 laid the groundwork by defining a
design approach for intent-aware user interfaces; Deliverable 5.2 described the first version of the
user interface framework; and D5.3 delivered first insights into usability aspects. In line with the
overall project organisation, this deliverable D5.4 completes the design and development of the user
interface software. While the first iteration of the software delivered personal user interfaces, this
deliverable presents the entire stack of the user interface system including a new set of social
interfaces.
The software code underlying these social interfaces will be integrated into the 2nd GAMBAS
application prototype and tested in real-world trials. This deliverable D5.4 focuses on the features
and design of the user interface software. A forthcoming study which will evaluate real-world usage
of the 2nd GAMBAS application prototype, including the new social features, will be reported in D5.5.
GAMBAS STREP
5
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
1.3 Structure
The remainder of this document is structured as follows. In Section 2, we describe the evolution of
the user interface prototypes over two iterations in relation to the proposed user interface model. In
Section 3, we present concrete user interfaces which have been developed to support the complex
information need of mobile users on the four layers of the model. Section 4 revisits the requirements
for intent-aware user interface which have been defined earlier in the project as part of the
requirement specification, and gives justification about their full coverage in the current
implementation.
GAMBAS STREP
6
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
2 Prototype Coverage
The development of user interface framework follows the conceptual intent-aware user interfaces
model which has been presented in D5. 1 [GIUIM] (Figure 1). The model is based on a stack of four
layers, where each layer encapsulates interactive elements for a specific aspect of a mobile intentaware transport application. Each layer is realized by a set of concrete user interfaces which provide
effective support and guidance to users in public transport scenarios.
Figure 1 Intent-aware User Interface Model
The realisation of the complete user interface model was undertaken in two major iterations
(GAMBAS prototype 1 and prototype 2), with each iteration covering a subset of the model features.
In the following, we give an overview how layers were developed across both iterations.
2.1 Prototype 1
The first prototype realised the user interface foundation required by the GAMBAS mobile travel
guide. In summary, the software encompasses the following components:
Personalized navigation: The user interface software provides support for continuous
navigation which allows transport users to receive recommendation of the best next travel
steps over the course of a journey.
Crowd information indication: The user interface software visualizes and indicates expected
levels of crowdedness on transport routes to allow users make more effective travel
decisions.
Voice input integration: The user interface software facilitates simplified user interaction by
enabling voice commands as input to trip planning and navigation.
2.2 Prototype 2
The second prototype extends the component developed during first iteration to integrate social
travel experiences into mobile user guides. This includes the following mechanism and services:
GAMBAS STREP
7
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Semantic annotation of user trips: The user interface software enables users to annotate
routes they are following, giving trips a meaning and social context that can be shared among
friends.
Exchange of social travel information: The user interface software is extended to collect and
represent travel information from a user’s social network. This way, users gain insight into
their friends’ travel activities and the routes they are following.
Social route planning: The user interface software integrates social network information into
route planning functionality by giving user access to their friend’s real-time locations that can
be used as destinations for planned routes.
The first and second iteration of the user interface framework has been incorporated into a
prototype implementation which is the basis for real-world trials with public transport users in
Madrid. An overview of the implemented user interfaces and a description of the possible user
interactions are given in the next section.
GAMBAS STREP
8
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
3 Intent-Aware User Interface Framework
In this section, we describe the concrete user interface of the GAMBAS mobile prototype application.
The following discussion follows the layer model outlined in the previous section, starting with the
lowest layer which provides the central user interface for route planning. Then, on the next upper
layer, we describe the integration of quality-of-transport information into user interface for the
representation of crowd information on travel routes. Next, we describe a set of personalized user
interfaces which assist users with continuous navigation features. Finally, we present the social user
interface layer which enhances the interfaces with social features that allows users to share travel
plans with their social network .
The interfaces and services on all layers can be accessed using the application menu shown in Figure
2. The top three entries in the application menu correspond to the transport, personal and social
layers discussed above.
Figure 2 User interface menu
3.1 Transport Layer
3.1.1 Use cases
The following table indicates which use cases are covered by the Transport Layer. For details see
Section 4 (Requirement Coverage).
User Interface Layer
Transport Layer
Supported Use Cases
UI_008
UI_017
3.1.2 Description
The transport layer provides route planning functionality (Figure 3, left) that can be used by traveller
before or during a journey. For route planning, a user can request schedules and journey information
GAMBAS STREP
9
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
for a trip from a given origin to desired destination. For this purpose, an interface has been
developed which encompasses the following elements:
An origin field
An destination field
A time field
A toggle button for selecting departure or arrival time
A list of returned route alternatives
By default, the origin is set to the user’s current location which is collected from GPS information on
the user’s mobile device. This enables ad-hoc planning of routes from the current position of a user
while being on the move.
The user can choose to use either text or voice input when interacting with the route planning
interface. Voice input is incorporated to ease the process of user input. By pressing the microphone
button, the user is asked to speak and indicate the name of the location which should be used either
as origin or destination address. The voice is translated into a textual string and used to automatically
update the fields with the required information.
Once all input has been specified and direction information is requested, the interface lists different
route options available for the desired journey. Each alternative is associated with contextual data
about the journey, allowing users to make an informed travel decision about which route to choose.
More precisely, each option shows the different transport modalities (walking, bus, train) in the same
order as they need to be taken for the journey. In addition, each option shows the distance to travel,
the duration of the journey, as well as the departure and arrival time of the journey (for non-walking
segments only).
Figure 3 Route plannig interface
GAMBAS STREP
10
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
3.2 Quality-of-Transport Layer
3.2.1 Use cases
The following table indicates which use cases are covered by the Quality of Transport Layer. For
details see Section 4 (Requirement Coverage).
User Interface Layer
Quality of Transport Layer
Supported Use Cases
UI_016
UI_017
3.2.2 Description
In addition to the directions returned by the route planning interface, additional quality-of-transport
information is available which gives further insights into the conditions of a transport journey. This
information becomes visible once the user clicks on a route option to look at its further details.
Route information is shown on a map to visualize the details of a journey. This representation helps a
user to gain a better spatial understanding of a journey. For this purpose, the map shows the travel
segments involved, from the origin to the destination location of the journey. A travel segment is
represented as a spatial polyline which encodes the route trajectory. The route trajectory of a public
transport segment contains all stops before the user needs to depart from the vehicle.
Different colour codes are used to distinguish between walking and public transport segments. For
public transport routes, the width of the polyline encodes the expected crowdedness of the
transport journey within the transport network. Crowdedness is a quality-of-transport measure that
refers to the number of people travelling on a route segment in between two stops, in relation to the
passenger capacity of a bus.
Thinner lines highlight parts of the route network where no overcrowding is expected. In contrast,
thicker lines give users an indication of congested transport segments. To allow users easy
perception of this information, a mapping has been defined from numerical crowd measurements to
semantic labels which are meaningful to users. While the numeric values encode percentages as to
which the capacity of buses is filled, the labels are defined as categories (low crowded, extremely
crowded) which express the severity of overcrowding. Once the user selects a route segment on the
map, a popup dialog is opened which shows the label of crowdedness for this segment.
GAMBAS STREP
11
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Figure 4 Indication of crowd levels on routes
3.3 Personal Behaviour Layer
3.3.1 Use cases
The following table indicates which use cases are covered by the Personal Behavior Layer. For details
see Section 4 (Requirement Coverage).
User Interface Layer
Personal Behavior Layer
Supported Use Cases
UI_005
UI_011
UI_012
UI_014
UI_018
UI_019
UI_021
UI_022
3.3.2 Description
While route planning is performed before the start of a journey, navigation is realized as a
personalized service which makes use of a user’s context information to adapt user interface and
information provided to a user during a journey. The navigation starts as soon as a user a selects a
specific route to follow.
The navigation service is accessible as a user interface shown in Figure 6. The interface is composed
of the following key components:
Geographic map which is updated according to user movement
User icons which encode the current travel activity of user
Navigation instructions which suggest next steps
GAMBAS STREP
12
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Figure 5 Navigation Service
The geographic map provides a static visualization of the travel plan associated with the journey. This
visualization is used as a base map which is overlaid with user’s current context. First, the user is
represented as an icon which is dynamically updated to his current location. This allows users to
navigate the travel plan in relation to their current context, e.g. to find a bus stop. Second, the map
encodes the travel activities of a user using different icons which represent either bus ride or walking
activity for further feedback.
Figure 6 Personal user interfaces
GAMBAS STREP
13
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
The area above the map is used a notification area to supply users with situational navigation
instructions. The instructions are tied to the user’s current context and contain information about
the next steps to take. In case a user is walking, the interface shows the remaining distance to the
end of the walking segment. If a user approaches a bus stop, the next bus arrivals are shown. Once
the user is on the bus, information about the number of stops is given before an alighting is required.
Similarly, corrective actions are notified that require special attention by users. For instance, the user
is notified in case an alighting has been missed or a wrong bus has been accidentally been boarded. It
is important to note that the interface has been designed to require no manual user input to elicit a
notification. Each notification is automatically triggered by changes in the user behaviour which are
detected from available context information.
Further, the personal user interfaces give users access to their trip ride histories. As shown in Figure
6, a trip history view is provided which lists a user’s past rides that have been recognized. Also, a
voice tag management interface has been implemented which allows for creating new or deleting
existing voice tags. This allows a user to introduce voice tags for stops which are important for his
frequent travel needs, e.g. home or work.
3.4 Social Behaviour Layer
3.4.1 Use cases
The following table indicates which use cases are covered by the Social Behavior Layer. For details
see Section 4 (Requirement Coverage).
User Interface Layer
Social Behavior Layer
Supported Use Cases
UI_013
UI_020
UI_021
3.4.2 Description
On top of the personal behaviour layer, an additional service is provided which augments personal
experiences with views and information on social transport usage. The idea is that public transport
can be a powerful platform for sharing information about journeys and rides among companion
travellers. As public transport systems represent a shared mode of transport, rich travel activity
information becomes available which can be visualized to let users perceive public transport usage as
an inherent social experience.
On top of the personal user interfaces, we have integrated a social networking service that fosters
information exchange among travellers in two directions. On the one hand, transport users can
annotate their trips with goals to inform their friends about their current routes and travel intents.
On the other hand, we give users access to social views of the transport system which encompasses
real-time status information about the latest travel activities of friends in their social circle.
Figure 7 shows the user interface developed for capturing social information about a user’s trip goal.
Once a user has selected a route for navigation, he is asked to provide a short message to describe
his travel activity. Basically, any description can be given which seems suitable for users to express
the goal of their journeys. Once a trip has been annotated, the information is stored on the mobile
device and linked to the user’s journey.
GAMBAS STREP
14
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Figure 7 User interface for sharing of trip goals
As shown in Figure 8, users can access social information about the latest real-time travel activities of
their friends. We provide two different interfaces, a coarse-grained view which provides summary
information about all friends and a finer-grained view which gives details about a single friend’s
current activity.
The coarse-grained view shows the all friends in a user’s social circle. Friendship relations are derived
from a user’s Facebook account [Facebook] which can be imported into GAMBAS. This is
accomplished by the privacy preservation framework which controls the formation of social circles
and privacy settings of shared context information [GPPS1, GPPS2]. For each friend, a list entry is
shown that gives an overview about the user’s current status. The status information specifies
whether a user is currently online or offline. If a user is found to be online, information about the
distance between the user’s current location and that of his friend is given. This is to highlight friends
in close distance who could be potentially interested in joint activities with the user.
For each online user, more detailed information can be accessed when clicking on the entry in the list
associated with a user. Then, a separate interface is shown which presents the full user profile,
including the friend’s description of his travel activity. This gives users a better idea of the rationale
behind the journey of their friends. Moreover, the current travel activity of a friend is represented on
a geographic map which is constantly updated. The map representation is able to reveal two
different kinds of information. On the one hand, the friend‘s geographic location is shown and
updated as he is moving. On the other hand, the map shows a representation of the travel plan
currently followed by his friend. As a consequence, the social view of a friend’s travel activity conveys
both real-time information about the current travel activity as well as future information about the
prospective route on which the friend is travelling.
To ease transport usage in social situations, the profile view of a friend provides a shortcut for route
planning. Once the user clicks on a navigation button, the user is forwarded to the route planning
interface with the friend’s current location set as a destination. Therefore, a direct way is provided to
GAMBAS STREP
15
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
meet friends at their current location, minimizing the effort for negotiating spatial coordinates to
find meeting points in urban spaces.
Figure 8 User interfaces for social travel Information
GAMBAS STREP
16
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
4 Requirement Coverage
This section revisits the relevant requirements for the development of intent-aware user interfaces
as defined in D1.1 Requirements Specification. In the following, we discuss the coverage of the
requirements in the user interface software prototype and demonstrate that all functional
requirements have been fully incorporated into the user interface software. The non-functional
requirements are outside the scope of this document as they relate to the study of user experience
which is subject to the user evaluation deliverables D5.3 and D5.4. In the following, we iterative over
the requirements and give the rationale as to which each requirement has been fully covered in the
user interface software implementation.
This table contains the id, a short description, and the type of the requirement and its priority as they
have been described in D1.1 Requirements Specification.
ID
Description
UI_005
The user interface performs proactive
recommendations on trips.
The personal user interfaces support the
visualization of possible routes.
The user interface component must
recommend personalised transport routes.
The user interface component must provide
personalised information related to user
intent.
UI_013. The user interface component
should enable users to become aware of
friend's travel behavior.
The user interface should anticipate user
information needs and be designed to
minimize user input.
The user interfaces should be able to
visualize crowd levels on public transport
routes
The user interface should provide mapbased representations of a user's transport
opportunities.
The user interface should provide means to
predict a user's travel profile from histories
of past tansport usage
The user interface should be able to
visuzalize a user's travel profile and link it
with journey information.
The user interface should be able to able to
represent the travel profiles from a user's
social network.
The user interface should be automatically
updated according to changes in the context
of a user.
The user interface should allow for voicebased user input.
UI_008
UI_011
UI_012
UI_013
UI_014
UI_016
UI_017
UI_018
UI_019
UI_020
UI_021
UI_022
GAMBAS STREP
Type
17
Priority
Functional and data
requirements
Functional and data
requirements
Functional and data
requirements
Functional and data
requirements
High
Functional and data
requirements
High
Functional and data
requirements
Medium
Functional and data
requirements
High
Functional and data
requirements
High
Functional and data
requirements
High
Functional and data
requirements
High
Functional and data
requirements
High
Functional and data
requirements
Medium
Functional and data
requirements
Medium
High
High
High
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Table 1 Intent-Aware UI Requirements
The rationale for the full coverage of each listed requirements is as follows:
UI_005: The navigation service performs proactive recommendations on the user’s next travel steps.
Based on information of a user’s current context and his selected route, notifications are triggered
which suggest the most effective travel actions to the users.
UI_008: Geometric maps and associated visualization components are available. The maps are able
to visualize route information for all bus services in Madrid.
UI_011: Route and navigation information is presented to a user in a personalized way. The user’s
current context and transport behaviour is used to trigger the most informative instructions for a trip
goal. This includes detection of deviating behaviours and suggestions of corrective actions, e.g. when
a bus alighting has been missed.
UI_012: The user’s intent in form of his overall trip goal is central to the navigation service. This
information is used by the continuous navigation service to provide step-by-step recommendation as
to which actions the user should take in order to successfully complete a trip.
UI_013: The most recent travel activities of users are exchanged within social circles. The social travel
information includes the semantic trip purpose and planned route. Also, the location of the friends
can be accessed from the social user interface.
UI_014: Updates to the personal user interface are triggered by the navigation services without any
need for explicit user interaction. Contextual information originating from transport activity
detection and location recognition is used as a trigger instead.
UI_16: Crowd information of bus routes is integrated into a map based representation of a user’s
travel plan. The degree of crowdedness (low,high) is visually encoded using visual clues (line width)
and semantic labels attached to route segment.
UI_17: A map based interface has been developed which shows a user’s travel plan using a
geographic representation. The map visualizes the trajectory of a route and distinguishes between
different modes of transport (walking, public transport) which make up a transport journey.
UI_18: The route planning interface has been augmented with different forms of user inputs. One
implicit form of user input is the prediction of a user’s next likely visited location. The predicted
location is used to suggest a possible destination for route planning.
UI_19: A representation of a user’s travel profile has been developed which encodes the routes
regularly visited by a user. The travel profile can be visualized to show a personalized view on the
transport network that only contains routes being relevant to the user.
UI_020: Social travel profiles exchanged among friends reveal information about travel activities of a
social group of travellers. The travel profiles are visualized to learn about the trip goals and routes
taken by a user’s friends.
GAMBAS STREP
18
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
UI_021: The user interface software has been implemented as a context-aware system. Changes in
the user’s context such as transport mode, location or travel behaviour trigger dynamic navigation
messages which are specific to behaviour and intent.
UI_022: A voice tag management interface is available which can be used to support voice-based
user input. Users can create voice tags to associate a specific voice fingerprint with location
information. Then, voice commands can be used to specify origin and destination information for
route planning.
GAMBAS STREP
19
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
5 Conclusion
This document describes the second prototype of the intent-aware user interface software
developed by the GAMBAS project and completes the implementation of the user interface stack.
For each layer of the intent-aware user interface model, a set of user interfaces is presented which
facilitates information access and user interaction on this layer. Finally, the interface stack is
complemented with social user interfaces to foster insights into and knowledge of social behaviours.
It is important to stress that this document itself only accompanies the prototype and it is not an
actual deliverable. The actual deliverable, i.e. the code of the prototype, can be accessed via the
shared code repository hosted by UDE.
GAMBAS STREP
20
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
6 Bibliography
[Facebook] Facebook, online at http://www.facebook.com
[GIUIM] GAMBAS Consortium, Intent-Aware User Interface Model, Public Deliverable D5.1.1,
November 2012, online at http://www.gambas-ict.eu
[GPPS1] GAMBAS Consortium, Privacy Preservation Specification 1, Public Deliverable D3.1.1,
September 2012, online at http://www.gambas-ict.eu
[GPPS1] GAMBAS Consortium, Privacy Preservation Specification 2, Public Deliverable D3.1.1, July
2013, online at http://www.gambas-ict.eu
GAMBAS STREP
21
D5.4 SOCIAL INTERFACES
Document Title
D5.4 Intent-aware User Interface Software - Social Interfaces
Project Number
FP7-2011-7-287661
Contract Delivery Date
M28
Project Acronym
GAMBAS
Document
Version
0.8
Project Title
Generic Adaptive Middleware for Behavior-driven
Autonomous Services
Actual Delivery Date
Deliverable Type
P (Prototype)
Document Author or Reviewer
Stefan Foell
Gerd Kortuem
Marcus Handte
Organization
OU
OU
UDE
Deliverable Access
CO (Confidential)
Work Package
WP5
WP5
WP5
Abstract
This document describes the second prototype of the intent-aware user interface software
developed by the GAMBAS project and completes the implementation of the user interface stack.
For each layer of the intent-aware user interface model, a set of user interfaces is presented which
facilitates information access and user interaction on this layer. Finally, the interface stack is
complemented with social user interfaces to foster insights into and knowledge of social behaviours.
It is important to stress that this document itself only accompanies the prototype and it is not an
actual deliverable. The actual deliverable, i.e. the code of the prototype, can be accessed via the
shared code repository hosted by UDE.
GAMBAS STREP
1
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Revision Organization Description
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
OU
OU
OU
OU
OU
OU
UDE
UDE
GAMBAS STREP
Created initial template
Added document structure
Added description of prototype iterations
Added description and screenshots of user interfaces
Added introduction and conclusion
Added requirements coverage
Internal review.
Prepared submission.
2
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Table of Contents
1
2
3
Introduction..................................................................................................................................... 5
1.1
Purpose.................................................................................................................................... 5
1.2
Scope ....................................................................................................................................... 5
1.3
Structure .................................................................................................................................. 6
Prototype Coverage......................................................................................................................... 7
2.1
Prototype 1 .............................................................................................................................. 7
2.2
Prototype 2 .............................................................................................................................. 7
Intent-Aware User Interface Layers ................................................................................................ 9
3.1
Transport Layer ....................................................................................................................... 9
3.2
Quality-of-Transport Layer .................................................................................................... 11
3.3
Personal Behaviour Layer ...................................................................................................... 12
3.4
Social Behaviour Layer .......................................................................................................... 14
4
Requirement Coverage.................................................................................................................. 17
5
Conclusion ..................................................................................................................................... 20
6
Bibliography................................................................................................................................... 21
GAMBAS STREP
3
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
List of Figures
Figure 1 Intent-aware User Interface Model........................................................................................... 7
Figure 2 User interface menu .................................................................................................................. 9
Figure 3 Route plannig interface ........................................................................................................... 10
Figure 4 Indication of crowd levels on routes ....................................................................................... 12
Figure 5 Navgation Service .................................................................................................................... 13
Figure 6 Personal user interfaces .......................................................................................................... 13
Figure 7 User interface for sharing of trip goals.................................................................................... 15
Figure 8 User interfaces for social travel Information .......................................................................... 16
GAMBAS STREP
4
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
1 Introduction
This document reports on the development of the second prototype of intent-aware user interface
software developed by the GAMBAS project. The document accompanies the actual deliverable, i.e.
the code of the prototype, which can be accessed via the shared code repository hosted by UDE. In
the following, we describe the key features of the software and present the four user interface layers
which make up the GAMBAS user interface. The key innovation added in the final design iteration is
the social user interface layer which supports social information exchange of public transport users.
1.1 Purpose
The aim of WP5 is the design, development and evaluation of a novel intent-aware user interface
concept, with a particular focus on mobile guidance in public transport scenarios which denotes one
key application domain in GAMBAS. This deliverable D5.4 presents the GAMBAS user interface
framework, i.e. the concrete visual user interfaces and user interface software that has been
developed as part of the GAMBAS project.
Early work in WP5 developed an intent-aware user interfaces model which maps out the user
experience of an innovative mobile guide system (W5.1). The model is inherently data-driven, making
use of context data, social information and public transport data use to create information-rich and
effective user interfaces for mobile users. Altogether, the model consists of four layers with different
features and services. (1) On the transport layer, public transport data is crawled and incorporated
into mobile interface software. (2) On the quality-of-transport layer, qualitative information about
conditions of transport usage and crowd levels on public transport routes is made accessible to
users. (3) On the personal behaviour level, transport behaviour recommendations are delivered to
travellers to improve navigation and decision-making in public transport networks. (4) On the social
behavioural level, transport usage becomes a social experience as fellow travellers and information
about their transport usage is visible. This deliverable D5.4 describes the complete GAMBAS user
interface framework, i.e. the concrete realisation of this model. The framework encompasses user
interfaces and software that for all four layers of the model. While previous reports have focused on
personal interfaces, this Deliverable D5.4 includes a description of the social layer which extends the
previous software release with features for shared, social transport experiences..
1.2 Scope
Deliverable 5.4 is the fourth deliverable in WP3: Deliverable 5.1 laid the groundwork by defining a
design approach for intent-aware user interfaces; Deliverable 5.2 described the first version of the
user interface framework; and D5.3 delivered first insights into usability aspects. In line with the
overall project organisation, this deliverable D5.4 completes the design and development of the user
interface software. While the first iteration of the software delivered personal user interfaces, this
deliverable presents the entire stack of the user interface system including a new set of social
interfaces.
The software code underlying these social interfaces will be integrated into the 2nd GAMBAS
application prototype and tested in real-world trials. This deliverable D5.4 focuses on the features
and design of the user interface software. A forthcoming study which will evaluate real-world usage
of the 2nd GAMBAS application prototype, including the new social features, will be reported in D5.5.
GAMBAS STREP
5
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
1.3 Structure
The remainder of this document is structured as follows. In Section 2, we describe the evolution of
the user interface prototypes over two iterations in relation to the proposed user interface model. In
Section 3, we present concrete user interfaces which have been developed to support the complex
information need of mobile users on the four layers of the model. Section 4 revisits the requirements
for intent-aware user interface which have been defined earlier in the project as part of the
requirement specification, and gives justification about their full coverage in the current
implementation.
GAMBAS STREP
6
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
2 Prototype Coverage
The development of user interface framework follows the conceptual intent-aware user interfaces
model which has been presented in D5. 1 [GIUIM] (Figure 1). The model is based on a stack of four
layers, where each layer encapsulates interactive elements for a specific aspect of a mobile intentaware transport application. Each layer is realized by a set of concrete user interfaces which provide
effective support and guidance to users in public transport scenarios.
Figure 1 Intent-aware User Interface Model
The realisation of the complete user interface model was undertaken in two major iterations
(GAMBAS prototype 1 and prototype 2), with each iteration covering a subset of the model features.
In the following, we give an overview how layers were developed across both iterations.
2.1 Prototype 1
The first prototype realised the user interface foundation required by the GAMBAS mobile travel
guide. In summary, the software encompasses the following components:
Personalized navigation: The user interface software provides support for continuous
navigation which allows transport users to receive recommendation of the best next travel
steps over the course of a journey.
Crowd information indication: The user interface software visualizes and indicates expected
levels of crowdedness on transport routes to allow users make more effective travel
decisions.
Voice input integration: The user interface software facilitates simplified user interaction by
enabling voice commands as input to trip planning and navigation.
2.2 Prototype 2
The second prototype extends the component developed during first iteration to integrate social
travel experiences into mobile user guides. This includes the following mechanism and services:
GAMBAS STREP
7
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Semantic annotation of user trips: The user interface software enables users to annotate
routes they are following, giving trips a meaning and social context that can be shared among
friends.
Exchange of social travel information: The user interface software is extended to collect and
represent travel information from a user’s social network. This way, users gain insight into
their friends’ travel activities and the routes they are following.
Social route planning: The user interface software integrates social network information into
route planning functionality by giving user access to their friend’s real-time locations that can
be used as destinations for planned routes.
The first and second iteration of the user interface framework has been incorporated into a
prototype implementation which is the basis for real-world trials with public transport users in
Madrid. An overview of the implemented user interfaces and a description of the possible user
interactions are given in the next section.
GAMBAS STREP
8
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
3 Intent-Aware User Interface Framework
In this section, we describe the concrete user interface of the GAMBAS mobile prototype application.
The following discussion follows the layer model outlined in the previous section, starting with the
lowest layer which provides the central user interface for route planning. Then, on the next upper
layer, we describe the integration of quality-of-transport information into user interface for the
representation of crowd information on travel routes. Next, we describe a set of personalized user
interfaces which assist users with continuous navigation features. Finally, we present the social user
interface layer which enhances the interfaces with social features that allows users to share travel
plans with their social network .
The interfaces and services on all layers can be accessed using the application menu shown in Figure
2. The top three entries in the application menu correspond to the transport, personal and social
layers discussed above.
Figure 2 User interface menu
3.1 Transport Layer
3.1.1 Use cases
The following table indicates which use cases are covered by the Transport Layer. For details see
Section 4 (Requirement Coverage).
User Interface Layer
Transport Layer
Supported Use Cases
UI_008
UI_017
3.1.2 Description
The transport layer provides route planning functionality (Figure 3, left) that can be used by traveller
before or during a journey. For route planning, a user can request schedules and journey information
GAMBAS STREP
9
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
for a trip from a given origin to desired destination. For this purpose, an interface has been
developed which encompasses the following elements:
An origin field
An destination field
A time field
A toggle button for selecting departure or arrival time
A list of returned route alternatives
By default, the origin is set to the user’s current location which is collected from GPS information on
the user’s mobile device. This enables ad-hoc planning of routes from the current position of a user
while being on the move.
The user can choose to use either text or voice input when interacting with the route planning
interface. Voice input is incorporated to ease the process of user input. By pressing the microphone
button, the user is asked to speak and indicate the name of the location which should be used either
as origin or destination address. The voice is translated into a textual string and used to automatically
update the fields with the required information.
Once all input has been specified and direction information is requested, the interface lists different
route options available for the desired journey. Each alternative is associated with contextual data
about the journey, allowing users to make an informed travel decision about which route to choose.
More precisely, each option shows the different transport modalities (walking, bus, train) in the same
order as they need to be taken for the journey. In addition, each option shows the distance to travel,
the duration of the journey, as well as the departure and arrival time of the journey (for non-walking
segments only).
Figure 3 Route plannig interface
GAMBAS STREP
10
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
3.2 Quality-of-Transport Layer
3.2.1 Use cases
The following table indicates which use cases are covered by the Quality of Transport Layer. For
details see Section 4 (Requirement Coverage).
User Interface Layer
Quality of Transport Layer
Supported Use Cases
UI_016
UI_017
3.2.2 Description
In addition to the directions returned by the route planning interface, additional quality-of-transport
information is available which gives further insights into the conditions of a transport journey. This
information becomes visible once the user clicks on a route option to look at its further details.
Route information is shown on a map to visualize the details of a journey. This representation helps a
user to gain a better spatial understanding of a journey. For this purpose, the map shows the travel
segments involved, from the origin to the destination location of the journey. A travel segment is
represented as a spatial polyline which encodes the route trajectory. The route trajectory of a public
transport segment contains all stops before the user needs to depart from the vehicle.
Different colour codes are used to distinguish between walking and public transport segments. For
public transport routes, the width of the polyline encodes the expected crowdedness of the
transport journey within the transport network. Crowdedness is a quality-of-transport measure that
refers to the number of people travelling on a route segment in between two stops, in relation to the
passenger capacity of a bus.
Thinner lines highlight parts of the route network where no overcrowding is expected. In contrast,
thicker lines give users an indication of congested transport segments. To allow users easy
perception of this information, a mapping has been defined from numerical crowd measurements to
semantic labels which are meaningful to users. While the numeric values encode percentages as to
which the capacity of buses is filled, the labels are defined as categories (low crowded, extremely
crowded) which express the severity of overcrowding. Once the user selects a route segment on the
map, a popup dialog is opened which shows the label of crowdedness for this segment.
GAMBAS STREP
11
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Figure 4 Indication of crowd levels on routes
3.3 Personal Behaviour Layer
3.3.1 Use cases
The following table indicates which use cases are covered by the Personal Behavior Layer. For details
see Section 4 (Requirement Coverage).
User Interface Layer
Personal Behavior Layer
Supported Use Cases
UI_005
UI_011
UI_012
UI_014
UI_018
UI_019
UI_021
UI_022
3.3.2 Description
While route planning is performed before the start of a journey, navigation is realized as a
personalized service which makes use of a user’s context information to adapt user interface and
information provided to a user during a journey. The navigation starts as soon as a user a selects a
specific route to follow.
The navigation service is accessible as a user interface shown in Figure 6. The interface is composed
of the following key components:
Geographic map which is updated according to user movement
User icons which encode the current travel activity of user
Navigation instructions which suggest next steps
GAMBAS STREP
12
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Figure 5 Navigation Service
The geographic map provides a static visualization of the travel plan associated with the journey. This
visualization is used as a base map which is overlaid with user’s current context. First, the user is
represented as an icon which is dynamically updated to his current location. This allows users to
navigate the travel plan in relation to their current context, e.g. to find a bus stop. Second, the map
encodes the travel activities of a user using different icons which represent either bus ride or walking
activity for further feedback.
Figure 6 Personal user interfaces
GAMBAS STREP
13
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
The area above the map is used a notification area to supply users with situational navigation
instructions. The instructions are tied to the user’s current context and contain information about
the next steps to take. In case a user is walking, the interface shows the remaining distance to the
end of the walking segment. If a user approaches a bus stop, the next bus arrivals are shown. Once
the user is on the bus, information about the number of stops is given before an alighting is required.
Similarly, corrective actions are notified that require special attention by users. For instance, the user
is notified in case an alighting has been missed or a wrong bus has been accidentally been boarded. It
is important to note that the interface has been designed to require no manual user input to elicit a
notification. Each notification is automatically triggered by changes in the user behaviour which are
detected from available context information.
Further, the personal user interfaces give users access to their trip ride histories. As shown in Figure
6, a trip history view is provided which lists a user’s past rides that have been recognized. Also, a
voice tag management interface has been implemented which allows for creating new or deleting
existing voice tags. This allows a user to introduce voice tags for stops which are important for his
frequent travel needs, e.g. home or work.
3.4 Social Behaviour Layer
3.4.1 Use cases
The following table indicates which use cases are covered by the Social Behavior Layer. For details
see Section 4 (Requirement Coverage).
User Interface Layer
Social Behavior Layer
Supported Use Cases
UI_013
UI_020
UI_021
3.4.2 Description
On top of the personal behaviour layer, an additional service is provided which augments personal
experiences with views and information on social transport usage. The idea is that public transport
can be a powerful platform for sharing information about journeys and rides among companion
travellers. As public transport systems represent a shared mode of transport, rich travel activity
information becomes available which can be visualized to let users perceive public transport usage as
an inherent social experience.
On top of the personal user interfaces, we have integrated a social networking service that fosters
information exchange among travellers in two directions. On the one hand, transport users can
annotate their trips with goals to inform their friends about their current routes and travel intents.
On the other hand, we give users access to social views of the transport system which encompasses
real-time status information about the latest travel activities of friends in their social circle.
Figure 7 shows the user interface developed for capturing social information about a user’s trip goal.
Once a user has selected a route for navigation, he is asked to provide a short message to describe
his travel activity. Basically, any description can be given which seems suitable for users to express
the goal of their journeys. Once a trip has been annotated, the information is stored on the mobile
device and linked to the user’s journey.
GAMBAS STREP
14
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Figure 7 User interface for sharing of trip goals
As shown in Figure 8, users can access social information about the latest real-time travel activities of
their friends. We provide two different interfaces, a coarse-grained view which provides summary
information about all friends and a finer-grained view which gives details about a single friend’s
current activity.
The coarse-grained view shows the all friends in a user’s social circle. Friendship relations are derived
from a user’s Facebook account [Facebook] which can be imported into GAMBAS. This is
accomplished by the privacy preservation framework which controls the formation of social circles
and privacy settings of shared context information [GPPS1, GPPS2]. For each friend, a list entry is
shown that gives an overview about the user’s current status. The status information specifies
whether a user is currently online or offline. If a user is found to be online, information about the
distance between the user’s current location and that of his friend is given. This is to highlight friends
in close distance who could be potentially interested in joint activities with the user.
For each online user, more detailed information can be accessed when clicking on the entry in the list
associated with a user. Then, a separate interface is shown which presents the full user profile,
including the friend’s description of his travel activity. This gives users a better idea of the rationale
behind the journey of their friends. Moreover, the current travel activity of a friend is represented on
a geographic map which is constantly updated. The map representation is able to reveal two
different kinds of information. On the one hand, the friend‘s geographic location is shown and
updated as he is moving. On the other hand, the map shows a representation of the travel plan
currently followed by his friend. As a consequence, the social view of a friend’s travel activity conveys
both real-time information about the current travel activity as well as future information about the
prospective route on which the friend is travelling.
To ease transport usage in social situations, the profile view of a friend provides a shortcut for route
planning. Once the user clicks on a navigation button, the user is forwarded to the route planning
interface with the friend’s current location set as a destination. Therefore, a direct way is provided to
GAMBAS STREP
15
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
meet friends at their current location, minimizing the effort for negotiating spatial coordinates to
find meeting points in urban spaces.
Figure 8 User interfaces for social travel Information
GAMBAS STREP
16
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
4 Requirement Coverage
This section revisits the relevant requirements for the development of intent-aware user interfaces
as defined in D1.1 Requirements Specification. In the following, we discuss the coverage of the
requirements in the user interface software prototype and demonstrate that all functional
requirements have been fully incorporated into the user interface software. The non-functional
requirements are outside the scope of this document as they relate to the study of user experience
which is subject to the user evaluation deliverables D5.3 and D5.4. In the following, we iterative over
the requirements and give the rationale as to which each requirement has been fully covered in the
user interface software implementation.
This table contains the id, a short description, and the type of the requirement and its priority as they
have been described in D1.1 Requirements Specification.
ID
Description
UI_005
The user interface performs proactive
recommendations on trips.
The personal user interfaces support the
visualization of possible routes.
The user interface component must
recommend personalised transport routes.
The user interface component must provide
personalised information related to user
intent.
UI_013. The user interface component
should enable users to become aware of
friend's travel behavior.
The user interface should anticipate user
information needs and be designed to
minimize user input.
The user interfaces should be able to
visualize crowd levels on public transport
routes
The user interface should provide mapbased representations of a user's transport
opportunities.
The user interface should provide means to
predict a user's travel profile from histories
of past tansport usage
The user interface should be able to
visuzalize a user's travel profile and link it
with journey information.
The user interface should be able to able to
represent the travel profiles from a user's
social network.
The user interface should be automatically
updated according to changes in the context
of a user.
The user interface should allow for voicebased user input.
UI_008
UI_011
UI_012
UI_013
UI_014
UI_016
UI_017
UI_018
UI_019
UI_020
UI_021
UI_022
GAMBAS STREP
Type
17
Priority
Functional and data
requirements
Functional and data
requirements
Functional and data
requirements
Functional and data
requirements
High
Functional and data
requirements
High
Functional and data
requirements
Medium
Functional and data
requirements
High
Functional and data
requirements
High
Functional and data
requirements
High
Functional and data
requirements
High
Functional and data
requirements
High
Functional and data
requirements
Medium
Functional and data
requirements
Medium
High
High
High
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
Table 1 Intent-Aware UI Requirements
The rationale for the full coverage of each listed requirements is as follows:
UI_005: The navigation service performs proactive recommendations on the user’s next travel steps.
Based on information of a user’s current context and his selected route, notifications are triggered
which suggest the most effective travel actions to the users.
UI_008: Geometric maps and associated visualization components are available. The maps are able
to visualize route information for all bus services in Madrid.
UI_011: Route and navigation information is presented to a user in a personalized way. The user’s
current context and transport behaviour is used to trigger the most informative instructions for a trip
goal. This includes detection of deviating behaviours and suggestions of corrective actions, e.g. when
a bus alighting has been missed.
UI_012: The user’s intent in form of his overall trip goal is central to the navigation service. This
information is used by the continuous navigation service to provide step-by-step recommendation as
to which actions the user should take in order to successfully complete a trip.
UI_013: The most recent travel activities of users are exchanged within social circles. The social travel
information includes the semantic trip purpose and planned route. Also, the location of the friends
can be accessed from the social user interface.
UI_014: Updates to the personal user interface are triggered by the navigation services without any
need for explicit user interaction. Contextual information originating from transport activity
detection and location recognition is used as a trigger instead.
UI_16: Crowd information of bus routes is integrated into a map based representation of a user’s
travel plan. The degree of crowdedness (low,high) is visually encoded using visual clues (line width)
and semantic labels attached to route segment.
UI_17: A map based interface has been developed which shows a user’s travel plan using a
geographic representation. The map visualizes the trajectory of a route and distinguishes between
different modes of transport (walking, public transport) which make up a transport journey.
UI_18: The route planning interface has been augmented with different forms of user inputs. One
implicit form of user input is the prediction of a user’s next likely visited location. The predicted
location is used to suggest a possible destination for route planning.
UI_19: A representation of a user’s travel profile has been developed which encodes the routes
regularly visited by a user. The travel profile can be visualized to show a personalized view on the
transport network that only contains routes being relevant to the user.
UI_020: Social travel profiles exchanged among friends reveal information about travel activities of a
social group of travellers. The travel profiles are visualized to learn about the trip goals and routes
taken by a user’s friends.
GAMBAS STREP
18
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
UI_021: The user interface software has been implemented as a context-aware system. Changes in
the user’s context such as transport mode, location or travel behaviour trigger dynamic navigation
messages which are specific to behaviour and intent.
UI_022: A voice tag management interface is available which can be used to support voice-based
user input. Users can create voice tags to associate a specific voice fingerprint with location
information. Then, voice commands can be used to specify origin and destination information for
route planning.
GAMBAS STREP
19
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
5 Conclusion
This document describes the second prototype of the intent-aware user interface software
developed by the GAMBAS project and completes the implementation of the user interface stack.
For each layer of the intent-aware user interface model, a set of user interfaces is presented which
facilitates information access and user interaction on this layer. Finally, the interface stack is
complemented with social user interfaces to foster insights into and knowledge of social behaviours.
It is important to stress that this document itself only accompanies the prototype and it is not an
actual deliverable. The actual deliverable, i.e. the code of the prototype, can be accessed via the
shared code repository hosted by UDE.
GAMBAS STREP
20
D5.4 SOCIAL INTERFACES
FP7-2011-7-287661
6 Bibliography
[Facebook] Facebook, online at http://www.facebook.com
[GIUIM] GAMBAS Consortium, Intent-Aware User Interface Model, Public Deliverable D5.1.1,
November 2012, online at http://www.gambas-ict.eu
[GPPS1] GAMBAS Consortium, Privacy Preservation Specification 1, Public Deliverable D3.1.1,
September 2012, online at http://www.gambas-ict.eu
[GPPS1] GAMBAS Consortium, Privacy Preservation Specification 2, Public Deliverable D3.1.1, July
2013, online at http://www.gambas-ict.eu
GAMBAS STREP
21
D5.4 SOCIAL INTERFACES