Bi Directional Interactions between User

Bi-Directional Interactions between Users and
Cognitive Buildings by means of Smartphone App
Franz Bittenbinder
Che Liu

Stefano Rinaldi
Paolo Bellagente

Angelo Luigi Camillo Ciribini
Lavinia Chiara Tagliabue

Dept. Architecture, Built Environment and
Construction Engineering (ABC)
Politecnico di Milano
Milano, Italy
stephanviktor.bittenbinder@mail.polimi.it
che.liu@mail.polimi.it

Dept. Information Engineering (DII)

Dept. Civil, Environmental, Architectural

Engineering and Mathematics
(DICATAM)
University of Brescia
Brescia, Italy
angelo.ciribini@unibs.it
lavinia.tagliabue@unibs.it

University of Brescia
Brescia, Italy
stefano.rinaldi@unibs.it
p.bellagente@unibs.it

Abstract—The new relationship between users and assets
through mobile service is changing the way to deliver services
by smart devices. The research developed for the project
SCUOLA – Smart Campus as Urban Open LAbs introduced
the idea of creating an app to include the users’ feedback into
the information chain of a pilot smart building. The paper
describe the early-stage of implementation and evolution of this
concept, which define a dialogue between building and users, by

realizing a bi-directional interaction via a newly developed
mobile application. The app – Smart Campus UNIBS mediates, as part of a complex system, between the two entities
using the connectivity of Smart Living and the process of Data
Analytics. The Smart Campus Demonstrator building at the
University of Brescia, Italy, is equipped with sensors to monitor
and control comfort, indoor air quality and HVAC parameters,
such as hygro-thermal parameters, illuminance, CO2 and
volatile organic compound (VOC), power of HVAC fans and
environmental external factors. The sensors aim at providing
data to develop adaptive, dynamic as well as predictive controls
virtually incrementing the smartness of the building. The step
further is to include the behavioral perspective linking the users
to the previous framework. By exchanging different kinds of
processed data, including BIM models, sensors and user
feedbacks, it is possible to achieve an interaction between the
built environment and the social landscape. The Smart Campus
Demonstrator building integrates the smart concepts involved
in the SCUOLA research project and beyond. It has been
chosen to reveal the potential of such reciprocal interchange
between humans and constructions towards the extended smart

city. The overall purpose is ultimately that of creating a
prototype for adaptive systems in architecture i.e. “Cognitive
Buildings” which can learn by users’ behavior.
Keywords—bi-directional app; users-responsive
behavioral servitization; cognitive building, smart cities

I.

building;

INTRODUCTION

There is no question that the indoor built environment plays
a critical role in the users overall well-being. People spend
about 90% of the time indoors, and buildings have a unique
ability to influence the health positively or negatively at level
This research activity has been partially funded by regional founding provided
by Regione Lombardia under Smart cities and communities grant no.
40545387 (“Smart Campus as Urban Open Labs – SCUOLA”).

of cognitive performance. Nevertheless, the interaction
between human and building is no more conceived as a single
directed flux and the user can here and now interact with the
building to improve the offered conditions and enhance the
performance of the building and his perceived capabilities to
act informed choices [1].
Smart buildings organized in smart district compose the
smart city, which develops and live on a network of
information, materials, energy and people moving between
physical places and virtual inputs [2]. Nowadays, the built
environment is enriched by information and services to address
the user to marketing choices and task. Objects, mobile apps
and devices are permeating the everyday life becoming the first
source of information from which to get an opinion to support
the decision process, preferably reinforced by a plump
community of reviews. The track of users’ preferences is a
market driver [3] and the advised opinion and the related
outcome strategy constructed over the past decades a huge
amount of data in need to be mined to extract knowledge.
The rise of big data [4] and analytics introduced the use of

the cloud, mobile, social media and the Internet of Things (IoT)
[5] also applied to the built environment which is on the other
hand became a producer and a consumer of information
through monitoring and sensing [6]. The building interaction
with control systems and data collection evolved towards the
progressive thresholds of efficiency [7] [8].
Smart technologies represented the most radical shift in
architectural practice and a main driver is the huge potential
energy savings, estimated in a range between 20% and 50% as
a staggering amount of energy is wasted on heating empty
offices, homes and partially occupied buildings [9]. The
possibility to track the users can define customized operations
in which the building measures the number of people inside
and adjusts heating and lighting accordingly, with a view to
turning an empty building "off", as a computer goes into
standby mode. A further concept is the localized heating and
cooling systems, which can provide a detailed, individual
climate for each user by means of arrays of responsive infrared
heating elements that are guided by sophisticated motion

tracking providing thermal “clouds”, following people through
space and ensuring pervasive comfort whereas improving
overall energy efficiency [10].
It is possible to define four level of interaction in the path
leading the evolution of the building sector until the 4.0:
1. First stage: It is aimed at improving energy efficiency
using Building Automation Controls (BAC) in the building
automated environment;
2. Second stage: It was oriented in enhancing operations
and based on Building Management System (BMS) for smart
building;
3. Third stage: It is the predictive building which
anticipates the occupancy needs and set itself to face
environmental and behavioral inputs using Information and
communications technology (ICT) to support managers and
operators;
4. Fourth stage: It is the cognitive building which learns
from the users’ behavior and traduces the data coming from
the outdoor, the indoor and the social environment using an
Internet of things (IoT) approach to reset in time the

responsiveness, making the building autonomous to react in
some situation.
The IoT makes possible a bi-directional interaction, which
is the core interest of the present research, giving the possibility
to a user to have his behaviors and needs directly involved into
the measure and control loop. The Smart Campus
Demonstrator building of the University of Brescia becomes
the core of an experimental bi-directional relation between the
user and the building through the Building Information
Modeling (BIM) [11] (Fig. 1).

II. IOT VISION FOR SMART AND COGNITIVE CITY
In the initial stage of the Internet, millions of people got
connected and an economic value of about trillions of dollars
grew through various new services. In the next stage of the
Internet, billions of Things will get connected and estimations
give 212 billions of connected devices by 2020.
The IoT application domains include mostly all known
ones and empower the vision of a built environment pervaded
by sensors and actuators in which homes do not waste energy,

where interactive walls display useful information, as well as
pictures of art, videos of friends. The smart city is made by
productive business environment where offices turn into smart
and interactive assets; factories relay real-time production data;
face-to-face meetings are established through holograms and
documents are fully integrated in the workflow [13]. The Fig. 2
is a conceptual map that shows the strict relationships between
Smart City and IoT concepts.

Fig. 2. IoT framework for the smart cities.

In this future city, the IoT technologies enhance the
productive areas, retail, residential and green spaces which
collaborate and efficient logistics environment embeds safety
and environmental concerns all over the process. In the concept
of cognitive city, the environment learns inputs from users and
promotes smart health, nonintrusive monitoring system,
preventing serious illness by adjusting the environment and
selecting appropriate drugs and diet based on food information
and user preferences and needs [14].

Fig. 1. Scheme of the bidirectional intercation between Smart Campus users
introduced in the IoT loop.

The BIM is the framework and repository of all information
coming from both sides. The building is equipped with sensors
[12] connected with the BIM and the users can provide their
feedback through an app to enhance and customize the
operations.

The Intelligent Transportation Systems integrates public
and private transportation, choosing the best path to avoid
delays and congestions, and promotes multimodal transport
providing a tailored experience based on users’ tracked
behaviors. As well as the ultimate retail environment supports
consumers to have a healthy and suitable shopping experience,
with a complete traceability of products.
III. COGNITIVE BUILDING AND INTERACTION DESIGN
The vision of cognitive buildings is connected with
interaction design. Cognitive buildings operate proactively

with the human activity within them [14]. The interchange
happens via data collection and data processing based on

advanced learning technologies. As the environment affects the
human activity (e.g. in a school building, students’ learning
performance diminishes if air quality degrade as CO2
concentration grows [15]), a cognitive building tries to
implement rules for different scenarios and react respectively
with outcome on energy consumption [16]. The data coming
from sensors installed into the building define the building
scenarios given by changing conditions. In the scenario of the
school, when a sensor gives feedback that the air quality in the
classroom is getting worse, ventilation will be triggered and the
room will improve the users’ comfort and health and define the
energy use of the building [17]. Cognitive buildings are a
future implementation of smart building equipped with sensors
through IoT paradigm and cognitive technologies. The bidirectional interaction is described by the input and output
given by the building and the users and the interchange of
command of actuation derived by users feedback. The
feedback about indoor conditions (e.g. thermal, acoustic, visual

comfort) and the behavioral data (localization and occupancy
rate) are communicated by app to the management system. The
BIM model and the sensors connected provide numerical
information about the indoor conditions however the comfort
can be tested by the users feedback. The feedback is thus the
way to include the user as a sensor of perception to teach to the
building a management routine that can be adjusted
dynamically (Fig. 3, Fig. 4).

vast amounts of data. Accordingly, Uncover patterns and
opportunities can be exploited as would be virtually impossible
through traditional research methods.

Fig. 4 Cognitive Building Concept.

IV. SCUOLA PROJECT: SMART CAMPUS AS URBAN
OPEN LAB

Fig. 3. Workflow of the user interaction by means of a Smartphone
Application.

The real competitive value comes by taking advantage of
digital capabilities with the power of cognitive computing.
Cognitive systems, such as IBM Watson™, the Q&A system
available from International Business Machines (IBM)
Corporation of Armonk, N.Y., which analyzes unstructured
textual content of electronic documents to answer questions
and derive conclusions from the textual content [18]. These
systems are an application of advanced natural language
processing (NLP), information harvesting, knowledge
representation and reasoning, and machine learning
technologies to the field of open domain question answering.
Cognitive systems understand the world by interaction, reason
by generating recommendations and hypotheses, and learn
from experts and from data and enriching by users’ interaction
and data ingestion. Cognitive systems never stop learning as
they interact with humans better than other machines because
do not rely solely on data, which is structured, such as a user’s
transaction history or geolocation, to mine deeper insights from

The SCUOLA project aims to test an integrated control and
monitoring system of the electricity flows into the university
campus, starting from renewable sources generation and
accumulation plants to various types of electrical loads [19].
The experimentation has involved several plants, both into the
University of Brescia and the Politecnico di Milano campuses
in Italy, ranging from photovoltaic fields with accumulation to
electrical vehicles recharging stations. All this plants are
coordinated, in conjunction with the energy distribution system
operator, by a distributed energy management system that
make use of advance models, both for production and load
prediction. The task is to identify the optimum point of work of
the overall system in terms of costs, occupants’ comfort or
energy savings according to both the system status and the
operational plan. Also focusing only on the University of
Brescia demonstrator, the SCUOLA projects makes available a
spread set of sensors making the building a step ahead towards
the concept of a cognitive building.
In Fig. 5, a conceptual scheme of the pilot Smart Campus
building is reported.

Fig. 5. Building systems as static IoT oriented system.

As shown in Fig. 5, sensors, actuators and new plants have
been added to the building: new photovoltaic plant (equipped
with a solar lab, used for testing PV panels) and a weather
station on the roof; smart plugs, environmental sensors (CO2,
VOC, temperature, humidity); a people counting system,
webcams suitable for human detection and flow recording
algorithms; electrical meters into the rooms; electrical vehicles
recharging station into the car park and informational totems
into the lobbies to share the systems’ status. The overall system
has been developed as an IoT ready system and its adherence
to an IoT framework has been already asserted in a previous
work [12], using the weather station as example of integration.
V. THE DEVELOPED APP: PUTTING PEOPLE INTO THE
IOT LOOP
The Smart Campus Demonstrator building represents the
first attempt and pilot project of University of Brescia, Italy, to
implement a cognitive system in one of its main buildings
located in the Campus [20]. To fulfill the objective on a series
of different typologies of sensors, which monitor the status of
the building related to indoor comfort and aiming at energy
saving are collecting data. The first phase of the users’
interaction in the SCUOLA project has been developed by the
setting of an app for the feedback of the students on comfort
level in the classrooms of the pilot building that is ongoing.
University of Brescia aims now at the creation of a mobile
application to give users the possibility not only to view the
data from the sensors but also to interact with the building itself
via feedback. The goal is mainly to promote the vision of the
cognitive building, which adapts its behavior and setting to the
users’ needs learned from their behaviors communicated via
the app Fig. 6. The data collected by the sensors are included
into the Building Information Model (BIM) of the building
providing a data mapping able to introduce thresholds of
comfort or indoor air quality on which to manage the building
setting (e.g. temperature set-points, ventilation rate,
illuminance, etc.). This data can also be used to create realtime synoptic charts to allow an easy access to the building
status or to be used to tune the control of the building
automation system (e.g. lighting systems, heating ventilation
and air conditioning system, etc.).

Fig. 6. The App allowes the users and visitors to- check of real-time data on
the building and to provide the feedback on indoor conditions.

The bi-directional interaction through the app embodies the
introduction of the human factor in the IoT structure to enable
the cognitive building to learn from behaviors providing data in
real-time with the capability to process them into adaptive and
predictive strategies for improved comfort and servitization.
With the app the users become heterogeneous mobile sensors
that reveals perceptions and gives input defining a dynamic
“learnscape” for the cognitive building. The integration of
“human sensors” into the system illustrated in section IV
changes the scenario from a static IoT ready system to a true
dynamic IoT application, in which the building is continuously
reconfiguring itself (or a single sub-system) following the
users’ behaviors.
The BIM is the heart of the system: all the physical object
of the building can be digitally represented and correlated with
the operational data to improve the performance of the
building, by means of cognitive computing as the one offered
by the IBM Watson™ suite.
The developed application is an interface between the
Smart Campus Demonstrator building and its users, it has an
interaction design that follows distinctive rules and creates
explicit responses to determined scenarios, identified as the
following.
A. Inform
Before a user goes to the building, he can get information
about the building via the mobile app looking up for instance
the energy performance and the energy production of the
photovoltaic panels on the roof. In this part, the user will be
able to get moreover information related to the idea of a
sustainable smart campus growing his awareness and
consciousness in the energy and environmental topics related to
the campus (i.e. education and dissemination purposes).
B. Check
On the way to the building, users are invited to use their
mobile devices as a tool of room locator. They can find the
room where they will have the lecture in an interactive map of
the building. Furthermore, users are also encouraged to check
the comfort conditions and occupancy of the rooms. They will
get live data from the sensors about temperature, illumination,
acoustics, air quality, occupancy and a few more in order to
assist in the choice for an adequate workspace supported by the
VE (Virtual Entity) defined by the BIM.

C. Report
During the visit to the building, users will be provided with
chance to send via the app their feedback regarding the
usability but foremost their perception of comfort. These data
will be collected and enriched with metadata to promote a data
mining process in order to optimize the building performance.
Data could also be used to adapt the building behavior, by
developing a suitable control logic for the building automation
system.
VI. BENEFITS OF THE INTERACTIVE APP
Being a system based on bi-directional interaction, the
developed app will offer benefits to both the users as well as
optimize issues related to the building management. Users will
primary benefit from improved comfort, which is defined by
thermal, visual, olfactory and acoustic aspects. By controlling
and monitoring temperature, illumination, air quality, and
acoustics it will be ultimately possible to create a more healthy
and productive environment. The application will simplify
moreover the accessibility to occupancy related information
and help users to get real-time update about indoor conditions
and the will improve the building energy efficiency (Fig. 7).

VII. CONCLUSION
The IoT applied to the buildings gives the opportunity to
increase the responsiveness of smart buildings and to move
towards the cognitive building concept, which promote a built
environment able to connect the data to the users’ need and
requirements changing during the lifespan of the smart city.
The environmental and social networks created by the
buildings and the users are overlapped and the behaviors
revealing and tracking could promote the optimization and
interconnection of resources in a vision of the circular
economy. The research project led by University of Brescia
aims at define a new relationship between built environment
and community by designing a bi-directional interaction. The
design of the Smart Campus UNIBS Bi-Directional App
applied to the Smart Campus Demonstrator building
establishes in the center of an out most contemporary discourse
of smart city. It is an excellent field for experimenting and a
great opportunity to promote innovative ways of redesign the
way of thinking about the user. The human behavior becomes a
node introduced as active and dynamic actuator of building
operation through the IoT loop and the potential of outcome are
growing by cognitive computing technologies.
ACKNOWLEDGMENT
The authors would like to mention and acknowledge the
Smart Campus School Project Team Leader Prof. Alessandra
Flammini for the kind availability of the design material and
useful discussion about the strategies. Special thanks go Eng.
Daniela Pasini and Silvia Mastrolembo Ventura for their
valuable collaboration.
REFERENCES
[1]

Fig. 7. The App allows check the indoor condition and occupancy data in the
buidling spaces.
[2]

Concerning the building, it will be possible to monitor
constantly the energy demand and supply as well as the
occupancy level and systems operating activities or the need of
maintenance processes. In the first phase, this information will
be used to optimize the performance of the technical
equipment, in the future the sensors would be directly
transferring the data to actuators [21] that adjust the building in
and out fluxes automatically for energy efficiency. The data
collection will be also crucial to tune the BEM to manage the
life cycle energy enhancement of the building by the
University of Brescia.
A supplementary benefit is thus moreover in the improved
transparency by means of the BIM, which will connect the
feedback to failure detection procedures. This will enable the
maintenance staff to experience actually the augmented reality
improving the process of problems finding and eases to solve
them in a more efficient way. By improving the continuous
maintenance the aim is even to extend the building lifecycle,
enhancing management for the client, reducing the costs due to
emergency interventions.

[3]
[4]
[5]

[6]

[7]

[8]

[9]

H. Schaffers, N. Komninos, M. Pallot, B. Trousse, M. Nilsson, A.
Oliveira, “Smart Cities and the Future Internet: Towards Cooperation
Frameworks for Open Innovation”,. Future internet assembly, vol. 6656,
no. 31, pp. 431-446, 2011.
H. Chourabi, T. Nam, S. Walker, J.R. Gil-Garcia, S. Mellouli, K. Nahon,
H. J. Scholl, H. J. (2012, January). “Understanding smart cities: An
integrative framework”, in Proc. of IEEE International Conference on
System Science (HICSS), Hawaii, USA, pp. 2289-2297, Jan. 2012.
J. Belissent, Getting clever about smart cities: new opportunities require
new business models, 2010.
R. Kitchin, The real-time city? Big data and smart urbanism.
GeoJournal, vol. 79, no. 1, pp. 1-14, 2014.
A. Zanella, N. Bui, A. Castellani, L. Vangelista, M. Zorzi, “Internet of
things for smart cities”, Internet of Things Journal, IEEE,vol .1, no.1,
pp. 22-32, 2014.
C. Perera, A. Zaslavsky, P. Christen, D. Georgakopoulos, “Sensing as a
service model for smart cities supported by internet of things”,
Transactions on Emerging Telecommunications Technologies, vol. 25,
no. 1, pp. 81-93, 2014.
P. Lombardi, S. Giordano, H. Farouh, W. Yousef, “Modelling the smart
city performance”, Innovation: The European Journal of Social Science
Research, vol .25, no. 2, pp. 137-149, 2012.
J. H. Lee, M. G. Hancock, M. C. Hu, “Towards an effective framework
for building smart cities: Lessons from Seoul and San Francisco”,
Technological Forecasting and Social Change, vol. 89, pp. 80-99, 2014.
C. Martani, D. Lee, P. Robinson, R. Britter, C. Ratti, “ENERNET:
Studying the dynamic relationship between building occupancy and
energy consumption”, Energy and Buildings, vol. 47, pp. 584–591,
2012.

[10] C. Ratti, “Sprawling circular innovation. Big data to open data,
Intelligent Assets: Unlocking the Circular Economy Potential”, vol. 61,
U.S.A.: Ellen Macarthur Foundation, 2016
[11] E. De Angelis, A. L. C. Ciribini, L. C. Tagliabue, M. Paneroni, “The
Brescia Smart Campus Demonstrator. Renovation toward a zero energy
classroom building”, Procedia Engineering, vol. 118, pp. 735-743, 2015.
[12] P. Bellagente, P. Ferrari, A. Flammini, S. Rinaldi, “Adopting IoT
framework for Energy Management of Smart Building: A real testcase”, in Proc. of International Forum on IEEE Research and
Technologies for Society and Industry Leveraging a better tomorrow
(RTSI), Italy, pp. 138-143, Sept. 2015.
[13] L. G. Anthopoulos, A. Vakali, “Urban planning and smart cities:
Interrelations and reciprocities. In The Future Internet”, pp. 178-189,
Springer Berlin Heidelberg, 2012.
[14] A. Bassi, “Application Domains for Internet of Things”, in Proc. of 4th
Use Case Workshop, Brussels, BE, March 20, 2013.
[15] L. Chatzidiakou, D. Mumovic, J. Dockrell, “The Effects of Thermal
Conditions and Indoor Air Quality on Health, Comfort and Cognitive
Performance of Students”, The Bartlett, UCL Faculty of the Built
Environment UCL Institute for Environmental Design and Engineering
London, October 2014.
[16] L. C. Tagliabue, M. Manfren, E. De Angelis, “Energy Efficiency
Assessment Based on Realistic Occupancy Patterns Obtained Through

[17]

[18]

[19]

[20]

[21]

Stochastic Simulation”, In Modelling Behaviour, pp. 469-478, Springer
International Publishing, 2015.
O. A Sianaki, O. Hussain, T. Dillon, A.R. Tabesh, “Intelligent Decision
Support System for Including Consumers' Preferences in Residential
Energy Consumption in Smart Grid”, in Proc. of International
Conference on Computational Intelligence, Modelling and Simulation
(CIMSiM), 28-30 Sept. 2010, Bali, pp. 154–159.
S. N. Gerard, M. G. Megerian. "Extracting semantic relationships from
table structures in electronic documents", U.S. Patent No. 8,914,419. 16
Dec. 2014.
S. Rinaldi, P. Ferrari, A. Flammini, N. Ali, F. Gringoli, "IEC 61850 for
micro grid automation over heterogeneous network: Requirements and
real case deployment", in proc. of IEEE International Conference on
Industrial Informatics, Cambridge, UK, July 22-24, 2015, pp. 923-930.
E. De Angelis, A. L. C. Ciribini, L. C. Tagliabue, M. Paneroni, “The
Brescia Smart Campus Demonstrator. Renovation toward a zero energy
classroom building”, Procedia Engineering, vol. 118, pp. 735-743, 2015.
J. A. Momoh, “Smart grid design for efficient and flexible power
networks operation and control”, in Proc. of IEEE Power Systems
Conference and Exposition, 15-18 March 2009, Seattle, USA, pp. 1–8.