Big Data Approach Paper
BIG DATA – Insights
Synopsis
People can only be classified by the data they generate in a limited number of
ways. It is upto the data analyst to gather as much data as possible, and by
examining vast amount of information in the form of data available from diverse
sources, uncover hidden patterns, unknown correlations and other useful
information that can be used to make a better decision.
This unceasing data has changed forever how the business operates. Many
businesses are looking to it to tell them more about their customers, their buying
habits, and the likelihood they’ll behave in certain ways under certain
circumstances.
Most of the senior executives, consider leveraging big data to be a top strategic
priority, as they seek to gain customer insights, effect more accurate budgeting
and better performance management, and develop new products.
Huge gap exists between what organization hope to accomplish and what they’re
able to, given their existing IT infrastructure & expertise. This opens doors to all
kind of new risks.
The big data is not to be considered as a single set of data, but it is the way data
grow and when there is some kind realization in the ways that might be able to
connect different sets of data together to create even more sets of information.
In a nut shell, Big data represents the ever – expanding collection of data sets
that sheer size, variety and speed of generation make difficult to manage and
harness information from.
The source of these electronics data are varied, being as simple as transactions
in a banking system, or as complex as unstructured content such as Facebook
pictures and videos.
One thing that experts world – wide agree upon is that big data is not simply
data mining on a greater scale, it can be thought of more as a survey or
prospecting. On the other hand data mining is conducted on structured data from
a limited set of sources.
Questions Boards Should Ask
■Does the company have the talent and technological capabilities needed to
collect, manage, analyze, and store big data securely and effectively?
■Does the company have an established policy that addresses the ethical
considerations of collecting, analysing, and using sensitive information? Is the
policy well known?
■Does collecting and using big data subject the company to any additional
compliance risks?
■What is the company’s strategy for using the data it collects?
■Where is the data stored? Are third parties used? If so, how are the risks
collectively managed?
1
BIG DATA – Insights
■Is the chief information officer able to explain effectively — in layman’s terms —
what big data is, how it is stored, what the risks are, and how the risks are
managed?
Why invest in big data business?
Big data business is a very diverse field, representing an interdependent
ecosystem. In layman language, it can be divided into three specific categories:
Data Users: Use data for internal purpose
Data Suppliers: Supply data as a product for others to use
Data Facilitator: Are organizations that help others to exploit data
This includes businesses that provide outsourced analytics services and
those offering either infrastructure or consultancy on data strategy.
This function is especially important during the transition phase because
many potential users of big data do not have the skills to implement a big
data strategy internally.
Many business models focus on the use of data within and between businesses.
However, as consumers are themselves increasingly faced with an abundance of
data there is a growing market for businesses that provide data-driven products
and services to end users.
The growing market for personal health and fitness devices, along with
smart home technologies are pro-typical examples.
The whole spectrum of business models also apply in the consumer-facing
segment: consumers will increasingly demand data analysis tools and
services, data focused products, and help and advice for managing data
their challenges.
Various practical and political hassles of the business?
The most commonly cited obstacles can be essentially divided into two
categories: practical obstacles concerning data availability and quality, along
with the necessary resources for analysis; and political obstacles that shape the
environment in which data is used.
An important practical obstacle is the quality of data sets. Some experts
say analysts spend as much as 90% of their time cleaning data. Data,
especially government data, is often provided in non-machine readable or
non-standardised formats requiring manual re-entry.
Past experience highlights the importance of being forward-looking in
anticipating future uses (and users) of data. Legacy datasets that were not
stored with appropriate human-readable metadata are now essentially
useless because nobody knows what the data mean. A similar point holds
for the format and physical infrastructure in which data is stored.
Although technology has revolutionised data availability, there are still
problems in providing access to data to those in an organisation who are
best placed to exploit it. Tools to facilitate the use of data by non-
2
BIG DATA – Insights
specialists are an exciting prospect, but are still not mature enough to
solve the problem.
A key political barrier to data use is the extent to which people are
protective of ‘their’ data. This often applies to a reluctance to share data
within an organisation as much as to an unwillingness to share data
between organisations, and speaks to the need for an organisation-wide
policy and strategy for data use.
There is widespread appreciation of the importance of privacy, but
managers bemoan the lack of standards and clear policy guidance in this
area.
For achieving success using big data, a well-defined business model
needs to be prepared centred on big data?
Data should be central to the business. The biggest success stories have either
essentially reinvented their entire business around the use of data or are ‘born’
data users.
A clear profit model is essential. Experts warn that optimistically collecting data
in the hope that it will somehow prove profitable is naïve. Managers and data
scientists should be clear on the plan for generating value or efficiency from data
before the data strategy is implemented.
The most successful firms understand the limitations of the technology behind
their big data operation and recognise the importance of combining analysis with
a sound understanding of the context, a good intuition for the industry, and a
critical attitude towards insights derived from data.
Having the right skills is crucial. Very few individual possess the right
combination of statistics, coding, and business skills, so cultivating talent in
teams is essential.
Global Big Data Survey Trends:
In the past, databases tended to be limited—they only had to meet the demands
of human users entering and retrieving data. With the emergence of ecommerce
and internet search engines, database technology is evolving to manage humans
and computers. Today, with the amount of data growing by 50 percent each year,
it is information technology that is capable of managing, processing, and finding
value.
Key Survey Highlights:
Data overload and the abundance of trivial information are challenges
many organizations face. There are a lot of data in general but also a lack
useful data. High-level data are available, but not the detailed data
essential too many decisions, plans, tasks, and functions.
Important data are not reaching practitioners in efficient timeframes. Fifty
percent of respondents report there is an undesirable delay in receiving
information about actual sales, demand forecasts, customer changes and
orders, and materials or component shipment status.
Despite current systems, data are still not always easily accessible. There
may be access restrictions, support limitations, security requirements, or
out-of-sight-out of-mind challenges. Another example is that physical or
3
BIG DATA – Insights
paper records are used in many cases. Separate databases serving
departments or partners in other nations may be used. Access to this data
may depend on factors practitioners are not aware of, such as availability
or security.
Current information technology has not yet delivered optimal satisfaction
in terms of what is easily measurable, reportable, or quantifiable data such
as scheduling, inventory levels, and customer demand across the supply
chain. Survey results suggest possible reasons, such as different data
formats and systems, and timeliness and data access challenges. This
may challenge processes such as sales and operations planning (S&OP)
that seek a shared and integrated understanding of supply and demand.
Supply chain dataflow includes direct suppliers and customers, but there
are gaps in a complete, or true end-to-end supply chain dataflow model for
most respondents. For example, only about half of survey respondents
report that logistics, distributors, and minor direct suppliers are part of
their supply chain dataflow. True end-to-end supply chain dataflow,
including suppliers’ suppliers and customers’ customers, remains a
challenge. Supply chain and operations management professionals have
more work to do in terms of improving tools, technologies, strategies, and
relationships, and big data can play a major role in this progress.
Big data is an up-and-coming skill and knowledge area. It may pay to make big
data a professional development priority because the expertise will be rare and
in demand. In addition, big data will probably be implemented in long-ranging
projects in increments as technology and resources permit. If big data were to
resemble past ERP implementations, practitioners can help develop project
management skills and high-level implementation ideas now and share those
with management teams. This interesting and thoughtful approach may position
a professional to advance in his or her career, improve a team’s performance, or
gain strategic knowledge of an organization’s supply chain and operations
management environment.
Big data best practices:
Developing good relationships with supply chain partners. Good relations are
necessary to facilitate shared data and insight. Building a foundation with these
relations in place, before one begins asking partners for advanced data sharing.
Addressing supply chain dataflow gaps where possible as soon as possible.
Organization may have developed procedures or practices that make such gaps
manageable, but are not as good as they could be. As powerful as the promise of
big data is, it probably can’t correct processes and procedures built around
current gaps.
Looking for areas of correlation that don’t seem obvious now, or seem
too complex to study at this time. For example, if demand decreases at times
when it is expected to increase and there isn’t a clear explanation, this is a good
assignment for big data systems. A new big data implementation may not
otherwise know what has high priority to the organization in terms of correlation
and investigation.
Implementing Data Analytics in Internal Audit
4
BIG DATA – Insights
Today’s Internal Audit environment demands audit departments to “do more,
with less”. A formal data analytics program can go a long way in helping an audit
function become more efficient, easily scalable and significantly reduce auditing
errors all while providing greater audit and fraud risk coverage. Data analytics
programs can provide long term continuous auditing or monitoring around legal
and compliance issues as well as the ability to do ad hoc audit testing, business
review and fraud investigations. Data analytics is the cornerstone for fraud
detection programs in many organizations, providing audit and fraud teams to
immediately and easily identify potential fraud without having to manually
“sample” large quantities in order to detect fraud and fraud patterns.
5
Synopsis
People can only be classified by the data they generate in a limited number of
ways. It is upto the data analyst to gather as much data as possible, and by
examining vast amount of information in the form of data available from diverse
sources, uncover hidden patterns, unknown correlations and other useful
information that can be used to make a better decision.
This unceasing data has changed forever how the business operates. Many
businesses are looking to it to tell them more about their customers, their buying
habits, and the likelihood they’ll behave in certain ways under certain
circumstances.
Most of the senior executives, consider leveraging big data to be a top strategic
priority, as they seek to gain customer insights, effect more accurate budgeting
and better performance management, and develop new products.
Huge gap exists between what organization hope to accomplish and what they’re
able to, given their existing IT infrastructure & expertise. This opens doors to all
kind of new risks.
The big data is not to be considered as a single set of data, but it is the way data
grow and when there is some kind realization in the ways that might be able to
connect different sets of data together to create even more sets of information.
In a nut shell, Big data represents the ever – expanding collection of data sets
that sheer size, variety and speed of generation make difficult to manage and
harness information from.
The source of these electronics data are varied, being as simple as transactions
in a banking system, or as complex as unstructured content such as Facebook
pictures and videos.
One thing that experts world – wide agree upon is that big data is not simply
data mining on a greater scale, it can be thought of more as a survey or
prospecting. On the other hand data mining is conducted on structured data from
a limited set of sources.
Questions Boards Should Ask
■Does the company have the talent and technological capabilities needed to
collect, manage, analyze, and store big data securely and effectively?
■Does the company have an established policy that addresses the ethical
considerations of collecting, analysing, and using sensitive information? Is the
policy well known?
■Does collecting and using big data subject the company to any additional
compliance risks?
■What is the company’s strategy for using the data it collects?
■Where is the data stored? Are third parties used? If so, how are the risks
collectively managed?
1
BIG DATA – Insights
■Is the chief information officer able to explain effectively — in layman’s terms —
what big data is, how it is stored, what the risks are, and how the risks are
managed?
Why invest in big data business?
Big data business is a very diverse field, representing an interdependent
ecosystem. In layman language, it can be divided into three specific categories:
Data Users: Use data for internal purpose
Data Suppliers: Supply data as a product for others to use
Data Facilitator: Are organizations that help others to exploit data
This includes businesses that provide outsourced analytics services and
those offering either infrastructure or consultancy on data strategy.
This function is especially important during the transition phase because
many potential users of big data do not have the skills to implement a big
data strategy internally.
Many business models focus on the use of data within and between businesses.
However, as consumers are themselves increasingly faced with an abundance of
data there is a growing market for businesses that provide data-driven products
and services to end users.
The growing market for personal health and fitness devices, along with
smart home technologies are pro-typical examples.
The whole spectrum of business models also apply in the consumer-facing
segment: consumers will increasingly demand data analysis tools and
services, data focused products, and help and advice for managing data
their challenges.
Various practical and political hassles of the business?
The most commonly cited obstacles can be essentially divided into two
categories: practical obstacles concerning data availability and quality, along
with the necessary resources for analysis; and political obstacles that shape the
environment in which data is used.
An important practical obstacle is the quality of data sets. Some experts
say analysts spend as much as 90% of their time cleaning data. Data,
especially government data, is often provided in non-machine readable or
non-standardised formats requiring manual re-entry.
Past experience highlights the importance of being forward-looking in
anticipating future uses (and users) of data. Legacy datasets that were not
stored with appropriate human-readable metadata are now essentially
useless because nobody knows what the data mean. A similar point holds
for the format and physical infrastructure in which data is stored.
Although technology has revolutionised data availability, there are still
problems in providing access to data to those in an organisation who are
best placed to exploit it. Tools to facilitate the use of data by non-
2
BIG DATA – Insights
specialists are an exciting prospect, but are still not mature enough to
solve the problem.
A key political barrier to data use is the extent to which people are
protective of ‘their’ data. This often applies to a reluctance to share data
within an organisation as much as to an unwillingness to share data
between organisations, and speaks to the need for an organisation-wide
policy and strategy for data use.
There is widespread appreciation of the importance of privacy, but
managers bemoan the lack of standards and clear policy guidance in this
area.
For achieving success using big data, a well-defined business model
needs to be prepared centred on big data?
Data should be central to the business. The biggest success stories have either
essentially reinvented their entire business around the use of data or are ‘born’
data users.
A clear profit model is essential. Experts warn that optimistically collecting data
in the hope that it will somehow prove profitable is naïve. Managers and data
scientists should be clear on the plan for generating value or efficiency from data
before the data strategy is implemented.
The most successful firms understand the limitations of the technology behind
their big data operation and recognise the importance of combining analysis with
a sound understanding of the context, a good intuition for the industry, and a
critical attitude towards insights derived from data.
Having the right skills is crucial. Very few individual possess the right
combination of statistics, coding, and business skills, so cultivating talent in
teams is essential.
Global Big Data Survey Trends:
In the past, databases tended to be limited—they only had to meet the demands
of human users entering and retrieving data. With the emergence of ecommerce
and internet search engines, database technology is evolving to manage humans
and computers. Today, with the amount of data growing by 50 percent each year,
it is information technology that is capable of managing, processing, and finding
value.
Key Survey Highlights:
Data overload and the abundance of trivial information are challenges
many organizations face. There are a lot of data in general but also a lack
useful data. High-level data are available, but not the detailed data
essential too many decisions, plans, tasks, and functions.
Important data are not reaching practitioners in efficient timeframes. Fifty
percent of respondents report there is an undesirable delay in receiving
information about actual sales, demand forecasts, customer changes and
orders, and materials or component shipment status.
Despite current systems, data are still not always easily accessible. There
may be access restrictions, support limitations, security requirements, or
out-of-sight-out of-mind challenges. Another example is that physical or
3
BIG DATA – Insights
paper records are used in many cases. Separate databases serving
departments or partners in other nations may be used. Access to this data
may depend on factors practitioners are not aware of, such as availability
or security.
Current information technology has not yet delivered optimal satisfaction
in terms of what is easily measurable, reportable, or quantifiable data such
as scheduling, inventory levels, and customer demand across the supply
chain. Survey results suggest possible reasons, such as different data
formats and systems, and timeliness and data access challenges. This
may challenge processes such as sales and operations planning (S&OP)
that seek a shared and integrated understanding of supply and demand.
Supply chain dataflow includes direct suppliers and customers, but there
are gaps in a complete, or true end-to-end supply chain dataflow model for
most respondents. For example, only about half of survey respondents
report that logistics, distributors, and minor direct suppliers are part of
their supply chain dataflow. True end-to-end supply chain dataflow,
including suppliers’ suppliers and customers’ customers, remains a
challenge. Supply chain and operations management professionals have
more work to do in terms of improving tools, technologies, strategies, and
relationships, and big data can play a major role in this progress.
Big data is an up-and-coming skill and knowledge area. It may pay to make big
data a professional development priority because the expertise will be rare and
in demand. In addition, big data will probably be implemented in long-ranging
projects in increments as technology and resources permit. If big data were to
resemble past ERP implementations, practitioners can help develop project
management skills and high-level implementation ideas now and share those
with management teams. This interesting and thoughtful approach may position
a professional to advance in his or her career, improve a team’s performance, or
gain strategic knowledge of an organization’s supply chain and operations
management environment.
Big data best practices:
Developing good relationships with supply chain partners. Good relations are
necessary to facilitate shared data and insight. Building a foundation with these
relations in place, before one begins asking partners for advanced data sharing.
Addressing supply chain dataflow gaps where possible as soon as possible.
Organization may have developed procedures or practices that make such gaps
manageable, but are not as good as they could be. As powerful as the promise of
big data is, it probably can’t correct processes and procedures built around
current gaps.
Looking for areas of correlation that don’t seem obvious now, or seem
too complex to study at this time. For example, if demand decreases at times
when it is expected to increase and there isn’t a clear explanation, this is a good
assignment for big data systems. A new big data implementation may not
otherwise know what has high priority to the organization in terms of correlation
and investigation.
Implementing Data Analytics in Internal Audit
4
BIG DATA – Insights
Today’s Internal Audit environment demands audit departments to “do more,
with less”. A formal data analytics program can go a long way in helping an audit
function become more efficient, easily scalable and significantly reduce auditing
errors all while providing greater audit and fraud risk coverage. Data analytics
programs can provide long term continuous auditing or monitoring around legal
and compliance issues as well as the ability to do ad hoc audit testing, business
review and fraud investigations. Data analytics is the cornerstone for fraud
detection programs in many organizations, providing audit and fraud teams to
immediately and easily identify potential fraud without having to manually
“sample” large quantities in order to detect fraud and fraud patterns.
5