Introduction

Business intelligence (BI) systems provide the businesses with a tool, which turns raw data into useful insights and data, which can be used to strategize the marketing efforts and generate revenues. With increasing competition, ever changing market trends and influences of various eternal factors the organization are required to be highly responsive and address the challenges and uncertainties to sustain in the market, which requires ability to have increased information about all the aspects of the business. The organizations are required to collect, store and process variety of data to understand the complexity of the market, need of market and utilize the information to deliver quality service and products to the customers. Big data is one such system, which includes technologies to capture, store, share, processes, transfer and secure variety of huge data continuously being uploaded, shared or retrieved from different related sources, which can be processed in manner that it will be of great use to the businesses to foresee threats and opportunities in the market and have a competitive advantage (Wu et al., 2013). The report discusses the significance of Big data to businesses, implementation and usage of Big data in business operations and the issues faced in implementation of Big Data.

Business Intelligence

Todays, business environment continues to be complex and needs systems, which can process the information faster and enable the organization to make decisions faster and with greater accuracy. Business intelligence (BI) systems have become the most important component of modern organization’s information infrastructure since they significantly contribute to the competitiveness and success of the organization (Finlay, 2014). BI systems and its influence on the businesses has continued to emerge and from its use for simple, static analytical applications it is evolved into solutions for strategic planning, monitoring operations, customer relationship management, evaluating and studying profitability of various products etc. BI systems are today adopted to collect, store and process, analyze and use information in an effective manner to gain competitive advantage over others (Dean, 2014). BI does not have any universal definition and hence some consider it as a tool for data analysis and many others believe that BI is used for describing concept and methodologies for business decision improvements using facts and information from other supporting systems and sources. Reinnschmidt and Francousie (2000) state that a BI system is an integrated set of tools and technologies and includes products that are programmed to collect, analyze, integrate and make the data useful and available (Foreman, 2013). BI today, is considered as the system, which enhances the capability of an enterprise to improve its business efficiency, which could lead to achievement of higher business goals (Baesens, 2014). Davenport et al. (2010) considers BI as a term, which can be compared to an umbrella, which is commonly used to describe technologies, processes and applications for collecting, storing, accessing and analysing data to enable organizations make effective decisions (Dumbill, 2012).

BI creates value for the businesses by serving as the basis for bringing basic changes in an organization, which includes establishing new co-operations, creating new markets, offering products to customers and acquiring new customers. According to Koronios (2010), the critical success factors for using BI effectively in the organizations can be classified in three dimensions, which include organization, technology and process (Dumbill, 2012),. Organizational dimension include sponsorship, committed management support, clear vision and well-established business case (Dean, 2014). The technological dimension refer to elements such as flexible, business-driven and scalable technical framework and sustainable data integrity and quality. The process dimension includes balanced team competencies, business-driven and interactive development approach and change management, which is user-oriented (Foreman, 2013).

 

Significance of Big Data to Businesses

Big Data is a part of BI and involve processing, analysing, capturing, searching, storing, transferring, information privacy and visualization of large set of data, which are complex (Finlay, 2014). Big data refers to initiatives and use of technologies, which involve diverse, fast-changing and huge data, which are too complex for conventional technologies and infrastructure to process and use (Dumbill, 2012). Big data refers to data creation, analysis, storage and retrieval, which is huge in terms of volume, velocity, variety, variability and veracity.

Volume refers to the quantity of data generated, which is important in the existing context. It is the data size, which determines its potential and value and verifies if the data can be considered as bog data (Baesens, 2014). For example, a Boeing aircraft generates 240 terabytes of flight duration in a single flight while flying within the country, which includes data they create and consumer, data from the smart phone, data from sensors embedded in the objects, which results in billions of data, which are constantly updated and also receive constant data about location, environment, weather updates and other data including video, which requires a more comprehensive data analysis and processing tool, which big data provides (Schmarzo, 2013).

Velocity refers to speed of data generation or the speed at which the data is generated and processed to meet the user demands (Finlay, 2014). Some of the examples include the data exchange from machine to machine, which is done between billions of device, data exchange between sensors and infrastructure when measured in real time generate huge data log, the online gaming systems supports several concurrent users wherein each user produces multiple inputs every second (Dumbill, 2012).

Variety refers to the category to which the Big data belongs to, which needs to be known for data analysis. The identification of the variety of Big data enables people analysing the data, which can then be used for necessary processes. Big Data enables processing of variety of data and does not include only strings, dates or numbers but also includes 3D data, audio and video, geospatial data, unstructured text, which includes log files and social media (Finlay, 2014)..

Variability refers to highlighting the inconsistency, which can be shown by the data in some cases, which if not handled can disrupt the process and not allow effective data management (Baesens, 2014). Veracity refers to accuracy of source data since it directly affects the quality of data, which would be captured and processed and might have great variations (Schmarzo, 2013).

 

Big data has several benefits for the businesses since it can save money, generate more revenue and enable the organization to achieve many other business objectives. Big data can help organizations to build new applications or use new strategies to attract the target segment. It allows the organization to collect billions of real-time data about the products, customers and resources and processes them in a manner that it instantaneously provides information on optimizing resource utilization (Foreman, 2013). For example, the organization can change its marketing strategy with respect to an area where there is a demand for particular product and currently the organization does not have much presence there (Minelli et al., 2013). The technologies used in Big data can take over the highly customized systems, which provides standard solutions whereas the technologies for Big data are open source and are not very expensive. Big data helps the businesses to be highly responsive to changes in the market and allow them to bringing changes in their approaches at a faster rate as compared to competitors (McAfee & Brynjolfsson, 2012). Big data allows sharing of increased amount of data, which includes real-time data, which has variety and allows the businesses to respond to customer demands more accurately and effectively.

Big Data allows businesses to have competitive advantage by developing customer intimacy wherein Big data considers them as central to corporate strategy. Big data uses the data of the customers from online communities, interactive websites, online feedback forms, third-party databases etc. (Minelli et al., 2013). It collects data from social-media platforms, which involves millions of data each day. By processing so much of customer information, interest and reviews allows the organization to strategize their action, development of new products or services or add value to existing processes thus, allowing to stay ahead of the competitors (Baesens, 2014). The organizations using data processed by Big data technologies can be used to customize products for the customers, thus gaining advantage over others. Big data enables the organizations to enhance its supply chain operations since it allows capturing of data through radio frequency identification (RFID) and micro sensors along with data captured from the GPS systems installed in the trucks carrying products for distribution (Foreman, 2013). The businesses can thus track the movement of the raw materials, products and its distribution. Big data thus, adds great value to the organization and enables it to be quicker in responding to various uncertainties and challenges.

 

Big Data Implementation

The use of Big Data can be crucial to the functioning and efficacy public sector organizations such as the governmental health departments, citizen information and statistics, insurance companies and such. Big Data cannot be operated upon using the software that work on personal computers or even smaller networks (McAfee & Brynjolfsson, 2012). Bigger data sets pose complex problems such as data duplication or redundancy, poor correlation among different datasets or poor accessibility due to long iterative methods used to access the data (Wu et al., 2013). Big Data is still not widely used by many organizations and its use is relatively limited and is restricted to government and some reputed public sector organizations. All these organizations have their own ways and methods of big data management. Though there are software suites from reputed companies such as Oracle, which find use in such organizations (Minelli et al., 2013). The designing of architecture and the methods of storage and the end use will still need to be decided and worked upon by the organizations. There are several approaches organizations use to manage big data. All these approaches have some common tasks fulfilled by the organization that wishes to manage and make us of bi data to its business or other public service purpose (Foreman, 2013). Some of the steps involved in the implementation of big data are

• Identification of business/organizational requirements
• Data requirement evaluation
• Data collection
• Association of big data with the organizational data
• Run big data usage as pilot testing
• Align big data with organizational cloud
• Integrate big data analytics into operational workflow
• Integrate big data analytics into decision making
• Training programs to manage and use the integrated big data
• Setting up organizational standards for governance of big data

(Kolb & Kolb, 2013)

The first step in the data implementation process is the identification of business requirements. An organization needs to have clear understanding as to why it wishes to implement big data in its organizational processes and decision making. The big data project would essentially be needed to align with the organizational goals and objectives. It is only after the understanding of the role of big data in the organizational processes as contributor to achieving the organizational objectives, a decision on implementing big data would be taken. The second step is the identification of the data that needs to be integrated into the organizational processes such as decision making, which forms the big data (Schmarzo, 2013). This step also involves the analysis of whether or not organization has the necessary infrastructure and systems to manage the incoming big data. It is in this step that the type of data that needs to be stored, managed, retrieved, update and discarded is decided. Not all the data fields in the dataset are useful to the organization (Kolb & Kolb, 2013).

As such the organization needs to decide as to the specific parts of the dataset, which are needed for the business processes. This step involves consultations with all the stakeholders to the project such as various organizational departments, marketing partners, product design team and such (Labrinidis & Jagdish, 2012). It is only after the consultation with the stakeholders that it is decided as to which parts of the datasets would be used in the implementation process. This step also decides on the access to various kinds of data by different stakeholders in terms of accessibility and privileges (Needham, 2013). The third step of big data collection involves the capturing of both internal and external data. Internal data consists of useful information from the organizational records such as year wise sales, sales by gender, sales by income group, customer list, repeat customers, customers who have deserted the company and such (Foster & Fawcett, 2013). Internal data may also contain information like sales figures by product or service types, sales figures by colour or package, sales figures of accessories to name a few. All the existing or past customer related data belongs to internal data of the organization. External data may consist of competitor information such as competitor product types and category, competitor sales figures if available, market trends such as customer preferences, market size, market segmentation information such as market size by gender, by age, by income group and such information (Mayer-Schonberger & Cukier, 2013). The organization will decide as to which all information it would need for assistance in the organizational processes such as marketing efforts, product design or decision making (Needham, 2013).  The next step involves identification of the ways big data is associated with organizational processes. The step involves finding out as to which processes need which data and information from the big data. For example, the step identifies the nature of data that would be used by the sales department, the type of data that would be used by the advertising agency or the type of data that would be needed by the production design team (Kolb & Kolb, 2013).

The data on colour preference and preference of packaging styles of the people could be of interest to the product design team (Foster & Fawcett, 2013). As such the implementation process needs to identify the types of data that would be required by different stakeholders of the implementation process. The next step involves the design of data experiments using prototypes and the preferred environments using the preferred programming languages (Labrinidis & Jagdish, 2012). This step will ensure that the IT environment is designed in such a way that both the business and IT objectives are met (Dean, 2014). Since only a small fraction of data is used on a test basis on a chosen process for testing, any adverse impacts, if at all will be minimal. Any anomalies found in the pilot testing project can be fixed through reprogramming or re-selection of operating environment (Schmarzo, 2013). Big Data is normally from an external source, which would be associated with the internal data of the organization. As such a cloud environment that accesses big data and the organizational cloud that has the internal data need to be integrated so that data flow, pre-processing of data, data integration, post-processing of data can be managed effectively (Mayer-Schonberger & Cukier, 2013). Since the organizational cloud will also have the big data access routines, it becomes important that effective ways of integration of internal and external clouds for faster accessibility and processing are adopted. Decision making is one of the important organizational processes. Decision making requires summary and inferences derived from the datasets rather than the data itself. The next step thus involves integration of data analytics into decision making, which would decide as to the necessary data processing outcomes that would aid in decision making processes related to various organizational issues and operations (Needham, 2013). The next step is to prepare a training program including the training topics and training material as required by the various stakeholders in order to be able to operate and use the big data. The training programs would be customized to meet the varying needs of each of the stakeholders so that they become proficient in the usage of big data (Wu et al., 2013). The last step is to construct a set of standards and norms that govern the capture and use of big data (Foster & Fawcett, 2013).

The standards and norms may consist of some common guidelines for data capture, data updating and data omission or deletion. The standards may also determine as to which data types would be useful to the organization and which data types are not and as such need to be omitted for the organizational usage (Mayer-Schonberger & Cukier, 2013). The standards and norms will also determine the privileges given to the organizational staffs of various ranks and positions. This will ensure that only those people with adequate privileges will be able to capture, monitor, control or use the data.

 

Issues faced in the Implementation of Big Data

Big data implementation involves several challenges since it has the task to consider variety of data, huge volume and velocity of data. The most common challenges include the knowledge of information technology infrastructure since it involves running several process threads at the same time (Minelli et al., 2013). The storage strategy would also be different since it stores the most used data on faster storage devices such as cache state disk and less used data is placed on slower hard disks (Labrinidis & Jagdish, 2012). To maintain the Big Data the organization requires strategic expertise from storage professionals who would be able to ensure that the different varieties of data are stored effectively in the respective storage device (Foster & Fawcett, 2013). Big Data requires increased storage space and requires the source data to be accurate and hence incomplete, duplicate and inaccurate data has to be removed regularly, which requires the organization o have effective data clean up approach (Labrinidis & Jagdish, 2012).

The business goals and IT Big data strategy should be effectively aligned so that the organization makes optimum use of the Big Data and get the returns on investment and get right direction to organizational strategy implementation. If the data is not effectively processed and utilized it can only add to the expenses and might not enable the businesses to achieve their goals instead it can deviate them form their business goals (Mayer-Schonberger & Cukier, 2013). The implementation of Big Data requires talented staff who have experience in handling data. The organizations would require Big data specialists who would be able to handle the data effectively and trouble shoot the issues if they arise without whom the organization might lose the data or the data might lose its credibility or give outputs, which are not accurate.

 

Conclusion

Big Data has enabled businesses to have an access to huge amount of data from different sources, which enables them to identify various patterns of the business development, customers purchasing behaviour, changing market conditions etc. Big data is of great importance to the businesses since it enables them to process data at a faster rate and utilize the variety of data in different combination, which helps them apply different strategies and gain advantage over the competitors. But implementing Big data involves various processes right from ensuring the sources, the quality of data from the sources, to have effective infrastructure to capture, store and process different types of data and gaining the competency to manage the Big data systems. The implementation of Big data involves having sound knowledge of technologies, management of storage devices and competent and talented IT staff who should be capable of handling any issues in retrieving or storing of data.

 

References

Baesens, B (2014), Analytics in a Big Data World: The Essential Guide to Data Science and its Applications, Wiley; 1 edition

 

Dean, J (2014), Big Data, Data Mining, and Machine Learning: Value Creation for Business Leaders and Practitioners, Wiley

 

Dumbill, E (2012), Planning for Big Data, O’Reilly Media; 1 edition

 

Finlay, S (2014), Predictive Analytics, Data Mining and Big Data: Myths, Misconceptions and Methods, Palgrave Macmillan

 

Foreman, J (2013), Data Smart: Using Data Science to Transform Information into Insight, Wiley

 

Foster P & Fawcett, T (2013), Data Science and its Relationship to Big Data and Data-Driven Decision Making, Big Data, Vol. 1, Iss. (1), pp. 51-59

 

Kolb, J & Kolb, J (2013), The Big Data Revolution, CreateSpace Independent Publishing Platform

 

Labrinidis, A & Jagdish, H (2012), Challenges and opportunities with big data, Journal Proceedings of the VLDB Endowment, Vol. 5, Iss. 12, pp. 2032-2033

 

Laursen, G & Thorlund, J (2013), Business Analytics for Managers: Taking Business Intelligence Beyond Reporting, Wiley

Mayer-Schonberger, V & Cukier, K (2013), Big Data: A Revolution That Will Transform How We Live, Work and Think, Hodder and Stoughton

 

McAfee, A & Brynjolfsson, E (2012), Big data: the management revolution, Harvard business review Vol 90, Iss. 10, pp. 60-68

 

Minelli, M, Chambers, M & Dhiraj, A (2013), Big Data Big Analytics: Emerging Business Intelligence and Analytic Trends for Today’s Businesses, Wiley

 

Needham, J (2013), Disruptive Possibilities: How Big Data Changes Everything, O’Reilly Media; 1 edition

 

Schmarzo, B (2013), Big Data: Understanding How Data Powers Big Business, Wiley; 1 edition

 

Wu, X, Zhu, X, Wu, G & Ding, W (2013), Data mining with big data, Knowledge and Data Engineering, IEEE,  Vol. 26, Iss. 1, pp. 97-107,