Machine learning, data integration and NoSQL are only some of the latest Big Data developments. Here we present an overview of the seven most important trends.
The digital transformation is unthinkable without Big Data, because only the knowledge gained from data allows companies to be agile. Agility in this connection means two things: continuously optimizing existing business processes based on data, and eliminating processes that are outdated to replace them with better ones.
The importance of Big Data is also growing due to increasing networking – think the Internet of Things (IoT). These new circumstances and the data they generate opens up previously unimagined possibilities for new business models to creative minds, as can be seen in the growing and successful entrepreneurial and startup scene.
Mindbreeze, a software producer for enterprise search, Big Data and knowledge management, has identified trends that reflect the current market developments in Big Data.
Machine learning or deep learning involves devices that automatically collect, store and analyze data. They are equipped with a kind of artificial intelligence (AI) that allows them to understand information in the data and recognize semantic connections.
Machine learning is particularly suited to the analysis of huge volumes of data, because “for smaller data sources you can figure things out yourself,” points out Norbert Wirth, Global Head of Data and Science for GfK market research institute.
Machine learning is especially popular when it comes to security. There are a number of projects that aim to use machine learning to improve building security. Banks also see potential in machine learning, for example to make online shopping more secure. Here the systems observe all transactions and try to differentiate criminal from normal patterns, and act on this information when needed.
The Internet of Things – the networking of the world – together with Big Data, gives companies opportunities to optimize their conventional business processes and possibly to shift to new business models, thus staying a step ahead of the competition. Companies that have already taken this step show the way. Heating systems manufacturer Vaillant, for example, has equipped its products with smart sensors that let customers control their heat centrally from a smartphone or tablet app. The digital transformation has thus turned this traditional heating business into a technology company, that also manufacturers heating systems.
Other companies have undergone a similar transformation by adapting their business model to new market conditions. For example, Deutsche Bahn uses its data in a very special way: Volker Kefer, acting Chairman of the Board of Deutsche Bahn and DB Mobility Logistics, reported during the CeBIT 2016 Preview event on three hackathons that led to the development of an app to show the readiness of elevators almost in real time. This indication is an advantage for customers as well as for the group itself – for customers because they can see if the elevator is working, and for the group because it can send a repair team quickly.
Predictive analytics make it possible to generate reliable predictions of future events from available data. A typical application for predictive analytics is predictive maintenance, an approach to planning of maintenance and servicing.
When a machine breaks down it can be expensive for the business, for example if a production line is interrupted and other machines are also prevented from working.
Rail traffic is a similar situation. If tracks can't be adjusted due to a defect, for example, they might have to be closed. That's why Deutsche Bahn has started equipping its rails with sensors. These sensors log the power needed by the motor to set the track. The consumption curve is compared to a target curve, and conclusions about the condition of the track can be drawn from the deviations. This lets the company respond rapidly, saving money and also improving customer satisfaction, because trains stay on schedule.
Access to all data is necessary for employees to be able to use available data optimally. Organizational and technical data silos in the company often prevent access to needed information, however. Enterprise search systems can help with this. These systems are able to connect data and information intelligently across departmental and application boundaries.
Access to the information sought must respect the rules and regulations of the company as well as data protection guidelines. Integrated rights management ensures that users in the company only find the data they're allowed to access.
Working with data past a certain volume can become very complex. When the data isn't structured and comes from many different sources, it's especially hard to make meaningful connections.
Programs that process the data visually can be useful here in drawing conclusions from it. Many-layered themes can be identified more quickly, or customers understood more quickly – from a bird's-eye overview to detailed analysis.
The analysis tools should be simple and self-explanatory so that non-specialists can understand them and make certain adjustments without the need for IT experts.
As previously mentioned, one area for Big Data is manufacturing – think Industry 4.0.
Smart factories are the key topic in this area – factories where as many components as possible are networked together and exchanging data. They support a high level of automation, for optimal and cost-effective use of the available resources, which can bring the cost of implementing individual customer wishes significantly closer to that of mass production.
But Big Data isn't limited to manufacturing. It's gaining ground in many other areas. The healthcare system is one example. Here Big Data is mainly used in research and diagnosis, such as in the field of cancer treatment.
When you bring in wearables – starting with smart watches and fitness trackers – Big Data enters the world of the individual, with self-tracking as one trend. The data this generates is coveted by medical insurance companies, among others. For example, Generali has announced lower rates in exchange for sharing medical data. This insurer wants to measure its clients' exercise and consumption habits with an app – all in accordance with German data protection legislation.
Self-tracking: Data from fitness trackers like the Garmin Vivosmart HR shown here is coveted by medical insurance companies. (Photo: Garmin)
Big Data generally comes from different sources – sensors in machines, cars or wearables, social networks or e-mails. The traditional relational databases aren't suited to this kind of data. NoSQL databases offer more scope for processing and storing unstructured data. That's why they're becoming more important for the business strategy of companies that are digitizing.
This development is also reflected in the current Magic Quadrant for Operational Database Management Systems published by market research firm Gartner in 2015. Where so far Oracle, IBM, Microsoft and SAP have been the clear leaders in this area, now such NoSQL providers as Mongo¬DB, DataStax, Re¬dis Labs and MarkLogic have joined them.