5 business intelligence myths that stand between you and a data-driven company

For decades, business intelligence (BI) and analytics tools have promised a future where data is easily accessible and can be turned into information and insights for making timely, reliable decisions. For most, however, that future has not yet arrived. From the C-team to the frontline, employees rely heavily on technical teams to understand data and gain insights from dashboards and reports. As the CEO of a data and decision intelligence company, I’ve heard countless examples of the frustration this can cause.

Why does traditional BI no longer deliver value after 30 years? And why do companies continue to invest in multiple, fragmented tools that require specialized technical skills? A recent Forrester report shows that 86% of companies at least two BI platforms, with Accenture find that 67% of the global workforce has access to business intelligence tools. So why is data literacy still such a common problem?

In most cases, the inaccessibility of analytical forecasting stems from the limitations of current BI tools. These limitations have perpetuated several myths, widely accepted as “truths.” Such misconceptions have undermined many companies’ efforts to deploy self-service analytics and their ability and willingness to use data in critical decision information.

Myth 1: To analyze our data, we need to bring them all together

Traditional approaches to data and analytics, shaped by the limited capabilities of BI, require a company’s data to be brought together in a single repository, such as a data warehouse. This consolidated approach requires expensive hardware and software, valuable computing time when using an analytics cloud, and specialized training.

Too many companies, unaware that there are better ways to combine data and apply business analytics to it to make intelligent decisions, continue to rely on expensive, inefficient, complex and incomplete approaches to analytics.

According to an IDG survey, companies draw on an average of 400 different data sources to power their BI and analytics. This is a mammoth task that requires specialized software, training and often hardware. The time and expense required to centralize data in an on-premises or cloud data warehouse inevitably negates any potential time savings that these BI tools should deliver.

Direct query means bringing the analytics to the data, rather than the other way around. The data does not need to be pre-processed or copied before users can retrieve it. Instead, the user can directly query selected tables in the given database. This is diametrically opposed to the data warehouse approach. However, many business intelligence users still rely on the latter. The time-consuming effects are well known, but people mistakenly accept them as the cost of performing advanced analytics.

Myth 2: Our largest data sets can’t be analyzed

Data exists in real time as multiple, fluid streams of information; it shouldn’t be fossilized and moved to the analytics engine. However, in-memory databases that rely on such a method are a staple of business intelligence. The problem with this is that a company’s most comprehensive datasets quickly become unmanageable or obsolete.

Data volume, speed and variety have exploded over the past five years. Organizations therefore need to be able to process large amounts of data on a regular basis. The limitations of legacy BI tools — some dating back to the 1990s, long before the advent of cloud data, apps, storage, and virtually everything else — that rely on in-memory engines to analyze data have created a sense that it’s an invincible battle.

Companies can solve the problems inherent in in-memory engines by going directly to where the data resides, allowing access to larger data sets. This also makes an enterprise analytics program future-proof. Direct query makes it infinitely easier to migrate from on-premises to cloud services like those of our partners, AWS and Snowflake, without completely rewriting the code.

Myth 3: We can’t unify our data and analytics efforts across the organization

Too often common practice is mixed with best practice. Ad hoc selections and combinations of BI tools produce a cocktail of preferences and functionality, with organizations often adopting departmental approaches. Sales may like one platform; finance may prefer something else, while marketing may choose yet another option.

Soon, each department has a different set of tools, creating information silos that make it impossible for the apps to talk to each other or share analytical information. According to the aforementioned Forrester research, 25% of companies use 10 or more BI platforms.

The problem is that splitting data preparation, business analytics, and data science across different tools hampers productivity and increases the time spent switching and translating between platforms.

Certain business areas work best when leaders let their departments take an individual approach. Analytics is not one of them. Leaders and decision makers need to be able to trust their data. But trust is eroded every time it passes a different set of tools on the way to creating actionable insights. The process inevitably leads to data conflicts and opacity. Consistency and understanding are crucial.

Myth 4: Chasing the AI ​​dream distracts us from the daily reality of doing business

Many technologies, including BI tools, claim to be AI-driven. The promise is to replace human labor with flawless machine-learning efficiency; the reality is more often disappointing. That’s why many companies have abandoned the idea of ​​using AI in their day-to-day analytics workflow.

Technology professionals can understandably be cynical about the real-world examples of widespread AI in the enterprise. People find that they are still manually structuring and analyzing their data, gaining insights and making the right decisions – all from scratch. The idiosyncrasies and decision-making processes of the human mind are challenging, if not impossible, to synthesize.

The trick to making AI a functional, effective tool in analytics is to use it in ways that support everyday business challenges without shielding it from it. Knowing exactly which AI-driven capabilities to use is vital. It may be intelligent, but like any tool it needs direction and a steady hand to deliver value. By automating the routine, people can use intuition, judgment and experience when making decisions. You don’t have to be afraid of a robot uprising.

Myth 5: To get the most out of our data, we need an army of data scientists

There is a huge demand in the industry for the ability to collect massive amounts of disparate data into actionable insights. But top management still believes they should hire trained interpreters to parse the hundreds of billions of rows of data that larger organizations produce.

Processing, modelling, analyzing and extracting insights from data are in-demand skills. As a result, the services of data scientists with specific and intensive training in these areas are becoming a high priority.

But as they add value, you reach a point of diminishing returns. And these employees are no longer the only ones who can perform data science. A generation of corporate employees has entered the workforce and they are expected to review and manipulate data on a daily basis.

Highly skilled data scientists are not necessary hires in some cases when non-technical business users have arranged self-service access to augmented analytics and decision intelligence platforms. These users have invaluable domain knowledge and insight into the decision-making chain within their company. What it takes to make their work more accessible is a solid foundation of data and analytics capabilities that traditional BI tools often struggle to provide.

Value propositions and broken promises

The current analytics and BI landscape has made it clear to business leaders that there are certain natural limits to their data and analytics efforts. While still useful for specific use cases, traditional tools are applied in loose combinations, varying from department to department. The frustration this causes – the inefficiency and the potential loss of time – is a direct result of the gaps in current BI capabilities.

Traditional BI prevents companies from making optimal use of their data. That much is clear: Enterprise-level companies generate massive amounts of data in various formats and use it for a wide variety of purposes. Confusion is inevitable when the method of collecting and analyzing data itself is confused.

Something more extensive is needed. Businesses don’t trust AI-driven processes because legacy BI tools can’t deliver on their promises. They don’t trust democratized access to data because their departments don’t speak the same analytics language. And they don’t trust their data because in-memory engines don’t scale to the degree they need to, leaving them with incomplete—and therefore unreliable—data.

Data and analytics innovation is how companies deliver value in the age of digital transformation. But in order to innovate, you need to know that your barriers are fragile.

Omri Kohl is co-founder and CEO of Pyramid Analytics.