Touched by AI: competitive intelligence yields new data insights

As artificial intelligence (AI) tooling becomes more widely adopted, data-driven approaches to competitive intelligence practices are rapidly gaining popularity. As a result, a new generation of decision-makers can explore changing markets and address the growing challenges across industries.

Change is being driven by a deluge of customer data now generated by website activity, surveys and social media. Meanwhile, companies are poised to harness the power of new AI tools to continuously monitor market trends and adjust their positioning, offerings and pricing strategies to maximize revenue opportunities.

As with so many things these days, AI/ML models are seen as a game changer helping to find data insights. The advent of major language models like GPT presents exciting opportunities for competitive intelligence, according to Kurt Muehmel, who holds the title of everyday AI strategic advisor to AI platform provider. Dataiku.

The difficult task of gathering information about competitors and customers could be streamlined through such techniques, he said.

“These models are very good at summarizing and synthesizing text. Therefore, they can be useful to summarize earnings call transcripts, for example, or to elaborate competitive position papers if provided with accurate data for their inputs,” Muehmel said.

That’s important because, with its wide range of approaches and sources, collecting data as part of competitive intelligence practices can be daunting. Data sources range from blogs or presentations from industry experts to financial reports, news media items, public data sources and more.

AI tools, models and processes are increasingly essential drivers of competitive advantage as they enable continuous extraction of information that drives strategic decision support.

Modern competitive intelligence algorithms now combine historical and real-time data with machine learning, enabling companies to predict market trends and optimize pricing strategies with remarkable accuracy. This gives organizations a competitive advantage and enables them to respond in real time to changing market trends and consumer preferences.

Businesses can process massive amounts of data to identify patterns and make accurate predictions about future market trends. This information can then be used to make informed decisions such as product development and marketing strategies, giving companies a much-needed edge in a crowded marketplace.

According to Muehmel, data analytics, AI and automation have made it possible for vendors of all sizes to monitor a wider range of competitors.

“Many SaaS platforms available today enable automated monitoring of competitor activity across regions and languages. This is a great benefit, especially for companies just starting their competitive intelligence practices,” Muehmel told VentureBeat.

He explained that developing internal capabilities to build analytics and AI that fit the needs of a particular organization is one of the key ways companies can reap significant benefits outside of the technology space.

“Using analytics and AI, organizations can improve every process in their value chain. Companies that manage to internalize advanced analytics and AI capabilities will be the winners in their industry for years to come,” said Muehmel.

Steps towards a competitive intelligence framework

At the heart of a successful competitive intelligence strategy lies a well-orchestrated cycle that includes four critical phases: planning and defining the research objectives, collecting relevant data, processing and analyzing the data, and ultimately acting on the insights gained.

Michael Fagan, chief data scientist at enterprise VR company enchant, believes that the most crucial ingredient for any competitive analysis is the data sources, as a single data set can often lead to a misinterpretation of the output. To overcome this, he suggests using multiple data sources, but cautioned that each has its own biases.

Over the course of his industry experience, typical data sources have included external markets, social media, and website tracking. The first step, of course, is to establish a basis for understanding. It remains an essential prerequisite for usable AI processing.

“We first had to match the datasets by understanding the natural distributions and applying weights. With this data, we were able to predict search share quite accurately on a weekly basis. It also showed our market share, which terms and topics were standard and what was emerging. Having this information can be sobering at first, but this is a baseline,” he said.

“Adding machine learning to the mix allows you to further interpret the recorded patterns and create automated processes so that the information obtained is timely enough to take action and have a positive impact on your business compared to your competitors” , Fagan told VentureBeat. “To stay ahead, you need to focus on your basics and make sure you have a solid governance structure and standard techniques to offset biases. Once you have this, you can always rely on the intelligence layer to add value.”

Likewise Jo Ramos, eminent engineer and director at IBM Expert Labsemphasized the importance of training a competitive intelligence AI model using a large, well-labeled dataset for the specific task for which it was designed.

“AI models require rigorous training to accurately capture or represent the patterns in the dataset before it can be applied to real-world use cases. Today, very few organizations have the skills, software and infrastructure necessary to build and innovate with advanced models like GPT-3,” said Ramos. “The organizations that pioneered this space have kept many of the supporting tools and technologies proprietary or in-house.”

Ramos says that as companies build their competitive intelligence framework, companies need to understand the importance of AI governance — defining policies and being accountable throughout the AI ​​lifecycle.

“At IBM we have one AI Ethics Council that supports a centralized governance, review and decision-making process for IBM’s ethical policies, practices, communications, research, products and services,” said Ramos. “Doing this will help your models adhere to principles of fairness, explainability, robustness, transparency, and privacy.”

What’s next for AI-based competitive intelligence?

For his part, Muehmel at Dataiku said the most important thing companies can do is make sure they have a solid strategy for applying analytics and AI to applications across their business, including, but not limited to, competitive intelligence.

“By focusing on competitive intelligence, companies should not hesitate to experiment with large language models to see if they can make relevant suggestions in terms of competitive positioning or if they can otherwise accelerate competitive intelligence gathering and analysis he added.

If AI models succeed in gathering information and helping to visualize data, teams can make more actionable decisions and save time collecting information. It’s still early days for many industries, but some are now on a new path to informed real-time decisions that promise greater competitive advantage.