The nuances of voice AI ethics and what companies should do

In early 2016, Microsoft announced Tay, an AI chatbot capable of talking to and learning from random users on the web. Within 24 hours, the bot started spewing racist, misogynistic statements, seemingly unprovoked. The team pulled the plug on Tay, realizing that the ethics of unleashing a conversational bot on the Internet was unexplored at best.

The real questions are whether AI designed for random human interaction is ethical and whether AI can be coded to stay within bounds. This becomes even more important with speech AI, which companies use to automatically and directly communicate with customers.

Let’s take a moment to discuss what makes AI ethical versus unethical and how companies can ethically integrate AI into their customer-facing roles.

What makes AI unethical?

AI should be neutral. Information comes in a black box – a model – and returns with some degree of processing. In the Tay example, the researchers created their model by giving the AI ​​a huge amount of conversational information, influenced by human interaction. The result? An unethical model that harmed rather than helped.

What happens if an AI gets CCTV data? Personal information? Photos and art? What comes out on the other side?

The top three contributing factors to ethical dilemmas in AI are unethical use, data privacy issues, and system bias.

As technology advances, new AI models and methods appear daily and usage grows. Researchers and companies use the models and methods almost arbitrarily; many of these are not well understood or regulated. This often results in unethical results, even when the underlying systems have minimal biases.

Data privacy issues arise because AI models are built and trained on data that comes directly from users. In many cases, customers unknowingly become subjects in one of the largest unregulated AI experiments in history. Your words, images, biometrics and even social media are fair game. But should they be?

Finally, we know from Tay and other examples that AI systems are biased. Like any creation, what you put in is what you get out.

One of the most prominent examples of bias surfaced in a 2003 trial that showed researchers had spent decades using emails from a vast amount of Enron documents to train conversational AI. The trained AI saw the world from the point of view of a deposed energy trader in Houston. How many of us would say those emails would represent? U.S POV?

Ethics in Voice AI

Voice AI shares the same core ethical issues as AI in general, but because voice closely mimics human speech and experience, there is greater potential for manipulation and misrepresentation. We also tend to trust things with a voice, including friendly interfaces like Alexa and Siri.

Voice AI will also most likely communicate in real time with a real customer. In other words, speech AIs are your business representatives. And like your human representatives, you want to make sure your AI is trained in and acts in accordance with company values ​​and a professional code of conduct.

Human agents (and AI systems) should not treat callers differently for reasons unrelated to their service membership. But depending on the data set, the system may not provide a consistent experience. For example, if more men call a center, it could result in a gender classification that is biased towards female speakers. And what happens when biases, including those against regional speech and jargon, creep into voice AI interactions?

A final nuance is that voice AI in customer service is a form of automation. That means it could replace current jobs, an ethical dilemma in itself. Companies working in the industry must manage the results carefully.

Building Ethical AI

Ethical AI is still a rapidly growing field and not much data or research is available to produce a set of complete guidelines. That said, here are some tips.

As with any data collection solution, companies must have solid governance systems in place that comply with (human) privacy laws. Not all customer data is fair game, and customers need to understand that everything they do or say on your website could be part of a future AI model. How this will change their behavior is unclear, but it is important to give informed consent.

Area code and other personal information should not obscure the model. At Skit, for example, we deploy our systems in places where personal information is collected and stored. We make sure that machine learning models don’t get individualistic aspects or data points, so training and pipelines are oblivious to things like caller phone numbers and other identifying features.

Then, companies must regularly conduct bias tests and manage data usage checks and balances. The primary question should be whether the AI ​​treats customers and other users fairly and ethically, and whether edge cases — including customer errors — are spiraling out of control. Since speech AI, like any other AI, could fail, the systems need to be transparent for inspection. This is especially important for customer service, as the product interacts directly with users and can make or break trust.

Finally, companies considering AI should have ethics committees that inspect and investigate the value chain and business decisions for emerging ethical challenges. Companies that want to participate in groundbreaking research must also invest time and resources to ensure that the research is useful for all parties involved.

AI products are not new. But the scale at which they are adopted is unprecedented.

If this happens, we will need major reforms in understanding and building frameworks around the ethical use of AI. These reforms will move us towards more transparent, fairer and more private systems. Together we can focus on which use cases make sense and which don’t, given the future of humanity.

Sourabh Gupta is co-founder and CEO of Skit.ai.