Andrew Ng: Even with generative AI buzz, supervised learning will create “more value” in the short term

You rarely come into conversation with an individual like Andrew Ng, who has left an indelible impact as a teacher, researcher, innovator and leader in the field of artificial intelligence and technology. Fortunately, I recently had the privilege of doing just that. Our article on the launch of Landing AI’s cloud-based computer vision solution, LandingLensgives a glimpse of my interaction with Ng, the founder and CEO of Landing AI.

Today we take a closer look at the thoughts of this pioneering tech leader.

Andrew Ng, one of the most prominent figures in AI, is also the founder of DeepLearning.AI, co-chair and co-founder of Coursera, and adjunct professor at Stanford University. He was also chief scientist at Baidu and founder of the Google Brain Project.

Our meeting took place at a time in AI evolution marked by both hope and controversy. Ng discussed the suddenly boiling generative AI war, the future prospects of the technology, his vision on training AI/ML models efficiently, and the optimal approach to deploying AI.

This interview has been edited for clarity and brevity.

Momentum is building for both generative AI and supervised learning

Venture Beat: Over the past year, generative AI models such as ChatGPT/GPT-3 and DALL-E 2 have made headlines for their ability to generate images and text. What do you think is the next step in the evolution of generative AI?

Andrew Ngo: I believe generative AI is very similar to supervised learning and is a general purpose technology. I remember 10 years ago, with the rise of deep learning, people instinctively said that things like deep learning would transform a certain industry or company, and they were often right. But even then, a lot of work was figuring out exactly which use case deep learning would apply to transform.

So we’re in the very early stages of figuring out the specific use cases where generative AI makes sense and will transform different businesses.

Even though there’s a lot of buzz around generative AI right now, there’s still huge momentum behind technologies like supervised learning, especially since properly labeling data is so valuable. Such rising momentum tells me that supervised learning will create more value than generative AI in the years to come.

Due to the annual growth of Generative AI, in a few years it will become one more tool that can be added to the portfolio of tools that AI developers have, which is very exciting.

VB: How does Landing AI see opportunities represented by generative AI?

ng: Landing AI is currently focused on helping our users build custom computer vision systems. We have internal prototypes exploring use cases for generative AI, but we can’t announce anything just yet. Many of our tool announcements through Landing AI focus on helping users inculcate supervised learning and democratizing access for creating supervised learning algorithms. We do have some ideas around generative AI, but we can’t announce anything just yet.

Experimenting the next generation

VB: What are some future and existing generative AI applications that get you excited, if any? After images, videos and text, will there be anything else for Generative AI?

ng: I wish I could make a very confident prediction, but I think the emergence of such technologies has led many individuals, companies and also investors to put a lot of resources into experimenting with next-gen technologies for different use cases. The sheer volume of experimentation is exciting, it means we’ll be seeing a lot of valuable use cases very soon. But it’s still a bit early to predict what the most valuable use cases will turn out to be.

I see a lot of startups implementing use cases around text and summarizing or answering questions around it. I see countless content companies, including publishers, signing up to experiments where they try to answer questions about their content.

Even investors are still figuring out the domain, so it will be an interesting process to explore the consolidation further and identify where the avenues are as the industry figures out where and what the most defensible companies are.

It amazes me how many startups are experimenting with this. Not every startup will succeed, but the lessons and insights of many people figuring it out will be valuable.

VB: Ethical considerations were at the forefront of generative AI conversations given the issues we’re encountering in ChatGPT. Is there a standard set of guidelines for CEOs and CTOs to consider as they begin to think about implementing such technology?

ng: The generative AI industry is so young that many companies are still figuring out the best practices to responsibly implement this technology. The ethical questions and concerns about bias and the generation of problematic speech really need to be taken very seriously. We also need to have a clear view of the good and the innovation that this creates, while at the same time having a clear view of the potential harm.

The problematic conversations Bing’s AI has been having are now much debated, and while there’s no excuse for even a single problematic conversation, I’m very curious what percentage of all conversations can actually go off the rails. So it’s important to keep statistics on the percentage of good and bad responses we observe, as it gives us a better understanding of the current state of the technology and where to get it from.

Address roadblocks and concerns around AI

VB: One of the biggest concerns surrounding AI is the possibility that it can replace human jobs. How can we ensure that we ethically use AI to complement human labor rather than replace it?

ng: It would be a mistake to ignore or not embrace emerging technologies. For example, in the near future, artists who use AI will replace artists who do not use AI. The overall artwork market may actually increase through generative AI, reducing the cost of creating artworks.

But fairness is an important concern, which is much bigger than generative AI. Generative AI is automation on steroids, and if livelihoods are greatly disrupted even as the technology generates revenue, both business leaders and government have important roles to play in regulating technologies.

VB: One of the biggest criticisms of AI/DL models is that they are often trained on huge data sets that may not capture the diversity of human experiences and perspectives. What steps can we take to ensure that our models are inclusive and representative, and how can we overcome the limitations of current training data?

ng: The problem of biased data leading to biased algorithms is now widely discussed and understood in the AI ​​community. So any research paper you’re reading now or previously published is clear that the various groups building these systems take data about representativeness and cleanliness very seriously and know that the models are far from perfect.

Machine learning engineers working to develop these next-gen systems have now become more aware of the issues and are putting tremendous effort into collecting more representative and less biased data. So we must continue to support this work and never rest until we have solved these problems. I am very encouraged by the progress that is still being made, even if the systems are far from perfect.

Even humans are biased so if we manage to create an AI system that’s much less biased than the average person even if we haven’t managed to mitigate all bias yet that system could do a lot of good in the world .

Get real

VB: Are there methods to ensure we’re capturing what’s real as we collect data?

ng: There is no silver bullet. Looking at the history of multiple organizations’ efforts to build these large language model systems, I can see that the data cleansing techniques were complex and multifaceted. When I talk about data-centric AI, many people think that the technique only works for problems with small data sets. But such techniques are just as important for applications and training of large language models or base models.

Over the years we’ve gotten better and better at cleaning up problematic datasets, even though we’re still far from perfect and it’s not time to rest on our laurels, but progress is being made.

VB: As someone who has been deeply involved in the development of AI and machine learning architectures, what advice would you give to a non-AI focused company looking to integrate AI? What should be the next steps to get started, both in understanding how to apply AI and where to start applying it? What are some key considerations for developing a concrete AI roadmap?

ng: My main advice is to start small. So rather than worrying about an AI roadmap, it’s more important to jump in and try to get things working, because the lessons of building the first one or a handful of use cases will form a foundation for the ultimately creating an AI roadmap.

In fact, it was part of this realization that led us to design Landing Lens, to make it easy for people to get started. Because if someone is considering building a computer vision application, they may not even know how much budget to allocate. We encourage people to get started for free and try to get something to work and whether that first try works or not. The lessons learned from trying to get to work will be very valuable and will form a basis for making decisions about the next few steps for AI in the business.

I see many companies take months to decide whether or not to make a modest investment in AI, and that is also a mistake. So it’s important to get started and find out by trying, rather than just thinking [it]with current data and observe whether it works for you.

VB: Some experts argue that deep learning may be reaching its limits and new approaches, such as neuromorphic computing or quantum computing, may be needed to advance AI. What is your opinion on this matter?

ng: I do not agree. Deep learning is far from reaching its limits. I’m sure it will reach its limits at some point, but at the moment we are far from there.

The sheer amount of innovative development of use cases in deep learning is huge. I am confident that deep learning will continue its tremendous momentum in the coming years.
Not to say that other approaches won’t also be valuable, but between deep learning and quantum computing, I expect much more progress in deep learning over the next few years.