Two Turing Prize winners: Self-supervised learning is key to aI reaching human intelligence

What is the most critical point in the future for building artificial intelligence to reach the level of human intelligence with deep learning? On this issue, the two big men in the field of artificial intelligence have agreed. At the 2020 ICLR Conference, two star scientists in the field of deep learning, and winners of the Turing Prize, Joshua Bengio and Yann LeCun, both spoke of trends in deep learning, saying that self-supervised learning is the key to making artificial intelligence reasoning more human.

ICLR, also known as the International Representation Learning Conference, is the top meeting in the field of deep learning. This year, affected by the outbreak, the first time in online form.

Yoshua Bengio is a pioneer in the field of natural language processing for artificial intelligence. He is currently a professor of computer science and computing at the University of Montreal, Canada. Yann LeCun, often considered the “father of convolutional networks”, is currently Facebook’s chief ausis scientist.

Two Turing Prize winners: Self-supervised learning is key to aI reaching human intelligence

Self-supervised learning

In the field of deep learning, supervised learning requires training AI models on marked data sets. In fact, deep learning was not a leading AI technology until a few years ago due to limited availability of useful data and insufficient computing power to process it. Self-supervised learning, on the other hand, can be done by building labels and training by learning the relationshipbetweens of data. Yang Likun believes that with the widespread use of self-supervised learning, it will play an increasingly important role. This step is also considered the key to human intelligence.

“Humans and animals acquire most knowledge through self-supervised models, not reinforced models. Self-supervised learning is about observing and interacting with the world. This observation is spontaneous, not done under test conditions. But we don’t know how to give machines the ability to learn. Yang Likun said.

Self-supervised learning is the “ideal state” of machine learning, but the focus is on how to automate the production of data labels on machines. The biggest obstacle is uncertainty. Typically, the data is such that it associates all possible values of a variable with the probability of its occurrence. They are a good representation of uncertainty when variables are discrete. But for now, researchers have not found a way to effectively represent the distribution of continuous variables.

One way to solve the problem of continuous data distribution is based on the Energy Model (EBM), which learns the mathematical elements of a data set and attempts to generate similar data sets, Yang said. Historically, this form of build modeling has been difficult to apply in practice, but recent research has shown that it can be adapted to complex topology.

Bengio argued at the seminar that artificial intelligence can be helped by the field of neuroscience, especially in the exploration of consciousness and awareness processing. He predicts that future research will clarify how advanced semantic variables are related to how the brain processes information, including visual information. These variables are things that humans communicate with language, and they may lead to a new generation of deep learning models.

“By combining with basic language learning, we can make a lot of progress, and at the end of the day we’re all building models that understand the world and how high-level concepts relate to each other. This is a joint distribution. Bengio said: “The human consciousness process uses assumptions about how the world changes, which can be understood as a higher level of expression. Simply put, we see changes in the world and then think of a word to explain them. “

No universal artificial intelligence.

In addition to the problem of data distribution, Yang Likun believes that the lack of background knowledge is also one of the obstacles to artificial intelligence’s inability to reach the level of human intelligence. For example, most people can learn to drive a car within 30 hours because they have mastered a physical model of the car’s behavior. By contrast, the intensive learning models deployed on self-driving cars start from scratch, and they must make thousands of mistakes before they can determine which operations are harmless.

“Obviously, we need to be able to build models to learn the world, and that’s why we do self-supervised learning – running predictive world models that allow systems to learn faster.” Conceptually, this is fairly simple, unless it is in an uncertain environment that we cannot fully predict. Yang Likun said. At the same time, he argues, it is not enough to achieve universal artificial intelligence (AGI) even with self-supervised learning and neuroscience learning.

Universal artificial intelligence means that machines acquire human-level intelligence. Some researchers refer to general artificial intelligence as strong AI or full AI, or machines that have the ability to perform general intelligence actions.

Yang Likun said this is because intelligence, especially human intelligence, is very special. “AGI doesn’t exist, there’s no universal artificial intelligence at all. We can talk about mouse-level intelligence, cat intelligence, dog intelligence, or human intelligence, but not universal artificial intelligence at all. Yang Likun said.

But Bengio believes that machines will eventually be able to acquire knowledge of the world without experiencing it, probably through acquiring language-based knowledge.

“I think it’s also a huge advantage for humans, and the reason why they are smarter than other animals is that we have our own culture that allows us to solve the problems of the world. For Artificial Intelligence to work in the real world, we need it not just machines that can translate, but machines that can truly understand natural language. Bengio said.