All viewpoints in this article are limited to before June 2024.

This article only represents my humble personal views and does not constitute any investment or decision-making advice.

The Eve of Entering the Field of Artificial Intelligence

In fact, I started learning and understanding artificial intelligence quite late. When I was in the fifth grade, I heard that AlphaGo had defeated the Go world champion Lee Sedol.

But at that time, I didn’t even understand the definition of artificial intelligence. Obviously, such a big event didn’t stir any waves in my heart. I only remember adults discussing how AI would impact the future and how many people it would replace. Since 2016, artificial intelligence has frequently appeared in annual buzzwords, and investments related to AI have been growing explosively since 2016.

Source: https://ourworldindata.org/grapher/private-investment-in-artificial-intelligence?time=2016..latest
Narrator: 2016 was the first wave of the AI capital boom. In 2016 alone, global investment in AI exceeded 20 billion dollars and continued to rise in the following years.

The Germination

Even so, the term artificial intelligence had already deeply penetrated my mind. The junior high school period was a gap because I was busy with my studies and had no time to look at the outside world. The education system didn’t allow me to focus on external matters. However, when I entered high school, I felt it was time. It was 2020, and the pandemic had just begun to spread globally. The simplest way to avoid the pandemic was to stay at home. I took advantage of this time to thoroughly study Python, laying the foundation for my AI exploration in 2021.

Every day, I would remind myself, “I’m just a high school freshman. How much can I understand about artificial intelligence?” Before I started learning formally, I had already searched for some AI introductory plans on various software and forums in both Chinese and English. Without exception, everyone recommended starting with probability theory, statistics, and linear algebra. I admit that this learning approach is correct, but for a high school student who hasn’t made it his academic and career focus, it felt a bit excessive. Moreover, I always believed that AI is a highly application-oriented discipline.

We all know there’s a significant difference between application and creation or research. The former usually uses relatively mature solutions, while the latter requires a solid foundation of relevant industry knowledge. So, my goal was to first play around with AI. Using mainstream and mature tools like TensorFlow, I attempted to build and train a few models myself.

Narrator’s Perspective: During my freshman year of high school, AI was still a relatively hot term, but by then, many arguments against AI had emerged, claiming that even AlphaGo, as strong as it was, could do nothing but play Go, labeling AI as a false proposition. Nonetheless, I was very eager to try out the latest human technology at that time, given its vast potential for imagination. It was hard not to be attracted to it.

Formal Practice

When I really started learning, I realized that all the grand mathematical systems seemed less immediate than a simple import of keras, tensorflow, or pytorch. I found that if you’re not researching and exploring AI, learning simple Python libraries becomes very easy because countless scientists have paved the way for you. What you need to do is use high-level programming languages to call their code. It’s not an exaggeration to say that even someone with no AI background can understand the code and build the simplest neural network in a very short time, just like the famous AI experiment of recognizing handwritten digits.

Below is a very basic MNIST dataset containing hundreds of handwritten digits.

Playing is playing, and learning is learning. This was my first attempt in the field of AI, simple but fun.

So, are you expecting me to say that I started new explorations after that?

Sort of, but not really. After that, I began trying to replicate some neural networks that performed well in image classification, such as the ancient VGG model and the still widely used ResNet model. Obviously, at that time, I didn’t have the ability to read papers directly and replicate them, so I found detailed videos of these papers online and followed them step by step to replicate them. The most impressive one was Google’s GoogLeNet model. The paper didn’t explain the origins of the parameters in this network model. I reasonably guessed that Google might have used their TensorFlow model to search the hyperparameters directly. After all, they had deep pockets. However, the number of parameters in that network was negligible compared to today’s large models, so spending a bit more time to let the model search through them wasn’t impossible.

This was my initial attempt in the field of AI. If interested, I can write about some AI projects I worked on during my internship at Smart (the car company) in future articles.

But the theme of this article is my thoughts on AI.

Why Say So Much Irrelevant Stuff

Because I want readers to know my level. I am not currently a researcher in the field of AI. I’m not even a qualified hyper-parameter tuner. I have only tried many models and studied Andrew Ng’s machine learning courses and SenseTime’s open-source mmYoLo. I am currently just a user.

Since I am not a professional, my thoughts and views are very subjective and come from personal experience. The reference value of these ideas is up to you to decide.

Back to the Main Topic

In my opinion, the development of artificial intelligence is quite twisted right now. It’s mid-2024, and AI has been hot for a year or two, triggered by the large model trend led by OpenAI. Honestly, we have seen many products created for the sake of AI. Are these products useful? To be honest, the concepts are good, but the experience is terrible. What caused this? Why is a technology that could potentially change humanity developing so awkwardly?

1. Investor Pursuit
   For those who have been in the AI industry for a long time, now is a once-in-a-lifetime opportunity. Jumping out of the original system to seize this startup boom. After all, looking at the current investment directions, as long as it’s related to AI or large models, there will be plenty of investors willing to invest. In the eyes of investors, holding cash only depreciates, and the AI track is still very new with much potential for speculation. At least in this track, our entrepreneurs can still talk about ideals and visions. Investors are also willing to listen. In contrast, other tracks are all in decline. So, as an investor, would you prefer to chase the current hotspot or go to a field with low liquidity and uncertain future development? I think the answer is clear. Therefore, this forms a phenomenon of mutual pursuit. Entrepreneurs want to seize the opportunity to achieve financial freedom, and investors don’t have better products to invest in at this moment.

   For example, the former Huawei genius teenager Zhi Hui announced his departure from Huawei in December 2022 and founded his startup company Zhiyuan Robotics.

2. Big Model Explosion, Companies Can’t Fall Behind
   Before the birth of ChatGPT, there were already several language models, such as GPT-2. However, their parameter sizes were small (compared to current large models). I believe the main reason is: previously, no institution was willing to spend a lot of money to push a model hard. For example, GPT-2’s parameter size was already astonishing, but its results were not good, so OpenAI was not willing to invest exponentially more funds to push the development and iteration of large models. Other research institutions or companies worldwide also faced this issue: humanity could not estimate the critical value of model “emergence.” Coupled with GPU computing power being far less than it is now, most companies were reluctant to research and develop larger language models. Until later, Microsoft invested 3 billion dollars in OpenAI, allowing OpenAI to resume developing large language models. Without this wave of investment from Microsoft, the ChatGPT we see today might have been delayed even further.

   Once a company succeeds, to avoid falling behind, most companies begin to invest more resources in model training and data collection. After all, this large language model is still based on Google’s previously released Transformer model, making it relatively easy for companies to replicate. So, that’s why we quickly saw models created and trained by various companies and startups worldwide after OpenAI’s success. This path had already been paved by OpenAI, and everyone had a clear understanding that once the parameter size reached a certain level, the model would naturally emerge and exhibit the so-called intelligence.

All Coincidences are Inevitable

Capital is profit-driven; it goes to the track that’s hot. Companies don’t like falling behind. When a company achieves great success, others naturally want to replicate it.

But do these two roles really act as promoters in the market? I think it remains to be seen. What is certain now is that we are indeed in an AI bubble period. How much bigger this bubble can get depends on whether the scientific and industrial communities can achieve new breakthroughs or if CEOs continue to boast about how great their AI products are.