How does artificial intelligence work? Its foundation is machine learning, and autocorrect makes it easy to understand intuitively.
Understanding how artificial intelligence works may seem very difficult, and often even unnecessary. After all, most people probably won’t be building their own models from scratch. However, this approach is mistaken. Understanding the technical fundamentals of how artificial intelligence works not only allows you to use it more effectively, but also to assess its results more accurately and understand why it sometimes makes mistakes. It’s a bit like driving a car—you don’t have to be a mechanic to drive one, but it’s worth knowing the difference between a brake and a clutch.
Table of Contents
Let’s Start with Self-Corrections
The simplest way to understand how artificial intelligence works is to look at something each of us uses: autocorrect on a phone or in a browser. The mechanism that makes the system suggest the next word when you write a message works similarly in language models. It does not understand the meaning of sentences in the human sense, but it can predict which word should come next based on thousands of examples. After the first word is written in a new utterance, artificial intelligence analyzes the user’s command + the first word it has written, and based on this generates the next word. An important concept is edit distance – checking how many letters need to be changed to turn one word into another.
."
This aptly illustrates the principles of machine learning – the model does not know the rules of grammar, but learns patterns from data. If it sees often enough that “dobry” follows “dzień,” it starts suggesting precisely this combination with a high probability.
Predicting one word based on the three preceding words is quite simple, because we can simply list all the combinations. If we want to predict two or three words ahead, we need to analyze more words from the preceding context, and the number of possibilities increases significantly. Therefore, simply adding computing power is not enough to turn autocorrect into ChatGPT.
."""
Machine Learning Basics
Machine learning can be understood as a process in which a computer is not given ready-made rules but discovers them on its own by analyzing data. Instead of telling it that a cat has four legs and whiskers, we show it thousands of pictures of cats until it identifies the features they have in common.
This process is usually divided into three stages:
- Dataset – a collection of examples that the system learns from. It may consist of text, images, audio, or a combination of these forms.
- Model – a mathematical structure that processes data. In language models, it takes the form of a neural network.
- Training – the stage in which the model analyzes data, makes incorrect predictions, and then corrects them in order to gradually learn accurate relationships.
The most important moment in the entire process is the so-called backpropagation– a mechanism that allows the model to correct its own errors. When the system “makes a mistake,” it calculates how far its prediction was from the correct answer and adjusts the weights of the connections between neurons. As a result, it becomes a little smarter with each iteration.
Neural Networks – How Machines Learn
Modern artificial intelligence is based primarily on artificial neural networks, inspired by the workings of the human brain. In practice, these are not biological neurons, but mathematical functions that process numbers.
Each neuron receives data (input), processes it, and then passes the result on (output). The network consists of many layers of such neurons. The first layers learn simple features—in the case of images, these may include detecting edges, colors, or points. Deeper layers learn increasingly complex concepts, such as the shape of an eye, a face, or an entire figure.
In language models, it works similarly. The initial layers learn to recognize sentence structure and grammar, subsequent layers analyze the meaning of words, and the highest layers can already generate coherent, logical sentences.
Each neuron has its own “weight,” which determines how strong its influence is on the final result. Training consists precisely in adjusting these weights—millions of them, and in the largest models, billions—until the network begins to operate optimally.
How ChatGPT Works – Language Processing Step by Step
ChatGPT and other language models are examples of so-called transformers – an architecture that revolutionized machine learning. Their key component is the attention mechanism, which allows the model to analyze the context of an entire sentence rather than just the words next to each other.
When you write a question, the model divides it into small fragments called tokens—most often individual words, syllables, or even letters. Then, each token is transformed into a vector of numbers that represents its meaning in context. This is done using an embedding matrix, which assigns each word (token) a specific position in a coordinate system. How do we know how these words should be arranged? This is precisely what training language models is for.
."""
Next, the network analyzes which words in the sentence are connected to one another and the extent to which they influence each other’s meaning. Consider the sentence: “When the dog that escaped from the yard chases the cat sitting on the fence, all the household members run outside.”
Here, the word “dog” is connected to “chases,” “escaped,” and indirectly to “household members,” because its behavior causes the subsequent events. The word “cat,” on the other hand, has a strong connection to “sitting on the fence” and also to “chases,” though from a different perspective.
In transformer architecture, each token “looks at” other tokens in the sentence, analyzing the context in which they occur.The model is trained to have an intuition from the outset about which words usually form meaningful relationships (e.g., that a verb connects with the subject or object).
This is a separate body of knowledge that a given model possesses. The words “cat” and “dog” are closer to each other in the embedding matrix than “dog” and “chases,” but the language model can detect which words “look at each other more often.”
As it passes through successive network layers, this “view” becomes increasingly precise — the model begins to detect not only direct relationships between words, but also indirect meanings, such as the fact that “the dog’s escape” and “the household members’ reaction” are part of a single cause-and-effect sequence. As a result, artificial intelligence can not only recognize the structure of a sentence, but also its meaning in the context of the entire passage.
Finally, after passing through many layers, the model predicts which token should appear next. In reality, every word you see in a ChatGPT response is the result of thousands of probabilities—and the model chooses the one that best fits the context.
This is done by building a large context vector. Knowing the relationships between words, the model creates a vector for the entire utterance. It then searches for the tokens closest to the coordinates of this vector.
That is precisely why AI can write naturally, translate texts, and answer questions—not because it “understands” content in the human sense, but because it mathematically predicts the most probable sequences of words.
What Is Temperature and How Does It Affect Responses
In language models, there is a concept of temperature, which determines the model’s level of randomness, often creating the impression of greater “creativity.”
Temperature is a number between 0 and 2 (sometimes higher) that controls the randomness of generated responses:
- At temperature 0, the model always selects the most probable word. The answers are precise but predictable—well suited to technical tasks.
- At temperature 1, a certain degree of freedom appears—the model may sometimes choose the second- or third-most probable option. The texts become more natural and flexible.
- At temperature 1.5 or 2, the answers can be surprising, sometimes even chaotic, because the model more often chooses less obvious paths.
That is why the same model can sound like a serious analyst or like a poet—it all depends on the temperature.
Mathematically, temperature operates within the softmax function, which transforms the model’s raw scores (the so-called logits) into a probability distribution. Each token in a sentence is assigned a numerical value—positive or negative—that reflects how well it fits the context. These values are essentially the distances between the vector representing the current context and the vectors of individual words in the vocabulary.
Softmax transforms these numbers into values between 0 and 1, which can be interpreted as probabilities. Previously, this could have been done more simply—for example, through min–max normalization (scaling all the numbers so that they fall within the range from 0 to 1) or by dividing each value by their sum. However, such methods do not adequately capture the relationship between strongly negative and strongly positive scores.
Softmax does this better because it uses an exponential function—it raises each value to the power of “e,” so even small differences in logits turn into large differences in probabilities. The temperature in this equation controls the “steepness” of this curve. When it is low, softmax strongly favors the highest score (the model is confident). When it is high, the graph flattens out, and even weaker candidates have a chance of being selected.
In short: temperature is a slider that, in the mathematical sense, regulates how much the model “spreads” its attention among possible words. By combining the softmax function with the concept of temperature, AI can balance precision and creativity—between mathematical coolness and literary improvisation.
.
How Is AI Different from Ordinary Programs?
A traditional computer program operates according to the “if–then” rule. For example: if a number is greater than 10, display a message. Artificial intelligence works differently—it has no ready-made instructions, but has learned them indirectly from data.
Thanks to this, it can operate in situations it has not encountered before. This is the key difference between traditional software and machine learning.
In classical code, every decision is anticipated by the programmer. In AI, decisions are the result of statistical patterns. That’s why a model can predict which word you will use, but it sometimes makes mistakes because it never “understands” the world the way a human does.
Why Artificial Intelligence Sometimes Makes Mistakes
AI errors are a natural result of how it learns. If inaccuracies appeared in the training data or context was missing, the model may draw incorrect conclusions. An example is so-called hallucinations—situations in which AI generates information that sounds credible but is completely untrue.
Why does this happen? Because the model has no access to reality—it operates solely on probability. If many similar sentences ended with a particular conclusion, it will consider that conclusion true.
It’s a bit like autocorrect changing “artificial” to “trick” because the latter word occurs more frequently. From a mathematical point of view, it’s a logical decision, but from a human perspective—it’s absurd.
Where Do Language Models Get Their Knowledge?
When a language model generates a response, it appears to understand the question. In reality, this is the result of complex relationships between tokens and neural network weights.
Scientists compare this to a map of meanings—each word is a point in multidimensional space, and the distances between them correspond to semantic similarities. That is why the model “knows” that the words “king” and “queen” are related, and that the difference between them is similar to the difference between “man” and “woman.”
This phenomenon is called embedding—and it is precisely what enables AI to create logical sentences and accurate analogies, even though it lacks consciousness.
All of the model’s knowledge is encoded and invisible.
All of the model’s knowledge is hidden in arrays and matrices:
- In embeddings – general relationships between words are represented (e.g., “king” – “man” + “woman” ≈ “queen”).
- In the weights of the transformer layers – this is where the model learns syntactic, semantic, logical, and factual relationships. These layers make it possible to process context and combine information from different parts of the text.
- In the output matrix (output projection) – it partially replicates the structure of the input embedding, but is tuned to predict the next token, so it also contains “reversed” knowledge of how particular facts are represented in words.
This means that knowledge is highly distributed, and when reviewing a given model’s code, we are unable to find specific places containing, for example, information that water boils at 100 degrees Celsius. Therefore, incorrect knowledge cannot be easily corrected.
If we want to be certain about specific information, we must use a RAG (Retrieval-Augmented Generation) system. In this case, before generating a response, the model searches for knowledge in a knowledge system… and then uses it to generate the response. ChatGPT searching for information on the web or in shared files uses RAG.
How Models Like ChatGPT Learn
Large language models learn in two stages:
- Pre-training – the model analyzes vast collections of texts (books, articles, websites) to learn general language patterns.
- Fine-tuning – after the initial training, the model is refined using smaller, carefully selected datasets. Often with the involvement of people who evaluate the quality of responses and help the model understand users’ intentions.
This second stage is crucial because it allows the model not only to speak correctly but also to respond meaningfully to questions. It is at this stage that AI learns how to avoid dangerous responses and misinformation.
How does pre-training work?
Initially, the language model starts with randomly initialized embedding matrices, weights, and output layers.
A lot of text is fed into the system. The program then divides it into smaller fragments, after which the model tries to guess the next words and compares its result with the expected one; based on this, it adjusts the weight parameters. Thus, after receiving the text: “Paris is the capital,” the model must write – “of France”
How does fine-tuning work?
After the pre-training stage, during which the model learns the general structure of language and the relationships between words, [fine-tuning](https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning%29)[,](https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning%29), meaning refining its behavior, takes place. In this process, the model is no longer learning language as such, but rather how to respond to specific situations. It receives a set of carefully selected examples—questions and expected answers—that show it how it should formulate responses in a conversation with a human. It is somewhat like a course in good manners: the model learns how to be a helpful, precise, and polite assistant.
Technically, fine-tuning proceeds similarly to pretraining. The model receives a fragment of a conversation (e.g., a user question) and must predict the correct continuation (i.e., the answer). It compares its answer with the one recorded in the data and adjusts its weights to move closer to the desired style. The dataset is smaller but of much higher quality—it demonstrates how to respond logically, specifically, and safely.
This stage is often extended to include so-called RLHF (Reinforcement Learning from Human Feedback), meaning reinforcement learning based on human feedback. In this variant, the model generates several answers to a single question, and people assess which one is the best. Based on this, an additional reward model is created to teach the main system to recognize better behaviors. As a result, ChatGPT and similar models can not only speak correctly, but also understand the tone of the conversation, its context, and the interlocutor’s intent.
Loss function.
In both cases, the model applies similar mathematical principles. First, the difference between the expected answer and what the model wrote is compared. Based on this, the parameters of the function are modified. The easiest way to understand this is through a simple problem, such as guessing the graph of a linear function when a pair of parameters is known.
Backpropagation
A language model consists of many layers. After running a query through the language model and comparing it with the model’s response, we know the magnitude of the error, based on which we must then modify the weights (the function’s parameters). But there is a certain pitfall here—we must change the parameters in many layers. The backpropagation mechanism helps with this.
Image Recognition by Artificial Intelligence.
Image recognition is the simplest useful thing that can be done with artificial intelligence. For a language model to be useful, it must be versatile, whereas a tool that recognizes people, cancer cells, letters, and digits can be much smaller. That is why it is a good idea to start learning how to create artificial intelligence with image recognition.
.
How Does a Computer Recognize Images?
Image recognition in artificial intelligence begins with transforming an image into numerical data. Each pixel in the image is represented by numerical values that describe its color and brightness. To make things easier for the model, the image is often reduced to a lower resolution. This does not involve removing information, but rather eliminating excess detail—the network does not need 8 million pixels to recognize a cat. In practice, this means that several adjacent pixels (for example, 2×2 or 4×4) are replaced with a single one whose value is the average of those four points. As a result, the model still “sees” the general shape and arrangement of objects, but without excessive detail. It is a bit like looking at a photograph from a distance—there are fewer details, but the proportions and contours remain clear. This reduced image becomes the input for the neural network.
A film showing what happens when you apply increased contrast and blur 1,000 times in a graphics program. AI does not use this function directly, but it intuitively demonstrates how it all works.
In the first layers, the network detects edges and contrasts, the simplest visual elements. It does this through so-called convolutions—small filters that move across the image and calculate brightness differences between neighboring pixels. This enables the network to distinguish, for example, the boundary between fur and the background. These filters (also called kernels) respond to vertical, horizontal, or diagonal lines—similarly to neurons in the human eye.
Combining these two methods—edge detection and reducing the resolution—enables the computer to transform an image of a digit measuring 1000 × 1000 pixels into, for example, a 4 × 4 pixel image. The program is able to remember all possible configurations of such an image; with two colors, this amounts to two to the sixteenth power.
Subsequent layers combine these basic features into more complex shapes—for example, ears, eyes, or facial contours. The deeper the layer, the more abstract the patterns it recognizes. In the final stages, the network creates a so-called feature map, which describes which visual elements appear in the image and the probability that they belong to a given class (e.g., cat, dog, car). Thus, the model does not see a “picture of a cat,” but rather a pattern of shapes and contrasts that, in millions of cases, turned out to be a cat.
This process allows the network not only to recognize an object but also to understand its context. By reducing resolution and analyzing edges, the model can operate quickly and effectively without having to process vast amounts of raw data. As a result, AI can view an image in a way similar to humans—understanding that it is not merely a collection of colored pixels, but a coherent picture of the world.
.
How does a program learn to recognize images?
Training an image recognition program essentially resembles the way language models learn to understand text, although here pixels are analyzed instead of words. Just as a language model predicts the next word, a visual model attempts to reconstruct the structure of an image and recognize which elements appear in it. During training, the program is given thousands of images along with descriptions (e.g., “cat,” “car,” “tree”). Each image is broken down into numbers—the pixel values—and then processed through successive layers of the neural network.
In the first layers, the network learns to recognize simple patterns: edges, contrasts, and horizontal and vertical lines. Then, in deeper layers, it combines these elements into increasingly complex structures, such as the shape of an eye, a car wheel, or the contour of a face. As a result, something like a map of the image’s features is created—a mathematical representation of what the network “sees.”
After each prediction, the model compares its answer with the true label and calculates how far off it was. This error, called the loss function, is the basis for adjusting the weights of the connections between neurons. After millions of such adjustments, the network begins to recognize objects with increasing precision. Just like a language model that learns the relationships between words, a visual network learns the relationships between shapes and colors—except that it does not predict the next word, but rather what the image depicts.
What Are the Limitations of Artificial Intelligence?
Although AI can write texts, create images, and solve problems, it still has its limitations. It has no consciousness, emotions, or cultural context. It does not know what truth is—it can only estimate what sounds like the truth.
It also cannot learn continuously—each new model is the result of retraining on fresh data. Therefore, ChatGPT does not “remember” previous conversations beyond the current session unless it is specifically adapted to do so.
Understanding these limitations allows you to use it more wisely: as a tool, not a source of absolute knowledge.
What does this mean for the user?
Understanding how artificial intelligence works does not require mathematics, but rather an awareness that a model is a probabilistic system. Once you accept this, it will be easier for you to interpret its results.
When AI answers a question, it does not say, “this is true,” but rather, “this is the most likely answer based on my data.” That is why it is always worth maintaining a little critical thinking—just as you check the result of autocorrect before sending a message.
Summary
Artificial intelligence is not magic, but statistics on a planetary scale. Every sentence generated by ChatGPT is a mathematical prediction, and every autocorrect correction is a miniature version of the same process.
Machine learning is the foundation that enables models to find patterns in data independently. Temperature, the attention mechanism, and neural networks are its tools.
Understanding these elements not only gives us greater control over how we use AI, but also allows us to appreciate the tremendous technological progress made in just a few years. And it all started with a simple question: how can we teach a computer to predict what we want to write next?