Skip to content
Back to all articles
By TJ SoftUpdated:

How Does ChatGPT Work? How Artificial Intelligence Works

Jak działa Chat GPT? Sztuczna inteligencja działanie

Artificial intelligence (AI) is a concept that has gained immense popularity in recent years. It is discussed in the context of the technological revolution, work automation, the creation of creative content, and even threats to civilization. At the same time, many myths and oversimplifications have grown up around this concept. What is artificial intelligence really? How does it work? What distinguishes it from an ordinary computer program? In what fields is it used? What opportunities and risks does it entail?

Answering these questions requires going back to the basics: both historical and logical. Understanding AI is not only a matter of knowing algorithms, but also of reflecting on what “intelligence” is—not only in machines, but also in ourselves.

Why Is It Worth Knowing How Artificial Intelligence Works?

At first, you can learn a great deal simply by using tools such as ChatGPT—experimenting, asking questions, and observing the responses. It is an intuitive way to discover the capabilities of artificial intelligence. However, when we start using these systems for more complex tasks, limitations and unexpected behaviors emerge. At that point, it is useful to understand how AI works under the hood: where its errors come from, why something “doesn’t work” as we expected, and what can be done to work around or improve it. Awareness of these mechanisms gives you greater control and allows you to get much more out of AI.

Table of contents

The Simplest Analogy: Autocorrect and Suggestions.

Before moving on to more complex topics, such as layers and tokens, it is best to understand the analogy with autocorrect.

ChatGPT works on a similar principle to autocorrect on a phone, but in a much more advanced form. Instead of suggesting a single word, it can generate entire sentences and coherent responses tailored to the context of the conversation. The model was trained on vast collections of text from the internet, books, articles, and conversations, allowing it to learn to predict which words and phrases appear in specific linguistic situations.

When you ask a question, ChatGPT does not search through ready-made answers; instead, it predicts the next words in real time, constructing the response step by step.

It does not possess knowledge in the traditional sense or an understanding of the world, but thanks to its ability to recognize linguistic patterns, it can formulate responses that are logical and tailored to the input content.

How does autocorrect work?

""

Autocorrect on your phone analyzes your words and uses them to predict what you intended to write. It does this by drawing on a database of the most frequently used words and word combinations. When you type, for example, “dziękue,” the system will recognize it as a typo and suggest “dziękuję” because that word occurs more frequently and fits the context.

What is the Levenshtein distance?

Levenshtein distance, also known as edit distance, is a metric that measures the difference between two sequences (e.g., two words). It is defined as the minimum number of single operations needed to transform one sequence into the other.

These operations are:

  • Insertion: adding a character (e.g., from “dom” to “domek”).
  • Deletion: removing a character (e.g., from “dobre” to “dobr”).
  • Substitution: changing one character to another (e.g., from “kot” to “kit”).

Each of these operations has a “cost” of 1. The sum of these costs gives us the Levenshtein distance. The smaller the distance, the more similar the words are.

,

Suggestions – n-gram models)

Suggestions for subsequent words in autocorrect work based on language models, which estimate the probability of a given word occurring based on the preceding context. The larger and more advanced the model, the more accurate the predictions.

The simplest mechanism is n-gram models, which analyze how often a given word occurs after another one.

  • Bigrams: the system takes the last entered word into account. If you write “I’ll go to,” the model checks which words most often appear after “to.” For example, it might suggest “shopping,” “a walk,” or “lunch,” because these combinations occur very frequently in training text corpora.

  • P(shopping∣on)>P(walk∣on)

  • Trigrams: the algorithm takes into account the two preceding words. If you write “I will go on a long”, the model will predict that the next word will be “walk”, “weekend”, or “walk”, because such phrases are common in Polish.

  • P(walk∣for a long time)>P(weekend∣for a long time)

The problem with n-gram models: taking into account the two or even four most recent words does not allow for predicting coherent utterances. Scaling this technology is very costly and inefficient. Each additional word taken into account means millions of additional combinations. Many, if not most, combinations of 10 words will never occur in natural language, causing the model to assign zero probability to new sequences.

N-gram models are unable to understand that a word at the beginning of a paragraph influences the choice of a word at its end. They also do not understand that, for example, among the last 10 words, word no. 2 refers to word no. 4, and so on.

How does ChatGPT work?

ChatGPT works by predicting successive words, with each subsequent word being selected based on a probability that depends on the preceding words in the statement. This can be described as follows:

  • First, it analyzes the context, meaning everything that has already been said or written in a given passage of text.
  • For a given position in the sentence (for example, after the words “I would like to go…”), the model calculates the probabilities of all possible words that could appear next—such as “to,” “into,” “from,” “there,” etc.
  • It selects the word with the highest probability, although sometimes, depending on the settings (e.g., temperature), it may also choose less obvious options in order to sound more creative.
  • After adding that word, the entire process repeats, except that the new context now includes the next word, which changes the model’s “field of view” and its predictions for subsequent positions.

In other words, the model predicts each word step by step, and what it has already generated influences the choice of what comes next. As a result, a coherent whole is created: a sentence, a paragraph, or even an entire conversation.

With this foundation in place, we can move on to more complex concepts.

How does machine learning work?

To understand how ChatGPT works, it is useful to know what machine learning is, as it forms the basis of how language models operate.

Building a language model from scratch is very difficult, but creating a program to recognize, for example, images or play games is easier.

When it comes to recognizing photos, it is easier to see artificial intelligence in action; when generating text, it is more difficult to see the moment when the text begins to make sense.

Machine learning is a method in which a computer model learns a task based on training data. It is not explicitly told what to do; instead, it is shown examples.

Example: we want a computer to recognize cats in photos. Instead of writing rules (“a cat has ears, a tail, and fur”), we show it thousands of photos labeled “cat” and “not a cat.” The algorithm learns to identify characteristic features and can then recognize a cat in a new photo.

The best example is recognizing letters and numbers, since such programs are the easiest to create and have practical applications.

3 Blue 1 Brown

How Does Artificial Intelligence Recognize Handwritten Digits?

In the image: the first layer is responsible for encoding the graphic into machine language (creating a pixel matrix), while the last layer identifies a specific digit. The middle layers perform the analysis, and what happens inside is the most interesting part.

The whole thing is based on two mechanisms:

  • Convolution (detecting specific patterns, such as edges, loops, lines, and arcs). This involves sliding a small window, e.g. 3×3 pixels, and calculating the main features.
  • Pooling – reducing the image dimensions while preserving its features. This involves dividing the graphic into smaller sections, e.g. 2×2 pixels, and selecting the pixel with the highest value from each, which are then used to create a new, smaller image.

For a graphic measuring 100×100 pixels, the number of possible combinations is 2 to the power of 10,000—a number so large that it is impossible to compare all the combinations. However, if we reduce the image to a square with sides 4 pixels long, consisting of 16 pixels in total, the number of combinations is 2 to the 16th power, or 65,536. This takes up 128 KB or 1 MB, depending on how we store it.

Learning Process

Initially, each neuron in one layer is connected to every neuron in the other layer. As learning progresses, the program assigns greater weights to some connections and smaller weights to others, allowing the trained model to operate more efficiently.

YouTube channel: 3 Blue 1 Brown

How Do Artificial Intelligence Neurons Work?

The main task of neurons is to find functions that describe a given image (or a given action). The easiest function to illustrate this with is a linear function, namely y=ax+b. A single neuron checks the distance between the graph of the function and the specific data, and then changes the parameters a and b. The next neuron does this with another point, thereby shifting/rotating the graph’s line so that it describes reality as accurately as possible.

How does machine learning work?

Machine learning is a method in which a computer model learns a task based on training data. It is not explicitly told what to do; instead, it is shown examples.

Example: we want a computer to recognize cats in photos. Instead of writing rules (“a cat has ears, a tail, and fur”), we show it thousands of photos labeled “cat” and “not a cat.” The algorithm learns to identify distinctive features and can then recognize a cat in a new photo.

.

We distinguish three main types of learning:

Supervised learning

The model receives input data together with labels (e.g., an image + category). Its task is to learn to predict the label for new data. Examples:

  • facial recognition,
  • medical diagnosis (e.g. benign/malignant tumor),
  • classification of messages as spam/non-spam.

Unsupervised Learning (unsupervised learning)

The model receives unlabeled data and must discover the structures itself (e.g., grouping customers according to their purchasing behavior). It is used, among other things, in:

  • market segmentation,
  • data compression,
  • anomaly detection.

Reinforcement Learning

An agent (program) makes decisions in an environment and receives rewards or penalties. Based on this, it learns an action strategy. Examples:

  • games (chess, Go, StarCraft),
  • robot control,
  • advertising optimization.

Deep Learning.

Deep learning is a form of machine learning based on deep neural networks. These networks have many processing layers that learn increasingly abstract representations of data. Example: the first layer detects edges in an image, subsequent layers detect shapes, then objects (e.g., eyes, a tail), ultimately recognizing a cat.

Thanks to deep learning, systems have been developed that recognize speech (e.g., Siri), perform machine translation (e.g., DeepL), power language models (e.g., ChatGPT), and generate images (e.g., DALL·E). However, training them requires enormous amounts of data, computing power, and investments totaling millions.

Layers: How AI Processes Information

What is a layer in an AI model?

A layer is one level of information processing within a model. Each layer receives data from the previous one and transforms it—it performs certain mathematical operations that extract features from the data. It then passes the result to the next layer.

Example with image recognition:

  • the first layer detects edges,
  • the second combines them into shapes,
  • the third recognizes objects (e.g., eyes, a tail),
  • the last decides whether it is a cat.

In language models (e.g., GPT), layers analyze text token by token, identifying syntactic, semantic, and contextual dependencies.

What does it mean for a model to have, for example, 96 layers?

This means that the data passes through 96 levels of transformations. Each layer is a set of neurons (that is, simple mathematical functions) that “learn” different aspects of the data. The more layers there are, the more complex relationships the model can learn—but the more computing power and data are also required.

Tokens: how AI splits text

What is a token?

A token is a fragment of text that the language model operates on. It can be:

  • the entire word (e.g., “cat”),
  • part of a word (e.g., “cat” and “kitten” divided into “cat” + the ending),
  • a single character (e.g., “c”, “a”, “t”).

The tokenization system depends on the specific model. GPT-3.5 and GPT-4 use a tokenizer based on so-called Byte Pair Encoding. This allows the model to efficiently handle many languages and deal with words it has never seen before.

Why Are Tokens Important?

  • The model does not read sentences like a human; it receives a sequence of tokens (e.g., 1,500 tokens).
  • Tokens affect the length of the response, usage costs (e.g., in an API), and the quality of the response.
  • One sentence may contain 10 tokens or 25, depending on the language and structure.

Knowing that ChatGPT operates on tokens, it is easier to understand why it struggled for a long time to count the letters in a given word, with “Strawberry” being a famous example.

ChatGPT does not “see” letters, only tokens—that is, fragments of words. To the model, “Strawberry” is not a single whole, but rather several tokens, such as “Str”, “aw”, and “berry”. The model does not count letters like a human; it predicts the answer based on what it has learned from the data. Sometimes it gets it right, sometimes it doesn’t.

Counting letters in words is a very good example of why it is useful to understand the general principles behind how artificial intelligence works. This helps explain why AI may struggle with tasks that are trivial for humans while being incredibly effective at highly complex tasks requiring expert knowledge.

How was it fixed? In newer versions (such as GPT-4-turbo), the model is better at recognizing when a question concerns characters rather than the meaning of the entire sentence. In such cases, the model assigns a separate token to each letter in the word we are asking about.

Embeddings: vectors of meaning

How Does AI “Understand” the Meaning of Words?

The model does not understand words in the human sense. Instead, it transforms each token into a vector—a sequence of numbers, such as 153 floating-point numbers. This is known as an embedding. The vector represents the meaning of a word in the context of other words.

Example:

  • the embedding of the word “king” will be close to the embedding of the word “queen”,
  • but will differ in a way similar to the difference between “man” and “woman”.

Thanks to embeddings, the model “knows” that “dog” and “cat” are closer to each other than “dog” and “shovel”—even though it does not know these words as such.

The vector used by ChatGPT is an ordinary vector known from the coordinate system. Two-dimensional vectors consist of two numbers, where one represents the length and the other the height, e.g. (3, 4). It is also easy to imagine three-dimensional vectors, which simply have three components (coordinates). Although imagining four-dimensional vectors is already more difficult, from a mathematical point of view, adding a new dimension is not a problem at all. It is easy to create vectors with any number of dimensions, and it is similarly straightforward to perform operations on them analogous to those known from two-dimensional vectors.

Thanks to vectors, we can see the distances between words. Why are vectors so important?

Vectors help us understand the relationships between words. For example (greatly simplified), the distance between the words king and queen is similar to the distance between the words man and woman.

When processing text through layers, the model analyzes each token in relation to the other tokens in the text and creates a new version of the token that carries more information. If the text described that “the king was wearing a purple robe,” then after processing, the token for “king” would contain information about the clothing. And the token for “robe” would contain information that it was on the king. This is described more precisely by the attention mechanism.

3 Blue 1 Brown

Attention: model note

What is the attention mechanism?

The attention mechanism allows the model to “focus” on relevant parts of the text. Some words are more important than others—the model learns which ones are worth considering in a given context.

Example sentence: “Ania gave the book to Basia because she had grown bored with it.”

A human understands that “her” refers to Ania. For a model, this is difficult. The attention mechanism allows it to analyze relationships and assign different weights to various tokens.

Transformer and self-attention

Modern language models are based on transformer architectures. A key component is self-attention—a mechanism in which each token “looks” at the other tokens and assesses how they are related to one another. This makes it possible to capture context and meaning not only locally (word by word), but also globally (e.g., an entire paragraph).

Logos and weight matrix.

Based on the processed vectors from the text, the model builds a context vector and compares it with a weight matrix (that is, the same matrix used for embedding, or converting a token (a number) into a vector with coordinates). Some models use the same matrix for both input and output, although different matrices can be used; the operating principle is similar.

The language model therefore computes the dot product between the context vector and the tokens from the output weight matrix. Then, a list of the distances of the given words (tokens) from the output token to the tokens in the matrix is created. Now it is necessary to determine the probability.

Softmax

We received a list of words, along with their distance from the output vector. The sum of these distances rarely amounts to 100, and some words may have negative values. It is now necessary to convert these values into percentages, ensuring that the sum of the percentage points equals 100%.

3Blue1Brown

What is softmax?

Essentially, it is a simple function that transforms a vector of numbers into probability percentages. It works similarly to linear normalization, but compared with it, it increases the differences between the results. You do not need to understand how the function works; you can treat it as a black box, but understanding it is useful for learning about the concept of temperature.

Linear normalization

You may not remember this term, but you know well how it works. For example: we have 9 white balls and one black ball; what is the probability of drawing the black ball?

We divide the number of black balls (1) by the total number of white and black balls (10), which gives us 0.1, or 10%.

If we use Softmax to calculate this, the chance of drawing the black word will be approximately 0.00035%. In the case of drawing balls, it does not make sense, but it is used in thermodynamics and statistical mechanics, where it is known as the Boltzmann function. **How does Softmax work?**The Softmax function has the following formula:

.

This is high-school-level mathematics, but many people may have trouble with it:

e – is a mathematical constant, something like Pi (π); the number “e” is approximately 2.71828
Z – is the entire vector, or, more simply, a set

From i – softmax background of this vector

Σ – that is, sigma, or the sum of many, many elements

Σn– sum of n elements

j– is the index of the current element

For example:

We have the numbers 1, 3, 5, 7,

.

Approximate values:

.

The sum of all softmax values is 1 – that is, 100%

How can we make the differences in probability between individual numbers (words/vectors/tokens) smaller? We need to introduce a number T by which we divide each element of the vector. This makes the difference between the powers of the numbers smaller, because the number e raised to the power of 1 is smaller than the number e raised to the power of 3. We achieve this effect when calculating the softmax for the number 3 with T = 3.

.

Temperature in Artificial Intelligence: Controlling Creativity

Temperature is a parameter that affects the randomness of the response generated by the model. It is colloquially referred to as creativity, because sometimes a higher temperature gives the impression of greater creativity.

Low temperature (e.g. 0.1) – responses are predictable, conservative, and consistent.

  • High temperature (e.g. 1.0 or higher) – responses are more varied and creative, but may be chaotic or incorrect.

The model always “knows” which next tokens are most likely. Temperature determines whether it chooses the most probable one or allows itself a less typical choice.

When to change the temperature?

  • Code writing – low temperature to avoid errors.
  • Writing stories, poetry, slogans – higher temperature for greater variety.
  • Data analysis or summaries – low, to maintain precision.

The name temperature comes from its resemblance to equations concerning physical phenomena.

Technically, temperature is the number by which we divide the components of the softmax function, making the output “flatter”—that is, the differences between the probabilities of specific words occurring are smaller.

“But dividing by 1 doesn’t change anything, and many sliders, e.g. in Google AI Studio, allow you to set the temperature from 0 to, for example, 1 or 2” – a simple transformation and simplification is applied here. In reality, moving the temperature level from 0 to 2 means changing the T parameter from 1 to 3.

What is top-p sampling? (nucleus sampling)

This is an alternative to temperature. Instead of randomly selecting from all available tokens, top-p selects only those tokens that together account for, for example, 90% of the total probability. It then samples from that set.

  • top-p = 0.1 – only highly certain responses.
  • top-p = 1.0 – complete randomness (all possibilities are in play).

Temperature and top-p can be used together, but often only one of them is used.

Other technical terms

Prompt

Prompt is the input text provided to the model. It can be a question, instruction, text fragment—anything the model should take into account when generating a response.

Context window (context window)

Context window is the maximum number of tokens that a model can “read” at once. GPT-3.5 has 4,096 tokens, while GPT-4 has up to 128k. If we exceed the limit, the model will “forget” older fragments. It is important to distinguish between the input context, meaning how much text ChatGPT is able to read, and the output context, meaning how much text ChatGPT can write.

Fine-tuning

Fine-tuning is the adaptation of a model to a specific application. Instead of building AI from scratch, an existing model is further trained on data specific to a company, field, or language.

Inference

This is the moment when the model generates a response. It takes a fraction of a second, but it can be computationally expensive—especially with large models.

Weight parameters ( weights )

Weight parameters are the fundamental mathematical components of every neural network—including models such as ChatGPT. In short, they are numbers that determine how strongly one element influences another during information processing.

We’re Bringing Back Autocorrect

Now that we know the basic concepts related to artificial intelligence, it is a good idea to return to the basic principles of autocorrection.

Levenshtein Distance (also known as Edit Distance)

This is one of the oldest and most commonly used measures. It determines how many operations (insertions, deletions, or substitutions of characters) must be performed to transform one word into another. The smaller the distance, the more similar the words are.

Example: “cat” → “horse” → distance = 1 (replacing “t” with “ń”)

N-gram Model

A statistical model that predicts a word based on its context. In the classic n-gram model, the probability of a word occurring is estimated based on the preceding n-1 words.

Example (bigram, i.e. 2-gram): If a user writes “I would like to eat pi,” the system assesses which word more often appears after “eat”: “pizza” or “pierogi.”

Formula: P(wₙ | wₙ₋₁) ≈ frequency(wₙ₋₁, wₙ) / frequency(wₙ₋₁)

Autocorrect features currently use many other mechanisms, including machine learning (which analyzes the most common errors), but in the basic version, they are based on the mechanisms mentioned earlier, while also taking the distance between letters on the keyboard into account. However, simple and fast language models are increasingly being used nowadays.

Levenshtein distance can be considered a simplified version of the mechanism of tokens and dependencies between tokens (although technically it is a completely different method), whereas the transformer and self-attention mechanism is a direct development of n-gram models.

Decision Trees and Random Forests – Simple Machine Learning Methods.

Although the solutions mentioned are not used in ChatGPT, it is worth knowing about them because they frequently appear in artificial intelligence topics. They are applicable where simplicity and speed are required and limited computing power is available.

A decision tree is a model that makes decisions based on “if… then…” rules. Each of its nodes checks a condition, such as whether a word is in the dictionary, differs by one letter from another word, or is preceded by a specific word. At the end of each branch, there is a decision: correct it or leave it unchanged. Such trees can be built automatically by training them on a large number of examples of real-world errors.

A single tree can be unreliable, which is why many of them are created—this is known as a random forest. Each tree receives slightly different data and features, then independently proposes a solution. The system makes the final decision based on voting. As a result, the model is resistant to errors, fast, and suitable for operation even on less powerful devices.

AI Basics: Summary

A language model is not a brain

It is worth remembering in conclusion: AI does not understand the world like a human. It has no consciousness, emotions, or intentions. A language model does not “know” that you exist. All it does is predict the next word based on the data on which it was trained.

However, thanks to the complexity of the network, the enormous number of parameters (billions), multi-stage processing, and the use of mechanisms such as embeddings, attention, and transformer layers, it can generate responses that seem sensible, accurate, and sometimes even brilliant.

Quick summary of the most important concepts:

layers – levels of information processing,

tokens – the basic units of text in the model,

embeddings – numbers describing the meaning of words,

attention – a mechanism for focusing on important fragments,

temperature and top-p – parameters affecting the creativity of responses,

Artificial intelligence is not just language models or image recognition.

In this text, I focused on language models and image recognition, which is why this article is relatively short. The knowledge from this text is already useful when using ChatGPT or other models in practice, and offers the greatest opportunities for “playing around.” In the next text, I will discuss photo recognition and how image generation works.

When it comes to building artificial intelligence, image recognition is the simplest (and even simple models already have practical applications), followed by text generation and then image generation.

However, it is important to remember that AI is not only about generating text and images.

Artificial intelligence is present in many of the technologies we use every day, often without realizing it.

  • Voice assistants: Siri, Alexa, and Google Assistant use AI to understand speech, process queries, and execute commands.
  • Recommendation systems: Platforms such as Netflix, Spotify, and YouTube analyze your preferences and, based on machine learning algorithms, suggest films, music, or videos you may enjoy.
  • Vehicle autonomy: Autonomous vehicles use AI to analyze data from sensors (radar, LiDAR, and cameras) and make real-time decisions regarding steering, braking, and obstacle avoidance.
  • Healthcare: AI assists in diagnosing diseases by analyzing X-rays and CT scans, as well as in discovering new drugs through chemical simulations.
  • Finance: Banks and financial institutions use AI to detect fraud, analyze credit risk, and forecast market trends.

The future of AI is not only about improving existing tools, but also about advancing in fields that until recently seemed impossible. These include, for example, robotics (robotic arms in factories, cleaning robots), environmental protection (analyzing satellite data to monitor pollution), and science (space and climate simulations).

In short, AI is a powerful tool for data analysis and process optimization, and its applications extend far beyond what we see on our computer screens.