BERT (Artificial Intelligence)

What is BERT?

BERT (abbreviated from Bidirectional Encoder Representations from Transformers) is an artificial intelligence model and machine learning algorithm used by Google to understand search intent.

Google has announced BERT as open-source project in 2018. Before BERT, Google determined the search intention by analyzing the keywords in the query. With BERT, Google uses technology Natural Language Processing (NLP).

Instead of just comparing the keywords from the query with the keywords on the results pages, BERT recognizes keywords in the query and analyzes context in which they were used. This allows Google to better understand what the user is actually looking for and to show them more relevant results.

BERT is not a generative AI like ChatGPT or Gemini. He is language comprehension model, intended for tasks such as classifying text, answering questions and recognizing entity names. BERT learns by analyzing existing text, anticipating hidden words in sentences, rather than creating new content.

Generative models such as GPT produce new text – sentences, paragraphs or whole documents. In contrast, BERT specializes in Analysis and understanding of the existing text, Not for its generation.

How does BERT differ from other AI models?

By the time it was published, BERT was unique because it could understand context in which a word is used.

Other AI models then could only predict the words to come after some words or groups of words – they were one-way (from left to right).

BERT, however, can predict the words to come Before and After Some words. That's why he bidirectional – analyse the text in both directions: from left to right and from right to left.

How BERT works

Classical machine learning models convert words into numbers (so-called vectors). A model like word2vec would use the same vector for the word chair in both sentences, without differentiating meanings.

BERT, however, uses different vectors because he understands that the word is used in different contexts. In other words, they are treated as two different words even though they are written the same. This allows him to more accurately predict which words should appear in a sentence.

One of the main ways to train BERT is to use masked words. These are words that are deliberately hidden during training.

When searching, BERT first recognizes the most important words in the query, then analyzes the words around them to understand their context. That is why it can reveal the real Intention to search.

How BERT improves Google search results

Google uses BERT to better understand What the User Really Wants. This is particularly useful for conversational queries, long-tail queries and queries that contain important suggestions such as ‘for’ and ‘to’, as they often have a key meaning.