In the world of Artificial Intelligence, making sense of human language is one of the hardest challenges. While a human can easily read a news article and instantly identify the people, companies, and countries mentioned, a computer just sees a string of zeroes and ones.
This is where Named Entity Recognition (NER)—a crucial subfield of Natural Language Processing (NLP)—comes into play.
In this guide, we’ll explain what NER is, how modern AI models perform it, and why it is essential for data analysis today.
What is an “Entity”?
In NLP, a Named Entity is a real-world object that can be denoted with a proper name. NER algorithms are designed to read a piece of raw text, identify these entities, and classify them into predefined categories.
The most common categories are:
- PER (Person): “Elon Musk”, “Marie Curie”, “Nelson Mandela”
- ORG (Organization): “Apple Inc.”, “United Nations”, “Google”
- LOC (Location): “New York”, “Mount Everest”, “Pacific Ocean”
- MISC (Miscellaneous): Nationalities, religious groups, events, or works of art.
If you input the sentence: “Tim Cook flew to Tokyo to visit the Sony headquarters.” An NER system will output:
Tim Cook➡️ PERTokyo➡️ LOCSony➡️ ORG
How Does NER Work?
Historically, developers used complex Regular Expressions (Regex) and massive dictionaries (gazetteers) to find names in text. If the word existed in a list of known countries, it was tagged as a Location. This approach failed miserably when faced with ambiguity (e.g., “Apple” the fruit vs. “Apple” the company).
Modern NER relies entirely on Contextual Machine Learning, specifically Token Classification.
Token Classification and BERT
Today’s gold standard for NER is the BERT (Bidirectional Encoder Representations from Transformers) architecture, created by Google.
Instead of reading words left-to-right, BERT reads the entire sentence at once (bidirectionally). This allows the AI to understand the context surrounding a word. When BERT sees the word “Apple”, it looks at the neighboring words. If the neighboring words are “CEO”, “revenue”, or “headquarters”, the AI assigns a high probability to the ORG tag. If the neighboring words are “juice”, “pie”, or “orchard”, it ignores the word.
Real-World Applications
NER is the invisible engine powering many of the digital tools you use every day:
- Customer Support Routing: Automatically identifying product names or company departments in a customer complaint ticket to route it to the right team.
- Search Engine Optimization: Helping search engines understand the actual subjects of an article, rather than just matching keywords.
- Medical Record Analysis: Extracting patient symptoms, drug names, and dosages from doctor’s unstructured notes.
- News Aggregation: Automatically tagging news articles with the names of politicians or companies they mention, allowing readers to filter news easily.
Try NER Locally
Traditionally, implementing NER required sending your data to cloud APIs. However, with the rise of browser-based AI, you can now run state-of-the-art BERT models directly on your device.
If you have a block of text and want to see NER in action without sending your data to a server, try our completely private, browser-based AI Entity Extractor. It uses Web Workers to process text in milliseconds, securely highlighting entities right in front of your eyes.