Named Entity Recognition (NER)

Named entity recognition (NER) is a natural language processing (NLP) task that finds the spans of text naming real-world things and labels each span with a type such as person, organization, location, date, or monetary amount. In "Marie Curie taught at the University of Paris from 1906," an NER model marks "Marie Curie" as a person, "University of Paris" as an organization, and "1906" as a date. This turns emails, reports, contracts, and transcripts into structured records that search engines, knowledge graphs, analytics, and privacy tools can use. The Message Understanding Conferences of the 1990s made NER a standard evaluation task, and the CoNLL-2003 shared task fixed the four types still used as a benchmark: person, location, organization, and miscellaneous. Larger schemes such as OntoNotes 5.0 define 18 types, including dates, quantities, products, and events.

How NER Works#

NER is usually framed as sequence labeling. A tokenizer splits the text into tokens, and the model assigns each token a tag in the BIO scheme: B- marks the beginning of an entity, I- a token inside it, and O a token outside any entity, so multi-word names such as "University of Paris" stay together. This code turns the tags a model predicts into entity spans:

# Token-level BIO tags, as predicted by an NER model for one sentence
tokens = ["Marie", "Curie", "taught", "at", "the", "University", "of", "Paris", "from", "1906", "."]
tags = ["B-PER", "I-PER", "O", "O", "O", "B-ORG", "I-ORG", "I-ORG", "O", "B-DATE", "O"]

# Merge each B- tag with the I- tags that follow it into one entity span
entities = []
for token, tag in zip(tokens, tags):
    if tag.startswith("B-"):
        entities.append([token, tag[2:]])
    elif tag.startswith("I-") and entities and entities[-1][1] == tag[2:]:
        entities[-1][0] += f" {token}"

print(entities)

It prints [['Marie Curie', 'PER'], ['University of Paris', 'ORG'], ['1906', 'DATE']], the structured output that downstream systems store and query.

The models that predict the tags have changed over three decades. Early systems used hand-written rules and gazetteers, lists of known names. Statistical taggers such as hidden Markov models and conditional random fields learned from annotated text with hand-crafted features such as capitalization and word suffixes. In 2016 bidirectional LSTM networks with a CRF output layer learned those features themselves, and fine-tuned transformers followed: BERT reached an F1 score of 92.8 on the CoNLL-2003 test set. Self-attention lets these models use the whole sentence as context, so "Apple" is tagged as an organization in "Apple released a new phone" and left untagged in "I ate an apple." Systems are scored with entity-level precision, recall, and F1 score, where a prediction counts only if both its span and its type match the reference exactly.

Newer models drop the fixed label set. GLiNER, a compact bidirectional transformer, takes entity type names as input and finds matching spans for types it never saw in training, and large language models extract entities when prompted with a schema and asked for structured outputs.

NER vs. Related Tasks#

• Entity linking: Connects each recognized mention to a specific entry in a knowledge base such as Wikidata, so "Paris" resolves to the city or to Paris Hilton.
• Relation extraction: Finds how entities are connected, such as which person works for which organization, often on top of NER output.
• Sentiment analysis: Measures the opinion a text expresses. NER identifies what a review talks about, and sentiment analysis measures how the writer feels about it.
• Object detection: The image counterpart of NER, locating regions of an image and labeling each with a class.

The open-label idea of GLiNER has a direct counterpart in vision: YOLOE-26, the open-vocabulary version of Ultralytics YOLO26, takes class names as text prompts at inference time and detects and segments objects of those classes, even ones outside its training categories.

Real-World Applications#

Document processing combines NER with optical character recognition (OCR): invoices, contracts, and forms are scanned, OCR reads the text, and NER pulls out fields such as vendor, date, and total amount for accounting and compliance systems. Production pipelines send uncertain predictions to people and learn from their corrections:

• PII redaction: Privacy tools such as Microsoft's open-source Presidio use NER together with pattern rules to find and mask names, addresses, and account numbers before text is shared or used for training.
• Clinical text mining: Clinical NER extracts medications, dosages, symptoms, and diagnoses from doctors' notes so hospitals and researchers can search and analyze records at scale.
• Search and recommendations: News sites and content platforms tag articles with the people, companies, and places they mention, which powers entity search and suggests related stories.