Chatbot

A chatbot is a computer program that holds a conversation with people in natural language, by text or by voice. Joseph Weizenbaum's ELIZA, written at MIT and published in 1966, is usually credited as the first chatbot: its DOCTOR script imitated a psychotherapist by matching keywords and turning the user's statements back into questions, and Weizenbaum was disturbed to see users, including his own secretary, confide in it as if it understood them. Chatbots have since gone through three generations, from rule-based scripts to intent-based bots to large language models (LLMs). OpenAI's ChatGPT, launched on November 30, 2022, made the third generation mainstream, and "chatbot" now usually means a chat interface to an LLM such as ChatGPT, Claude, or Gemini.

How Chatbots Work#

• Rule-based chatbots: Match the user's words against hand-written patterns and reply from templates. They are predictable and cheap, and they fail on any phrasing their authors did not anticipate.
• Intent-based chatbots: A natural language understanding (NLU) classifier maps each message to an intent, such as checking a balance, and extracts details like an account or date, then designers' dialog flows decide the reply. Banks, airlines, and telecom companies built customer-service bots this way because every answer could be reviewed in advance.
• LLM chatbots: Generate each reply one token at a time from the conversation so far. A system prompt sets the bot's role and rules, and the model handles phrasing no one anticipated.

The model behind an LLM chatbot is a transformer trained for dialogue with instruction tuning and reinforcement learning from human feedback (RLHF), and it has no memory between calls. The application stores the conversation and sends all of it, formatted by the model's chat template, with every new message:

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM2-135M-Instruct")

# The chatbot stores the conversation and sends all of it with every new turn
messages = [
    {"role": "system", "content": "You are a support assistant for an online store."},
    {"role": "user", "content": "Where is my order?"},
    {"role": "assistant", "content": "Could you share your order number?"},
    {"role": "user", "content": "It's 4417."},
]

# The chat template flattens the history into one prompt for the model to continue
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
print(prompt)

The printed prompt wraps each turn in <|im_start|> and <|im_end|> markers with its role and ends with an open <|im_start|>assistant line, so the model's next tokens become the reply and "It's 4417" makes sense only because the earlier question is in the same prompt. Every turn makes this prompt longer, so long conversations are trimmed or summarized before they exceed the context window, and prompt caching avoids reprocessing the unchanged history on each turn.

Grounding, Tools, and Images#

A support chatbot that answers from its training data alone will state wrong policies with confidence, so production bots retrieve the relevant documents and account data first with retrieval-augmented generation (RAG), check inputs and drafts with AI guardrails, and hand off to a person when a reply cannot be supported:

Chatbots that call tools through function calling, such as looking up an order, are becoming AI agents, and multimodal chatbots accept photos and screenshots in a turn through vision-language models (VLMs). The Ultralytics LLM interface sends a prompt, an image, or a full message history to any OpenAI-compatible chat model and returns the provider's own response object, so a YOLO26 pipeline can ask a multimodal model to describe a scene only after detection finds a person in it.

Risks#

Hallucinations carry legal weight when a business deploys the bot. In Moffatt v. Air Canada, decided in February 2024, the airline's website chatbot told a grieving passenger that a bereavement fare discount could be claimed after travel, which contradicted the airline's actual policy. British Columbia's Civil Resolution Tribunal rejected Air Canada's argument that the chatbot was responsible for its own statements and ordered the airline to pay the difference. Chatbots also face prompt injection and jailbreaking attempts, and models tuned to please users can drift into AI sycophancy, agreeing with false claims instead of correcting them.

Chatbots vs. Related Terms#

• Virtual assistant: Software such as Siri or Alexa that carries out requests across apps and devices. A chatbot usually lives in one website, app, or messaging channel and focuses on conversation.
• AI agent: A system that pursues a goal by choosing and executing actions, often over many steps without a person in the loop. A chatbot becomes an agent once it acts through tools as well as answering.
• Conversational AI: The broader field of technologies behind chatbots and assistants, including speech recognition, NLU, dialog management, and text-to-speech.