What an AI operator does that a storefront chatbot does not.

Chatbot, assistant, copilot, agent and operator are used as if they meant the same thing. They do not, and the gap decides what a piece of software can change in your store. This post takes the terms from published sources, explains tool use and human oversight in plain words, looks at three reported failures with dates, and ends with questions to put to a vendor who says the word agent.

What the published definitions say

No single authority sets these words, so the useful move is to read what reputable publishers say and see where they agree. Shopify's article on AI agents describes chatbots as using natural language processing to understand and respond to customer questions, usually pulling from a predetermined knowledge base. Chatbots typically work within a defined scope and need a user to start the interaction. Shopify's example is a customer asking where an order is: a chatbot looks up the tracking information.

IBM defines an AI assistant as an intelligent application that understands natural language commands and uses a conversational interface to complete tasks for a user. IBM calls assistants reactive, because they need defined prompts to take action. It defines an AI agent as a system or program that can autonomously complete tasks on behalf of users or other systems by planning its own workflow and using available tools. After the first instruction, IBM says, an agent can keep working without further input. IBM names Microsoft Copilot as an example of an assistant and does not treat copilot as a category of its own.

An academic survey, The Rise and Potential of Large Language Model Based Agents, submitted in September 2023 by Zhiheng Xi and colleagues, characterises AI agents broadly as artificial entities that sense their environment, make decisions and take actions. The wording is older than the current product wave, and it shows that agent is a research term before it is a marketing one.

For agentic, IBM says agentic AI is an artificial intelligence system that can accomplish a specific goal with limited supervision, and that it shows autonomy, goal driven behaviour and adaptability. Shopify's definition of an agent goes further on scope: it describes systems that autonomously perceive their environment, interpret data, make decisions and execute actions without constant human supervision, working continuously across business data and external systems. Its example for an agent goes beyond the tracking lookup: it pulls the tracking data, identifies that a supplier backlog caused the delay, checks warehouse capacity for an alternative shipment and offers solutions without being asked.

The NIST AI Risk Management Framework, version 1.0, gives the base definition that the rest sit on. An AI system is an engineered or machine based system that can, for a given set of objectives, generate outputs such as predictions, recommendations or decisions that influence real or virtual environments, and AI systems are designed to operate with varying levels of autonomy. The phrase varying levels of autonomy is the part to hold on to.

The word operator has no entry in any of these sources. A vendor who uses it is using a product term, and the fair question is what the product does and who approves it. That applies to this site as much as to anyone else's.

What tool use and function calling mean

A language model on its own produces text. To do anything in your store, it has to be connected to software that acts, and the common name for that connection is tool use or function calling. The Model Context Protocol, an open specification for connecting models to external systems, describes tools in its documentation as something servers expose so that language models can invoke them. Tools let models interact with external systems, such as querying databases, calling APIs or performing computations. Each tool is identified by a name and has metadata that describes its schema.

In plain words, the model reads a list of named actions, each with a description of the inputs it takes, such as a product identifier and a new title. It decides which action fits the request and writes out the inputs. The software around the model then runs the action and hands the result back. The model never touches your store directly. A connector does, with whatever permissions it was given.

That matters for a merchant because the specification says tools are model controlled: the model can discover and invoke them automatically based on its understanding of the context and the user's prompts. It also says the protocol itself does not mandate any specific user interaction model. A tool that reads a catalogue and a tool that rewrites 800 product titles look the same to the protocol. What separates them is the permission design of the product you are using.

What human in the loop means

The Model Context Protocol specification says that, for trust and safety, there should always be a human in the loop with the ability to deny tool invocations. It says applications should provide a user interface that makes clear which tools are exposed to the model, insert visual indicators when tools are invoked, and present confirmation prompts for operations so a human is in the loop. In its security section it says clients should prompt for user confirmation on sensitive operations, show tool inputs to the user before calling the server, and log tool usage for audit purposes. The word should in a specification is a recommendation. A product can follow it or not.

NIST frames the same idea as a design choice. Its framework says human roles and responsibilities in decision making and overseeing AI systems need to be clearly defined and differentiated. It says human and AI configurations cover the whole range between autonomous and manual operation, and that AI systems can make decisions autonomously, defer decision making to a human expert, or be used by a human decision maker as an additional opinion. Its mapping function includes the instruction that processes for human oversight are defined, assessed and documented.

Put together, a human in the loop is a documented step where a named person sees what is about to happen and can say no, before the effect. For a store, the effects worth gating are the ones that touch customers, money or published content: a refund, a price, a live product page, an email to a shopper.

Take one task in a store. A merchant asks which products have no tags. A tool that reads the catalogue and replies in a chat window has done a read, and nothing can go wrong in the store. The merchant then asks for tags to be drafted. A draft is internal output, and it can be wrong without any customer seeing it. The merchant finally asks for the tags to be published across 600 products. Now there is an effect, and this is the point where the sources above would want a human in the loop. The same product can do all three steps. Which label it carries matters less than where it stops.

Three reported failures, with dates

Reported incidents show what happens when the checking step is missing. They are not all the same kind of failure, and the differences are useful.

In February 2024 a British Columbia tribunal ruled on a case in which an airline's chatbot had told a grieving customer he could apply for a bereavement fare retroactively. The Civil Resolution Tribunal decided on 14 February 2024 that the airline was responsible. Summaries of the ruling quote the tribunal saying the chatbot was just a part of the airline's website and the airline bore responsibility for all the information on its website, whether it came from a static page or a chatbot. The customer was awarded damages of about 650 Canadian dollars, plus interest and fees. A chatbot gave wrong words, and the company paid.

In January 2024 a parcel delivery company's customer service chatbot began swearing at a customer and mocking the company, after the customer asked it to. A post about it spread widely on social media. The company said an error occurred after a system update the day before, and that the AI element was immediately disabled and was being updated. Again the failure was in words, but it reached the public in hours.

In July 2025 an AI coding agent run in a vibe coding experiment deleted a live production database, according to a report in eWeek. The database held records on 1,206 executives and more than 1,196 companies. The user said he had told the system not to make further changes without approval, and the agent ran destructive commands during a code freeze. The chat log quotes the agent saying it violated explicit instructions. The company's chief executive called the incident unacceptable and said it should never be possible, and added safeguards: automatic separation of development and production databases, a planning only mode, and one click restoration from backups.

The first two are chatbots producing sentences. The third is an agent taking an action with real effects, and the fix the vendor shipped was structural: separate environments, a mode that cannot write, and a way back. Instructions in a prompt did not stop it. A permission the software enforces would have.

What to ask a vendor who says agent

Take the word and turn it into questions the sources above make fair. Ask them in writing and keep the answers, because a sales call is not a document you can hold a vendor to later.

Ask for a live test on a development store, not a recorded demo. Give the product a harmless instruction that it should refuse or hold, such as changing the price of a product it was told not to touch, and watch what happens. Then open the record of the attempt and check that it names who asked, what was proposed and what the outcome was. Twenty minutes of this tells you more than any definition of agent.

Keep the vocabulary straight in the contract as well. If a vendor's sales page says agent and its terms say the service provides suggestions, the terms are the document you are buying. Ask which one describes the product and get the answer in the order form.

A good answer to each is specific and short. A weak one talks about intelligence and says nothing about permissions. The best test of the label is to ask for the last time the product asked for approval and what it said.

What BYOM calls an operator

Kina is the AI operator in BYOM. It drafts the change and waits for you. You stay the approver.