truck8.ai

ChatGPT or Custom AI on Your Own Business Data

A business ChatGPT, a no-code workflow or your own agent working in your systems: what each step can do, where it stops and what happens to your data.

8 min readTom Mekenkamp

ChatGPT or custom AI: three steps for AI on your business data

'Can we make ChatGPT work on our own data?' I get that question in almost every first conversation. One person means: I want to search our manuals. Another means: I want AI to enter orders in our ERP. Those are two very different things, with a different price tag and a different risk.

I see it as a staircase with three steps. At the bottom, a business plan for ChatGPT, Claude or Copilot, where you upload files and connect your mail and your drive. In the middle, no-code automation that puts AI steps into your existing workflows. At the top, your own agent that works in your systems, with permissions per user and a log of everything it does.

Most companies don't need the top step. A few do. Below, for each step, you'll find what it can do, where it stops and how to tell where you stand.

Step 1: a secure business GPT with your own files

A business plan does more than most people think. You put documents in a project, share that project with colleagues and ask questions about it. Through connectors you link the assistant to Google Drive, SharePoint, your mailbox or your calendar. ChatGPT Business (formerly Team) and Enterprise use company knowledge to search several connected sources at once, and respect the permissions people already have in those sources. With Claude, an admin first enables a connector such as the one for Microsoft 365 for the organisation, after which each employee connects their own account.

Microsoft 365 Copilot sits closest to your data if your company already works in Microsoft 365. It works with your mail, Teams chats and SharePoint documents, and only shows information the user has at least view access to.

When is this enough? When people mostly ask questions, write and summarise based on documents that are already in good order. A salesperson preparing a quote with the product sheets in a project. An HR officer searching the staff handbook. A director putting five reports side by side. Which of the three fits your company best is covered in ChatGPT vs Copilot vs Claude for business.

Where it stops: everything starts with a conversation. An employee asks, the assistant answers or carries out a single action. A fixed workflow that runs the same steps for every incoming request, with your rules and checks, is not something you build here. And the systems your business really runs on, such as your ERP, your planning software or an industry package, often have no connector.

Step 2: no-code AI automation for SMEs

The next step is a workflow that runs by itself. With tools like Make, Zapier or n8n you put an AI step between two systems: an email arrives, AI extracts the details, the details go into your CRM and someone gets a notification. This is the step where many SMEs get the best return, because no developer is involved and the work happens every day without anyone starting it.

It works well for sorting customer questions, pulling data out of PDFs and forms, and preparing draft replies. How to set up a first workflow like that is described step by step in building your own AI agents.

Where it stops: no-code works as long as the flow is straight. Add exceptions (a customer with different terms, a request that is half complete, an order split across two sites) and your scenario grows into a tangle of branches nobody can follow. And the AI step only sees what you hand it in that one step. It can't decide for itself to look up the customer history or the stock level.

Step 3: an AI agent on your own data and systems

A custom agent is software that combines a language model with access to your sources and systems. It differs from the first two steps in four ways.

  • Searching your own documents (retrieval). For every question, the agent finds the relevant passages in your manuals, contracts, price lists and old quotes, and answers from those, with the source attached. You decide which documents go in and which version counts.
  • Working in your systems (tool use). The agent looks up a customer in your CRM, checks stock in your ERP or prepares a draft order. It chooses at each step which data it needs, instead of working only with what a fixed step hands it.
  • Permissions per user. The agent only sees and does what the person behind the request is allowed to. A planner sees different data from the managing director, and an agent working for sales can't change invoices.
  • A log. Every question, every source, every action and every approval is recorded. If a customer calls in three months about a wrong price, you can see what the agent read, what it proposed and who approved it.

When do you need your own agent?

When the work runs through several systems, when it involves decisions that depend on data you only fetch along the way, or when you need to show who decided what. That last one weighs more than most people expect.

I build these agents myself, with Claude as the model and the Model Context Protocol (MCP) for the connections to systems. The principle works with other models too. A worked example, from incoming request to order in the ERP, is in AI for quotes and order entry.

What goes wrong with AI on business data

On every step I see the same three problems. They have little to do with the model and everything to do with the data.

Outdated documents

An assistant that finds last year's price list quotes it with the same confidence as the new one. A shared drive often holds five versions of the same document. Pick one source per topic, clean up or exclude the rest, and make someone responsible for that source. It's dull work and it pays off more than any model choice. That's why, when I implement AI, I almost always start in the shared drive, not in the AI tool.

Access rights that are wrong

Connectors respect the permissions in your source system. That sounds reassuring, until you find out the salary folder has been open to everyone for years. Nobody used to find that folder. Now the assistant finds it in a second for whoever asks. Review your permissions before you switch on a connector, not after.

Made-up answers when data is missing

If the answer isn't in your documents, a language model is happy to fill the gap itself. Ask for the lead time of a product that isn't in the catalogue, and you get a plausible number. With a custom agent you can catch that: let it answer only with a source, and let it say 'I don't know' when there is none. With a standard plan it depends on the discipline of the user.

Secure AI for business: where does your data go?

Security is the second question in every conversation. These are the facts that matter.

  • Training. On business plans, the big vendors don't train on your data by default. OpenAI says so for ChatGPT Business, Enterprise and the API, Anthropic for Claude for Work and the API, and Microsoft for Microsoft 365 Copilot. Free and personal accounts are a different story.
  • Feedback. If someone on a Claude organisation account gives feedback on an answer, Anthropic may use that conversation. An admin can switch the feedback button off for the whole organisation.
  • Storage in Europe. OpenAI offers data residency in Europe for Enterprise, Edu and the API, not for Business. For European customers, Microsoft 365 Copilot falls under the EU Data Boundary, except for the Claude models Microsoft offers inside Copilot.
  • Who sees what. With connectors, the source system decides what someone gets to see. With a custom agent, you decide, per role and per system.
  • Which model, which region. With a custom agent you choose the model, the provider and the region where processing happens. With a standard plan you choose from what the vendor offers.

The basic rules still apply

A business plan doesn't mean you can stop thinking about what you put in. Which data never belongs in an AI tool and which agreements you need under the GDPR is covered in using ChatGPT safely in business.

What does AI integration cost your business?

Step 1 is mostly licences and setup: creating projects, cleaning up sources, switching on connectors and teaching your team to work with sources. Step 2 costs build time per workflow, plus maintenance when one of the connected systems changes. Step 3 is the biggest piece of work, and most of it isn't building. It's working out the process, the permissions and the exceptions.

A comparison with project pricing, and the costs people tend to forget, is in what AI implementation costs.

ChatGPT or custom AI: how to choose

Don't skip a step because the top one sounds more exciting. A team that has never worked with a business assistant doesn't know what to ask of an agent. And an agent on messy data makes the same mistakes, only faster. Pick the lowest step that can handle the work.

  • Step 1 if your team mostly asks questions and writes based on documents that are already in order.
  • Step 2 if the same work comes back every day in the same way and runs between two or three systems.
  • Step 3 if the work runs through several systems, permissions differ per person, or you need to show afterwards who decided what.

Key takeaways

  • AI on your own business data comes in three steps: a business plan with connectors, no-code automation, and your own agent in your systems.
  • A business ChatGPT, Claude or Copilot is enough as long as people ask questions and write based on documents that are in order.
  • A custom agent adds retrieval with sources, actions in your CRM or ERP, permissions per user and a log.
  • The biggest risks are in the data: outdated documents, permissions that are too broad, and answers to questions no source covers.
  • Business plans don't train on your data by default. Storage in Europe and control per role differ by vendor and by plan.
TM

Written by

Tom Mekenkamp

AI consultant & founder of truck8.ai

15+ years leading transformations at AB-InBev, Royal BAM and beyond. Now building AI products and helping SMEs implement AI.

Which step fits your business?

I look at your work, your data and your systems with you, and set up what fits: a business AI plan that is configured safely, a workflow that runs by itself, or your own agent on your ERP.

View AI Implementation