What Is Software, and Will LLMs Replace It? - Strumenta

Open original

The article argues that large language models lack the structured data, integrity constraints, and deterministic processes that business software provides. Instead of replacing software, LLMs will plug into existing systems via protocols like MCP, serving as a flexible front end while databases and rule engines remain the backbone.

  • Structured software (e.g., CRMs) enforces data integrity and constraints—like requiring a company record before logging a deal—that free-form LLM chat can't guarantee or replicate reliably.
  • The Air Canada chatbot case, where misleading bereavement fare info led to tribunal costs, shows the real risk of letting LLMs act without deterministic guardrails.
  • Protocols like MCP let LLMs plug into existing systems (Salesforce, Jira) to query and act within enforced schemas/permissions, rather than replacing those systems outright.
  • The likely future is LLMs as a flexible front end sitting behind deterministic software and databases, not a wholesale replacement of traditional business systems.

We’ve grown used to asking LLMs for everything: “Show me sales for the last five years,” and instantly we get charts or slide decks. But this ease masks what traditional software still does best. Take a CRM: it stores opportunities as structured records linked to companies, contacts, lead sources and past contracts. That structure lets you run reliable queries—say, “which referrals closed above €50k in six months?”—in milliseconds. A free-form chat can’t enforce data integrity or guarantee the same answer next month. Beyond data storage, software enforces rules you’d quickly lose if you trusted only an LLM. You can’t record a deal without first creating the related company. You can’t delete a company while contracts remain open. These constraints guard against garbage data down the line. Visual tools—graphs, filters, dashboards—let you spot outliers and trends faster than any text response. And built-in workflows capture years of domain know-how: approvals, required attachments, sequence of steps. That’s not friction—it’s the guardrails that keep your operation running smoothly. The key isn’t to replace these systems with chatbots but to connect LLMs to them. Tools like the Model Context Protocol (MCP) act like USB-C for AI: a single interface letting models access real databases, enforce schemas and permissions, and trigger valid actions. Salesforce’s Agentforce and Atlassian’s remote MCP servers don’t dissolve CRMs or Jira into conversation; they let LLMs reach into those systems on their terms. Industry cases, like Air Canada’s chatbot misguiding a bereavement fare claim and costing the airline in tribunal fees, show what happens when you skip those guardrails. In the end, LLMs will sit behind deterministic software, not replace it.

Ask about this article