AI Isn't Magic. It's a Multiplier.
Over the past two years, almost every founder has asked us the same question: "Can AI help us pull all our company knowledge together?"
The honest answer: yes, but only if your know-how is already structured. AI doesn't have the ability to turn chaos into clarity. It has the ability to amplify whatever order already exists. Dumping a pile of scattered WhatsApp threads into a RAG pipeline doesn't give you magic answers — it gives you statistically plausible but logically wrong hallucinations.
Three Scenarios Where AI Actually Works in Enterprise KM
1. RAG Over a Structured SOP Library
If you already have a well-organised body of SOP documents in Outline, Notion, or Confluence, RAG (Retrieval-Augmented Generation) is the most reliable entry point. The mechanic: embed each document into chunks, index them, retrieve top-k relevant chunks at query time, then feed those into the LLM to generate an answer.
Preconditions for it to work:
- Documents are well-structured (clear title, scope, decision rationale per piece).
- Chunk size and overlap are tuned (typical 512–1024 tokens / 50–100 overlap).
- You have an evaluation set (at least 50 common questions with reference answers) for regression checks.
Where it fails: when the documents themselves are incomplete, mutually contradictory, or full of stale info. The AI cannot tell the difference.
2. Agents Over Business Systems
A second pattern with genuine traction: using an LLM agent to automate low-frequency, cross-system actions. For example:
"Find the enterprise customers we closed deals over HK$500,000 with in 2023 and haven't followed up with in the last six months."
This query joins CRM + contract data, filters by time range, and resists pre-baked BI dashboards (you can't hardcode every shape of query). An agent with tool-calling can dynamically compose SQL or API calls.
Preconditions: your business systems expose well-documented APIs, and you have at least an internal data dictionary.
3. Fine-Tune for Brand Voice
This is one of the few scenarios that genuinely warrants fine-tuning (rather than RAG or prompt engineering): when you want the LLM to write in a specific tone, style, or vocabulary — marketing copy, proposal drafts, customer email replies.
But fine-tuning has real cost and maintenance overhead. In most cases, three to five few-shot examples plus a detailed style guide in the prompt are enough. Reserve fine-tuning for genuinely high-volume, style-critical use cases.
Anti-Patterns We Don't Recommend
- Dumping all Slack / WhatsApp threads into RAG. Conversation messages are full of context-dependent information; stripped of that context, they become noise.
- Using LLMs as a substitute for judgement. LLMs are good at recall, summarisation, and draft generation. They are not good at judgement.
- Going to production without evaluation. A wrong LLM answer often looks just like a correct one. Without an eval set, you have no way to detect regressions.
Where to Start
Pick one real question your team asks every week. Build a minimum RAG demo with 5–10 documents. Try it yourself twenty times, then let five colleagues use it for a week. Do not start by building an enterprise-wide system. Most large AI knowledge projects die from over-scope.