Developer GuideArchitecture
Integration Patterns
How to embed Docket into existing applications and systems.
Integration Patterns
Docket is designed to drop into existing systems without rewriting them. Here are the four primary integration patterns, from simplest to most decoupled.
Pattern comparison
| Pattern | Effort | Coupling | Best for |
|---|---|---|---|
| Embedded | Low | Tight | Node.js monoliths, internal tools |
| Sidecar | Medium | Loose | Microservices, polyglot stacks |
| API Gateway | Low | Loose | Existing REST APIs, mobile backends |
| Webhook | Lowest | Loose | Event-driven systems, SaaS connectors |
1. Embedded mode
Import Docket directly into your Node.js application. You control the server, the routes, and the lifecycle.
┌─────────────────────────────────────────────────────────────┐
│ YOUR NODE.JS APPLICATION │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Your Express / Fastify / NestJS server │ │
│ │ │ │
│ │ app.get('/api/search', async (req, res) => { │ │
│ │ const result = await docket.query({ │ │
│ │ question: req.query.q │ │
│ │ }); │ │
│ │ res.json(result); │ │
│ │ }); │ │
│ └─────────────────────────────────────────────────────┘ │
│ │ │
│ │ import { buildUnifiedApp } │
│ ▼ from 'docket' │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Docket Core (same process) │ │
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │
│ │ │ Store │ │ Blob │ │ LLM │ │ Embed │ │ │
│ │ │SQLite │ │FileSys │ │Ollama │ │Ollama │ │ │
│ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘Code example
const { buildUnifiedApp } = require('docket');
// Build Docket inside your app
const docket = await buildUnifiedApp({ logger: false });
// Use it directly
const memory = await docket.createMemory({
rawRef: 'uploads/photo.jpg',
contentType: 'image/jpeg'
});
const answer = await docket.query({
question: 'What did I photograph last week?'
});When to use
- You already have a Node.js backend and want to add memory without new infrastructure.
- You want to customize routes, auth, or middleware around Docket behavior.
- You are building a product where Docket is a feature, not the whole product.
Trade-offs
- Tight coupling — Your app and Docket share memory, event loop, and crashes.
- Harder to scale — You scale your app and Docket together, even if they have different bottlenecks.
- Version lock — Docket upgrades require app deployments.
2. Sidecar mode
Run Docket as a separate process on the same host. Your app talks to it over localhost HTTP.
┌─────────────────────────────────────────────────────────────┐
│ HOST │
│ │
│ ┌─────────────────────┐ ┌─────────────────────────┐ │
│ │ Your Application │ │ Docket Data Plane │ │
│ │ (any language) │ │ (Port 3000) │ │
│ │ │ │ │ │
│ │ fetch('http:// │ │ /ingest /query │ │
│ │ localhost:3000 │◄─▶│ /memories │ │
│ │ /query', ...) │ │ │ │
│ │ │ │ │ │
│ └─────────────────────┘ └─────────────────────────┘ │
│ │
│ ┌─────────────────────────┐ │
│ │ Docket Control Plane │ │
│ │ (Port 3001) │ │
│ │ — admin only │ │
│ └─────────────────────────┘ │
│ │
│ Both share ./data/docket.db │
│ (SQLite) or same Postgres │
│ │
└─────────────────────────────────────────────────────────────┘Code example (Python app talking to Docket)
import requests
# Ingest a document
requests.post('http://localhost:3000/ingest',
files={'file': open('report.pdf', 'rb')},
data={'async': 'false'}
)
# Query
response = requests.post('http://localhost:3000/query',
json={'question': 'What were the Q1 revenue numbers?'}
)
print(response.json()['answer'])When to use
- Your stack is polyglot (Python, Go, Ruby, Java) and you need memory capabilities.
- You want to upgrade or restart Docket independently of your app.
- You run in containers and want one Docket sidecar per app instance.
Trade-offs
- Network hop — localhost is fast, but still a hop.
- Operational complexity — Two processes to monitor, two logs to aggregate.
- Schema coupling — Your app depends on Docket's REST contract.
3. API Gateway mode
Docket sits behind your existing API gateway or load balancer. Clients hit your gateway; the gateway proxies memory routes to Docket.
┌─────────────────────────────────────────────────────────────┐
│ CLIENTS │
│ (Web app, mobile, third-party integrations) │
└─────────────────────────┬───────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ API GATEWAY / LB │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ /api/v1/users ──────▶ Your User Service │ │
│ │ /api/v1/orders ──────▶ Your Order Service │ │
│ │ /api/v1/search ──────▶ Docket Data Plane │ │
│ │ /api/v1/memories ──────▶ Docket Data Plane │ │
│ │ /admin/* ──────▶ Docket Control Plane │ │
│ └─────────────────────────────────────────────────────┘ │
│ │ │
│ │ routing rules │
│ ▼ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Docket Cluster │ │
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │
│ │ │ Data │ │ Data │ │ Data │ │ │
│ │ │ Plane 1 │ │ Plane 2 │ │ Plane N │ │ │
│ │ └────┬────┘ └────┬────┘ └────┬────┘ │ │
│ │ └─────────────┴─────────────┘ │ │
│ │ │ │ │
│ │ ▼ │ │
│ │ ┌─────────────┐ │ │
│ │ │ Shared DB │ │ │
│ │ │ (Postgres) │ │ │
│ │ └─────────────┘ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘Gateway routing rules (example)
# nginx / traefik / Kong config
routes:
- match: PathPrefix(`/api/v1/search`)
service: docket-data-plane
stripPrefix: false
- match: PathPrefix(`/api/v1/memories`)
service: docket-data-plane
- match: PathPrefix(`/admin`)
service: docket-control-plane
middlewares:
- ip-whitelist:
sourceRange: ["10.0.0.0/8"]When to use
- You already have an API gateway and want to add search/memory without new endpoints.
- You need to scale Docket data planes independently from your main services.
- You want to apply existing auth, rate limiting, and logging to Docket routes.
Trade-offs
- Gateway complexity — Routing rules must be maintained.
- Path collisions — Make sure Docket routes don't clash with your existing API.
- Latency — Extra hop through the gateway.
4. Webhook / event-driven mode
Your existing system pushes events to Docket's ingest endpoint. Docket processes them asynchronously and stores the results.
┌─────────────────────────────────────────────────────────────┐
│ YOUR EXISTING SYSTEM │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐ │
│ │ CRM │ │ Issue │ │ File Storage │ │
│ │ (Salesforce)│ │ Tracker │ │ (S3 / Drive) │ │
│ │ │ │ (Jira) │ │ │ │
│ └──────┬──────┘ └──────┬──────┘ └────────┬────────┘ │
│ │ │ │ │
│ │ webhook │ webhook │ webhook │
│ │ "new lead" │ "issue opened" │ "file uploaded"
│ ▼ ▼ ▼ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ DOCKET INGEST ENDPOINT │ │
│ │ POST /ingest (async) │ │
│ └─────────────────────────────────────────────────────┘ │
│ │ │
│ │ async queue │
│ ▼ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Docket Workers │ │
│ │ · extract text │ │
│ │ · embed │ │
│ │ · classify sector │ │
│ │ · store │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘Code example (webhook handler in your app)
// Your existing Node.js app
app.post('/webhooks/jira', async (req, res) => {
const issue = req.body;
// Forward to Docket for memory extraction
await fetch('http://localhost:3000/ingest', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
text: `${issue.key}: ${issue.summary}\n${issue.description}`,
metadata: { source: 'jira', issueKey: issue.key },
async: true // don't wait for processing
})
});
res.sendStatus(204);
});When to use
- You have event-driven architecture (webhooks, Kafka, SQS).
- You want to build a knowledge base from existing tools without migration scripts.
- You need fire-and-forget ingestion — Docket processes in the background.
Trade-offs
- Eventual consistency — Memories are not immediately queryable.
- Duplicate handling — Your webhooks may fire multiple times; Docket does not deduplicate by default.
- No feedback loop — If ingestion fails, the caller may not know.
Choosing a pattern
Are you building in Node.js?
│
┌──────────────┴──────────────┐
Yes No
│ │
▼ ▼
┌───────────────┐ ┌─────────────────┐
│ Do you need │ │ Do you have an │
│ tight control │ │ API gateway? │
│ over routes? │ └────────┬────────┘
└───────┬───────┘ │
│ ┌────────┴────────┐
Yes Yes No
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ Embedded │ │ API Gateway │ │ Sidecar │
│ (import) │ │ (proxy) │ │ (localhost) │
└───────────────┘ └───────────────┘ └───────────────┘
│
No
│
▼
┌───────────────┐
│ Sidecar │
│ (localhost) │
└───────────────┘
Do you use webhooks / events?
│
Yes
│
▼
┌───────────────┐
│ Webhook │
│ (fire/forget)│
└───────────────┘Multi-tenant integration
For SaaS products that serve multiple customers:
┌─────────────────────────────────────────────────────────────┐
│ TENANT A │
│ ┌─────────────────┐ ┌─────────────────────────────┐ │
│ │ Tenant A App │◄────▶│ Docket Data Plane │ │
│ │ │ │ (isolated namespace) │ │
│ └─────────────────┘ │ · prefix: "tenant-a/" │ │
│ │ · RBAC policy per memory │ │
│ └─────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────────┐
│ TENANT B │
│ ┌─────────────────┐ ┌─────────────────────────────┐ │
│ │ Tenant B App │◄────▶│ Docket Data Plane │ │
│ │ │ │ (isolated namespace) │ │
│ └─────────────────┘ │ · prefix: "tenant-b/" │ │
│ │ · RBAC policy per memory │ │
│ └─────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────┐
│ Control Plane │
│ (shared) │
│ · manage all │
│ tenants │
└─────────────────┘Each tenant gets a logical namespace via accessPolicy prefixes. The control plane manages all tenants centrally. This pattern is planned for post-v0.2.0.