Introduction
This is the story of building AM — a personal AI system designed to know me, work alongside me, and operate autonomously on my behalf. Not as a product or a service, but as a bespoke software system running on my own hardware.
Design Philosophy
The system is built on a few core principles:
- Privacy by default — all data stays on my hardware. No cloud dependency.
- Persistent memory — the system remembers everything we've discussed and acts on that context.
- Tool use — the AI can read files, run commands, search the web, and take actions.
- Autonomous operation — scheduled tasks and background monitoring without requiring my attention.
Architecture
The system has several key components:
- The agent runtime — the core loop that processes requests, selects tools, and generates responses
- Memory stores — both short-term session context and long-term persistent memory across sessions
- Tool system — an extensible set of capabilities the agent can invoke
- Profiles — different configurations for different use cases
Key Capabilities
Memory Across Sessions
The most transformative feature is persistent memory. The system doesn't forget what we discussed yesterday, last week, or last month. It remembers my preferences, projects, and recurring tasks.
Autonomous Task Execution
The system can be scheduled to run tasks on its own — checking website status, monitoring RSS feeds, running backups, or compiling reports. These run as background jobs and deliver results when ready.
Tool Integration
Through a plugin system, the agent can access file systems, run terminal commands, search the web, manage containers, and interface with APIs. This makes it far more capable than a chat-only system.
What I've Learned
Building a personal AI is an ongoing journey. The most important lessons:
- Start simple — a working minimal system is better than an ambitious plan
- Memory quality matters more than model quality — a smaller model with good context beats a larger model that forgets
- Let it act — the real value comes when the system does things, not just when it talks