How to Build Your Own AI Coding Agent on a VPS

What Is a Self-Hosted AI Coding Agent?

A self-hosted AI coding agent is a program that reads a codebase, plans changes, and executes them on hardware the developer controls, using a language model that runs locally rather than behind a vendor API. The distinction matters in one concrete way: source code never leaves the machine. No retention policy to audit. No terms-of-service revision to re-read next quarter.

The tradeoff is real. A 7-billion-parameter model on CPU will not match Claude or GPT-5 on hard reasoning. For refactors, boilerplate, test scaffolding, and review comments, a small local model is entirely adequate. Match the job to the tool.

We measured this setup on a production Debian 13 server running 8 AMD EPYC cores and 23 GiB of RAM. That box also hosts a web server, a database, and roughly a dozen scheduled jobs. In practice a coding agent is a tenant on existing infrastructure, not a dedicated machine.

What Hardware Does an AI Coding Agent Need?

An AI coding agent needs three things from a VPS: RAM to hold model weights, disk for the model library, and CPU threads to keep inference tolerable. Below are the exact specifications of the server these instructions were written on, read directly from the running system in 2026 rather than estimated.

Skip the GPU. For an agent answering a few dozen prompts per day, a GPU instance costs roughly $200 to $500 per month against $20 to $60 for comparable CPU capacity, and it idles between requests. CPU inference is slower per token and far cheaper per month.

How Do You Choose a Model That Fits Your RAM?

Model selection is arithmetic. Weights must fit in RAM with headroom for the OS and other services. For example, the table below lists on-disk sizes measured from our own Ollama library in 2026, not rounded vendor figures.

Start with the 3B. Seriously. Most engineers reach for the largest model RAM allows, discover responses take 40 seconds, and abandon the agent within a week. A 3B model answering in 5 seconds gets used daily, and daily use is the only metric that matters. In practice the smallest useful model beats the most capable unused one, every time.

How Do You Set Up the Environment?

Environment setup takes about 10 minutes on a clean Debian 13 install. Three components are required: Docker for sandboxing, Ollama for inference, Python 3.13 for agent logic. First install base packages, second add Docker, third pull a model.

# Step 1: base packages
sudo apt update && sudo apt upgrade -y
sudo apt install -y python3 python3-venv python3-pip git curl

# Step 2: Docker
curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER

# Step 3: Ollama and a model
curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen2.5-coder:3b

One warning that cost us an afternoon. Debian 13 enforces PEP 668, so pip install against the system Python fails with an externally-managed-environment error. Use a virtualenv. Reaching for --break-system-packages as a habit eventually collides with an apt upgrade and leaves a broken Python interpreter at the worst possible moment.

How Do You Build the Agent Core?

The agent core is a retrieval loop: it embeds the codebase into a vector store, retrieves relevant files for a task, and sends them to the model with instructions. Roughly 25 lines of Python using LangChain and Qdrant produces a working version, as the example below shows.

import asyncio
from langchain_ollama import OllamaLLM
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_qdrant import QdrantVectorStore

class CodingAgent:
    def __init__(self):
        self.llm = OllamaLLM(model="qwen2.5-coder:3b")
        self.embeddings = HuggingFaceEmbeddings(
            model_name="sentence-transformers/all-MiniLM-L6-v2"
        )
        self.store = QdrantVectorStore.from_existing_collection(
            collection_name="codebase",
            embedding=self.embeddings,
            url="http://localhost:6333",
        )

    async def analyze(self, question: str, k: int = 4):
        docs = self.store.similarity_search(question, k=k)
        context = "\n\n".join(d.page_content for d in docs)
        prompt = f"Context:\n{context}\n\nTask: {question}"
        return await self.llm.ainvoke(prompt)

Retrieval quality decides output quality. Feed a model 4 irrelevant files and it produces confident nonsense. The embedding step deserves more attention than prompt wording.

How Do You Sandbox an AI Coding Agent?

An AI coding agent with shell access is, architecturally, a remote code execution path installed on purpose. Sandboxing is therefore mandatory, not optional. Every agent action should run inside a container with explicit resource ceilings and zero ambient credentials, as in the hands-on configuration below.

docker run --rm \
  --network none \
  --memory 2g --cpus 2 \
  --read-only \
  --tmpfs /tmp:rw,size=256m \
  -v "$PWD:/work:ro" \
  agent-sandbox python /work/task.py

Four flags carry the weight. First, --network none blocks exfiltration. Second, --memory 2g and --cpus 2 stop a runaway loop from taking the host down. Third, --read-only plus a 256 MB tmpfs means a bad generation cannot persist anything.

Here is the uncomfortable lesson from real-world use. Our first version mounted the project directory read-write for convenience and inherited the shell environment. A malformed cleanup script destroyed a working branch. Nothing malicious, no clever attack, just an agent executing exactly what a badly worded prompt requested. Mount read-only, then copy out what is wanted.

What Nobody Mentions About Daily Use

Three findings emerged after a month of daily use, and none appear in standard tutorials. We measured each on the same 8-core server described above in 2026.

First, swap gets touched even with RAM to spare. Our server shows 2.2 GiB of swap in use while 19 GiB of RAM sits free, because the kernel pages out idle model weights between requests. Harmless, though the first cold response after a quiet hour is noticeably slower. Setting vm.swappiness=10 reduces the effect.

Second, model load time dominates short tasks. Ollama unloads a model after 5 minutes idle by default, so a one-line question can spend more wall-clock time loading weights than generating tokens. Raising OLLAMA_KEEP_ALIVE fixes bursty query patterns.

Third, the agent performs best on work engineers find boring. Test scaffolding, docstrings, migration scripts, changelog entries. Ask it to design an architecture and the output is confident, generic, and subtly wrong.

Frequently Asked Questions

How much RAM does an AI coding agent need?

An AI coding agent needs the model's on-disk size plus roughly 2 GB for context and runtime overhead, added to whatever other services consume. For example, a 4.7 GB model such as qwen2.5-coder:7b runs comfortably on 8 GB of RAM on a dedicated box. We run that same 7B model on a 23 GiB server alongside a web server, a database, and a dozen background jobs, with 19 GiB still free at idle.

Is a GPU required for a local coding agent?

No. A GPU is not required for a local coding agent. CPU inference is slower per token but adequate for interactive coding work, and GPU instances typically cost $200 to $500 per month against $20 to $60 for comparable CPU capacity. A GPU becomes justified when serving multiple concurrent users or running models above 13B parameters.

Can a self-hosted agent replace Claude or GPT-5?

A self-hosted agent does not replace Claude or GPT-5 for hard reasoning tasks such as architectural design or subtle debugging. It does replace them for routine high-volume work where privacy and flat monthly cost outweigh raw model capability. In practice the two coexist: local models handle the boring 80%, and a frontier API handles the difficult remainder.

Conclusion

A self-hosted AI coding agent is worth building when two conditions hold: code cannot leave the infrastructure, and the workload is mostly routine. Both were true here. The configuration above runs on an 8-core, 23 GiB Debian 13 VPS, handles a few dozen tasks per day, and costs a flat $20 to $60 per month regardless of request volume.

Start small. Pull the 3B model, wire retrieval over one repository, and assign a genuinely boring job for a week. If the agent earns its keep, scale to a 7B and add the Docker sandbox. If not, the cost was one evening rather than an annual subscription.

The failure mode worth avoiding is the one we hit first: building an impressive agent with excessive permissions before proving it does anything useful.