Skip to content
All posts

Engineering · AI

How Our Ops Agents Keep Your Services Running 24/7

Four agents, each doing one thing well, instead of one giant control plane. Here's how we keep your services alive without waking anyone up.

Most hosting platforms run one giant control plane with a rotation of humans on call. When something breaks at 3am, a pager goes off, someone groans, opens a laptop, and tries to remember last quarter's runbook.

We didn't want to do that to ourselves, so we built it differently. Four small automated agents. Each does one job. They talk to each other.

The four agents

The Provisioner is what you talk to when you click deploy. It reads your request, picks a server with capacity, configures the container, sets up SSL via Let's Encrypt, and wires up your subdomain. It calls our deployment platform, which in turn talks to Docker on the server. A typical deploy takes 1-3 minutes. The slow ones are usually waiting on a DNS propagation we can't control.

The Monitor wakes up every minute and checks every container. Process crashed? App stopped responding? It notices, then hands the problem off.

The Healer is the one that tries to fix things. When Monitor reports a service down, Healer restarts it, backing off between attempts. If a few restarts in an hour don't bring it back, Healer stops retrying and escalates. At that point our AI co-pilot looks at the failure, explains the likely cause in plain language, and suggests a fix.

The Alerter is the one we spent the most time on, honestly. The default in this industry is to spam you. We didn't want that. So Alerter is allowed to send you one notification per incident. Not five. One. It includes what broke, what we did about it, and whether you need to act. Repeat alerts for the same problem are held back for 15 minutes.

Why not one big system?

Because small things are easier to fix when they break. Provisioner has a bug? Monitor doesn't care. Healer makes a bad call? We adjust its policy without touching anything else. Same reason microservices work for backend code: blast radius.

There's a tradeoff, of course. Coordinating four agents is more complex than one process. But the debugging story is so much nicer that we'll happily pay that cost.

Logs and metrics, no extra tools

Your dashboard shows live CPU, memory, network, and disk I/O for every container. We mount the Docker socket read-only into our API and read stats straight from the Docker Engine HTTP API. There's no Prometheus to configure. No Grafana to learn. Just the numbers.

What we're working on

Anomaly detection is on our list. If your traffic suddenly does a 10x, we want to warn you early.

Hostendor

Want to try this on your own project? The 14-day trial comes with ₹300 of credit, and you don't need a card to start.

More from the blog