How do I know when my AI agent run finishes or silently stalls?

An AI agent run that stalls does not raise an error — it simply stops producing output, and nothing tells you. The fix is two signals rather than one: have the run send a message when it finishes, and put a heartbeat monitor underneath it so that silence itself becomes the alarm. A completion ping proves the run ended; a heartbeat proves the run is still alive between check-ins. With both in place, an agent that dies at 3am wakes your phone instead of waiting for you to notice at breakfast.

Why do long agent runs fail so quietly?

A web request that fails gives you a status code. A build that fails gives you a red X. An autonomous run gives you neither, because most of the ways it ends badly do not look like errors from the inside:

• It stalls. A tool call hangs on a socket with no timeout, and the process sits there consuming nothing and reporting nothing.
• It loops. The agent retries the same failing step forever. Technically it is working. Practically it is finished.
• It exits clean on a dead credential. An expired key returns a 401, the code catches it, logs a line, and returns exit status 0.
• The host went away. An out-of-memory kill, a spot instance reclaim, or a container restart leaves no trace inside the run at all.
• It finished, and you missed it. Output went to a log file, a terminal that closed, or a queue nobody watches.

Every one of those produces the same observable thing: nothing. That is the whole problem. You cannot alert on an error that never gets raised, so you have to alert on the absence of an expected event instead.

The two signals you actually need

Signal one: the completion ping

At the end of the run — success or failure — the agent makes one HTTP call that says what happened. This is the signal you want most of the time, because most agent runs are things you started deliberately and want the answer to. It is a single request, it carries a short human-readable summary, and it lands on your phone as a push notification.

Signal two: the heartbeat

A completion ping only fires if the run reaches the end. A run that hangs never reaches the end, so the ping never fires — and a missing notification looks exactly like a run that is still going. A heartbeat monitor closes that hole. You tell Pingwire how often the run should check in, the run calls a unique URL each cycle, and if a check-in does not arrive on time Pingwire opens an incident and notifies you. Silence becomes a positive signal.

Use both. They fail in opposite directions, which is exactly why the pair is worth more than either one alone.

How do I send a notification when an agent run finishes?

One POST with a Bearer key. Wrap the run so both the success path and the failure path report, and put the report in a finally block so an exception still tells you something:

import os, requests, traceback

PING = "https://pingwire.dev/api/v1/messages.php"
KEY  = os.environ["PINGWIRE_KEY"]

def notify(title, text, priority="normal"):
    requests.post(
        PING,
        headers={"Authorization": f"Bearer {KEY}"},
        json={"channel": "agents", "title": title, "text": text,
              "priority": priority},
        timeout=10,
    )

status, summary = "failed", "unknown"
try:
    result = run_agent()                 # your long-running job
    status  = "done"
    summary = f"{result.steps} steps, {result.tokens} tokens"
except Exception as e:
    summary = f"{type(e).__name__}: {e}"[:200]
    raise
finally:
    notify(
        "Agent " + status,
        f"nightly-research: {status} — {summary}",
        "high" if status == "failed" else "normal",
    )

Two details worth copying. The timeout=10 stops your notification call from becoming the thing that hangs. And truncating the exception text keeps the push readable on a lock screen — a 4,000-character stack trace collapses into unreadable mush on a phone, and the useful part is almost always the first line.

If your agent is a shell pipeline rather than Python, the same idea is one line; the mechanics are covered in sending a push notification from a bash script.

How do I catch a run that stalls halfway through?

Create a heartbeat monitor, set the expected interval to something a little longer than one normal cycle, then have the run check in each time it completes a cycle. A plain GET is enough — the token in the URL is the credential:

# at the end of every agent loop iteration
curl -fsS --retry 3 https://pingwire.dev/hb/YOUR-TOKEN > /dev/null

# and when the run aborts, report the failure immediately
curl -fsS https://pingwire.dev/hb/YOUR-TOKEN/fail

In Python, the same two lines sit at the bottom of the loop body:

HB = "https://pingwire.dev/hb/YOUR-TOKEN"

for task in queue:
    handle(task)
    requests.get(HB, timeout=5)        # still alive, still working

Now pick the interval honestly. If a cycle usually takes four minutes and occasionally takes nine, an expected interval of five minutes will page you for a slow-but-healthy run, and after the third false alarm you will start ignoring the notification — which is worse than having no monitor at all. Set the interval to the slow case and add a grace window on top. A monitor you trust is worth more than a monitor that is technically tighter.

The check-in cadence you can use depends on your plan: Free covers 3 monitors at a 5-minute interval with 7 days of history, Pro covers 25 monitors at a 1-minute interval with 90 days of history. Pingwire is in free mode right now, so everyone gets the Pro limits at no cost.

What about runs with no natural loop?

Some agents do one long thing rather than many short things. Emit a heartbeat on a timer instead of per iteration — a background thread that pings every couple of minutes works, and it has a useful property: if the main thread deadlocks but the process survives, the heartbeat keeps arriving and tells you the monitor is not the thing to trust here. For that case, prefer a heartbeat tied to observable progress (a counter that must increase) rather than to the clock.

Can the agent set this up itself?

Yes. Pingwire ships an MCP connector, so an assistant that holds your Pingwire connection can send messages, schedule reminders, create monitors, and read incident status as part of its own workflow — no glue code in between. MCP is an open standard for connecting assistants to tools; the specification lives at modelcontextprotocol.io.

That flips the usual arrangement. Instead of you instrumenting the agent from the outside, the agent registers its own watchdog before starting the work, and tears it down after. The broader pattern — an agent reaching a human on purpose — is covered in how an AI agent can send you a push notification.

What should the notification actually say?

A notification you cannot act on from the lock screen is a notification you will open later, which means it is not really an alert. Three rules hold up well:

• Name the run first. nightly-research: failed beats Agent run failed, because you probably have more than one agent and the name is what decides whether you get up.
• Put the verdict in the first few words. Notification titles are truncated hard on phones — assume only the beginning survives.
• One line of cause, not the whole trace. ReadTimeout: api.example.com tells you whether this is yours to fix. The trace can wait for the terminal.

Reserve high or urgent priority for runs where a delay actually costs something. If every agent alert is urgent, none of them are.

When the run matters enough to escalate

For an agent whose failure genuinely hurts — a nightly job that feeds a customer-facing dataset, say — attach an escalation policy to the monitor so an unacknowledged incident keeps trying rather than expiring into silence, and acknowledge incidents deliberately so the record shows a human took it. If the agent's output is something other people depend on, a public status page saves you from answering the same question five times.

A checklist to copy

1. Send a completion ping from a finally block, so exceptions still report.
2. Give every notification call a short timeout — never let the alarm hang on the thing it is watching.
3. Add a heartbeat monitor with the interval set to the slow case, plus grace.
4. Call the /fail URL on abort so you do not wait for a missed check-in.
5. Prefer a progress-based heartbeat over a clock-based one where you can.
6. Lead the message with the run name and the verdict; truncate the cause to one line.
7. Escalate only the runs where being late actually costs you something.

None of this makes an agent more reliable. It makes an unreliable agent visible, which is the part you can act on. The endpoints, payload fields, and rate limits are all on the Pingwire developer docs page.