Skip to content
← All articlesCloud & DevOps

Incident Management for Live Apps: A Practical Guide

How to run incident management for live applications: severity levels, on-call, communication, post-incident reviews and what to ask an offshore support team.

Every live application fails eventually. A deployment goes wrong, a third-party API times out, a database fills its disk. What separates a minor blip from a front-page outage is rarely the failure itself. It is how quickly the team notices, who takes charge, and how clearly everyone communicates.

This guide sets out a lightweight incident management process that a startup or mid-sized product team can adopt, whether the engineers sit in your office or with an offshore support partner.

What counts as an incident?

An incident is any unplanned event that degrades or interrupts the service your users depend on. A slow checkout, failed logins and a broken nightly import all qualify. A cosmetic bug does not; it belongs in the normal backlog.

Writing this definition down matters. Without it, teams either escalate everything and burn out, or wait too long on problems that are quietly costing revenue.

Step 1: Define severity levels

Severity levels give everyone a shared language. Keep them to three or four, and describe them by user impact rather than technical cause.

Severity Typical meaning Expected response
Sev 1 Core service down or data at risk for many users Immediate, around the clock, all hands
Sev 2 Major feature broken or serious degradation, workaround exists Prompt response in agreed hours
Sev 3 Minor feature affected, small group of users Next working day
Sev 4 Cosmetic or low-impact issue Planned into the backlog

Agree the exact response times in your support contract. Our guide to application support SLAs covers how to express them.

Step 2: Detect problems before users do

If customers are your monitoring system, you are already late. At a minimum, set up:

  • Uptime checks on key pages and API endpoints, run from outside your own network.
  • Alerts on symptoms, such as error rate, latency and failed payments, rather than only on server metrics like CPU.
  • A single alert channel that reaches the on-call engineer, not a shared inbox nobody watches.

Good alerting depends on good telemetry. See our overview of logs, metrics and traces for what to collect.

Step 3: Set up on-call that people can sustain

On-call does not have to mean one hero answering every page.

  • Keep a primary and a secondary responder so a missed alert is never a dead end.
  • Rotate weekly, and give people time off after a rough shift.
  • If you work with an offshore team, a “follow the sun” arrangement can cover more hours without waking anyone at 3 a.m., provided handovers are written and explicit.
  • Keep a short runbook for each alert: what it means, how to check it, and the first safe action.

Step 4: Assign roles during an incident

Confusion is the biggest time-waster during an outage. Even a small team should name three roles:

  1. Incident lead. Makes decisions, keeps the team focused, and decides when to escalate.
  2. Responders. Investigate and fix. They do not stop to answer questions from stakeholders.
  3. Communicator. Updates the status page and stakeholders so responders can concentrate.

In a two-person team one person may hold two roles, but the lead and the person typing commands should ideally be different people.

Step 5: Stabilise first, diagnose later

The first goal is to restore service, not to find the root cause. Common fast mitigations include:

  • Rolling back the latest release (easier if you use zero-downtime deployment strategies and feature flags).
  • Switching off a failing feature.
  • Scaling up or restarting an overloaded component.
  • Failing over to a standby.

Record what you tried and when, in a shared channel or document. This timeline becomes the backbone of your review.

Step 6: Communicate clearly

Silence frightens customers more than bad news. A good update has four parts: what is affected, what we know, what we are doing, and when the next update will come. Post that next update on time even if the message is “no change yet”.

Keep internal and external messages separate. Customers need impact and expected resolution, not a log of every command.

Step 7: Hold a blameless post-incident review

Within a few working days of a Sev 1 or Sev 2, review what happened. The aim is learning, not blame. Ask:

  • What was the impact and how long did it last?
  • How was it detected, and could it have been detected sooner?
  • What made the response slower or faster?
  • What changes will prevent a repeat or reduce the damage?

Each action needs an owner and a due date. A review with no tracked actions is a story, not an improvement.

Incident readiness checklist

  • Severity levels documented and agreed
  • External uptime checks and symptom-based alerts in place
  • On-call rota with a named secondary
  • Runbooks for the top alerts
  • Status page or customer communication channel ready
  • Backups restored successfully at least once in a test
  • Post-incident review template and action tracking

What to ask an outsourced support partner

If you are handing over production support, ask how they handle each step above. Specifically: Who is on call outside local business hours? How are incidents escalated to you? Will you receive written incident reports? How is access to production controlled and audited? A mature partner will answer these concretely.

Next steps

AkMo Technologies provides application support and maintenance for international clients, including monitoring, incident response and structured reporting. If you want to formalise your incident process or hand over production support, contact us for a conversation. For planning your recovery targets, see our guide to disaster recovery planning for cloud apps.

Let’s build what’s next.

Tell us about your platform, your product idea or the systems you want to modernise. Someone from our Chennai engineering team will get back to you personally.