Respond to an App incident

Use this procedure when an App is unavailable, severely degraded, returning unexpected errors, or showing signs of data or security impact.

The goal is to establish scope, preserve useful evidence, apply only a safe containment action, and give support enough context to act quickly. Don’t make several unrelated changes at once. Each change makes it harder to identify the cause and verify the recovery.

Establish the impact

Record the following before changing the App:

• the affected App and Team;
• the public URL, admin URL, or customer journey that is failing;
• when the problem started, including the time zone;
• whether every visitor is affected or only a region, account, browser, or operation;
• the expected result and the result you observed;
• the last known time the same path worked;
• whether data loss, unauthorized access, or exposed credentials are suspected.

Test one representative path from a second network or device when possible. This helps distinguish a local cache or network problem from an App-wide failure. Don’t repeatedly submit checkout, import, migration, or other state-changing operations as a test.

Check the App state

Open the App and review these surfaces in order.

1. Overview: confirm the App still exists, note its status, copy the App ID, and open the live URL. A Provisioned status means setup completed; it doesn’t prove every customer journey is healthy.
2. Deploys: look for a deploy or configuration change near the incident start time. Open the latest operation and record its deploy ID, source revision, current phase, and failure message.
3. Diagnostics: look for an automatically created investigation and recent traces covering the failing path. Match the method and request URL, then open that row. Check the cache result, review the ranked attributable hotspots, and use the connected waterfall to confirm parentage and inclusive waiting time. Record the investigation ID, trace ID, largest material hotspot, severity, findings, evidence window, and suggested actions. An older report or trace describes the state at that time and doesn’t update after a later change. A truncated trace is incomplete evidence.
4. Usage: check whether CPU, memory, or burst status changed near the incident. Treat Utilization unavailable as unknown, not as zero usage.

Keep the identifiers and timestamps in one incident note. Screenshots are useful for transient messages, but copy identifiers and error text as plain text so support can search them.

Follow the symptom-specific path

Use the narrowest guide that matches what failed:

• A recent release failed or the problem began after a release: recover from a failed deploy or review rollback safety.
• Only a custom hostname fails: review domain status, DNS, and certificates.
• The App is healthy but near its purchased capacity: review usage and manage resources.
• An automatic investigation exists: review the investigation report.
• SSH access, a database restore, or a media restore is involved: use the matching recovery guide before starting another operation.

Don’t assume a code rollback repairs data, DNS, certificates, secrets, or resource exhaustion. Don’t start a restore merely because the App is down. A restore changes data state and should be used only when the incident is actually a data-recovery problem.

Contain customer impact

Choose one containment action only when its effect is understood:

• Turn on maintenance mode when the domain still reaches the App and a controlled maintenance response is safer than the current public behavior. Maintenance mode can’t repair broken DNS or certificate routing.
• Roll back only when a known-good release exists and the deploy’s data-impact assessment permits the intended rollback.
• Revoke or rotate a credential that is known to be exposed. Record which credential changed and when, but never put its value in an incident note or support ticket. Use Vars and secret references or SSH access management for the affected credential path.
• Stop a repeated customer or automation action that is making the incident worse, such as continually resubmitting a failed deploy or import.

If the correct containment is unclear, preserve the current evidence and contact support rather than guessing.

Open a critical support request

For an active outage, suspected data loss, or security incident, open a Critical support ticket:

1. From Overview, select Open a ticket so the App is selected automatically. You can also open the Team’s Support page and select the affected App.
2. Use a concise subject that states the impact, such as Checkout unavailable on Storefront.
3. Include the incident start time, affected paths, scope, last known good time, recent change, App ID, deploy ID, investigation ID, and any safe containment already applied.
4. State whether you suspect data loss, unauthorized access, or an exposed credential.
5. Submit the ticket, then add any redacted files needed to explain the problem on your first reply or any later reply. Each reply can include up to 20 files, and each file must be 25 MiB or smaller. For email replies, keep the entire encoded message under 25 MiB; larger messages still add the reply, but their attachments aren’t imported.

You can email support@synsmarts.ai from the email address on your synsmarts account at any time. Include the Team and affected App names or IDs when starting by email because the portal normally records that context directly. A new email starts at Normal priority without a structured App association until support triages the details in your message. When replying to a ticket notification by email, preserve its subject so the reply remains on the same ticket.

Never include passwords, private keys, API tokens, secret values, payment-card data, or unredacted customer records. Redact credentials and customer data from screenshots, logs, configuration, and command output before attaching them. See Get support for the complete ticket procedure.

Keep the incident record current

Reply on the same ticket when:

• the impact or affected scope changes;
• you apply or reverse a containment action;
• a deploy, restore, or investigation reaches a new state;
• the App recovers or fails again;
• you discover a reliable reproduction path.

Don’t open duplicate tickets for the same active incident. A single timeline with current App associations is more useful than several partial reports.

Verify recovery

Before declaring the incident resolved:

1. Confirm the portal shows a terminal successful state for any deploy, configuration change, or restore used in recovery.
2. Test the original failing path over HTTPS.
3. Test one adjacent critical path, such as login after a checkout repair or admin access after a domain repair.
4. Confirm maintenance mode is off when normal traffic should resume.
5. Watch the App long enough to catch an immediate recurrence.
6. Reply to the support ticket with the result and the change that restored service.

Close the ticket only after your Team has verified the customer-visible result. Reopen it if the same incident returns.