After-Hours / Weekend TAT Power Outage Response

NOC Standard Operating Procedure (SOP)

Title: After-Hours / Weekend TAT Power Outage Response

Owner: NOC Department

Maintained by: NOC Team

Approved by: IT & NOC Manager

Version: 1.0

Effective Date: [12/16/2025]

Purpose

To establish a standardized response procedure for after-hours and weekend power outages at the Talkeetna Alaska Teleport (TAT), ensuring proper monitoring, communication, documentation, and escalation until commercial power is restored.

Scope

This SOP applies to all NOC personnel responsible for monitoring, responding to, and documenting power-related incidents at the Talkeetna Alaska Teleport outside of normal business hours.

Responsibilities

NOC Technicians

• Monitor alerts and Freshdesk tickets related to power events

• Verify generator operation, fuel levels, and alarms

• Perform required notifications and escalations

• Maintain accurate and timely ticket updates

IT & NOC Management

• Provide escalation support as needed

• Review incident documentation and post-incident reporting

Procedure

1. Alert Intake and Ticket Creation

• Receive power outage alerts via email and/or Freshdesk.

• All power outage, ATS, and generator startup alerts must be sent to:

  noc-alerts@microcom.tv

• Freshdesk tickets will be automatically generated for the following events:

  • Generator 1 – Genset Started

  • Generator 2 – Genset Started

  • ATS 1 – Source 1 Disconnected

  • ATS 1 – Source 1 Not Available

  • ATS 1 – Source 2 Connected

  • ATS 1 – Source 2 Available

2. Generator Verification

• Log in to Power Command. (https://portal.powercommandcloud.com/dashboard)

• Verify that both generators have successfully started.

• Update the corresponding Freshdesk tickets confirming generator startup.

• Verify and document fuel levels for each generator (include fuel percentage in ticket updates).

3. Visual Confirmation via Cameras

• Log in to the camera system.

• Visually confirm that generators are running.

• Update tickets confirming visual verification.

4. Alarm Monitoring

• Continuously monitor Power Command for any generator or system alarms.

• If alarms occur, document details and update tickets immediately.

5. External Notifications

• Contact Matanuska Electric Association (MEA) to report the power outage: (https://mea.outagemap.coop/)

  Location: Talkeetna Alaska Teleport

  Address: 21518 S. Comsat Rd., Talkeetna, AK

  Phone: 907-746-7697 (746-POWR)

6. Internal Escalation

• Contact TAT on-call staff.

• If no response, escalate in the following order:

  1. Site Manager – Mike Willmon (907-903-4910)

  2. Alexander Schumann (907-229-9993)

  3. Chuck Schumann (907-229-8338)

7. Ongoing Monitoring and Updates

• Update all related Freshdesk tickets every 30 minutes during the outage.

• Approximately 20 minutes after startup, one generator will shut down.

• Confirm and document the following expected event:

  • Generator 2 – Genset Stopped

• The first scheduled update must include confirmation that one generator has stopped.

• Continue monitoring generators, alarms, and cameras.

• Repeat verification steps for Power Command and camera systems until power is restored.

8. Power Restoration Events

   Upon commercial power restoration, the following tickets will generate automatically:

• ATS 1 – Source 1 Available

• ATS 1 – Source 1 Connected

• ATS 1 – Source 2 Disconnected

• ATS 1 – Source 2 Not Available

• Generator 1 – Genset Stopped

9. Final Verification and Closure

• Log into the camera system to verify that all generators have shut down.

• Update tickets confirming generators are no longer running.

• Ensure all related tickets contain a complete timeline of actions taken.

• Close tickets only after full verification is complete.

Documentation & Logging

• All actions must be documented in the corresponding Freshdesk tickets.

• Updates must include timestamps, verification steps, and any communications made.

• Tickets must reflect a clear audit trail from outage start through resolution.

Notes

• Do not assume automatic systems are functioning correctly; always verify manually.

• Maintain clear, concise ticket updates suitable for management review.

• If expected events do not occur, escalate immediately.