Triage moments
Triage is the process of examining moments Hotpot has received in the current shift and assigning those moments to groups. You can do this as often as you like, capturing the sentiment and experience immediately, at the end of the day, or at the end of the week!
Understand moments
Moments are messages that may require attention. Hotpot is concerned with the various signals that represent work and disturbances in your organization. We're talking about alerts, pages, and incidents as well as day-to-day stuff like near-misses.
Hotpot moments and their types are not connected to any kind of notification or workflow. They are after-the-fact creations meant to capture something that happened. Creating a moment of type page, for example, never pages someone. It's merely bookkeeping!
Where do moments come from?
Moments come from integrations as well as from you and your organization through tools like Slack. You can use to send us all your useful moment data through the Hotpot API.
As an example, you might use Hotpot with Datadog and Sentry. Each of those tools can page you through Hotpot, but they can also send all their warnings and other signals to Hotpot for you to keep track of. With Hotpot receiving all these moments you can use them in triage!
Moment types
There are many types of moments, each representing important operational signals.
Alerts
Alerts represent generated alerts, usually from some sort of monitoring or observability tool. These alerts need not be critical but can be of any urgency. Hotpot wants them all! The goal is to collect all the signals these tools emit and use them to help improve awareness and track how often they are noisy or actionable.
Incident
When your organization declares an incident, it's often a big deal. Sending these events to Hotpot helps connect the context of an incident with all the other signals you generate. After an incident you may find interesting connections with alerts that did (or did not!) fire in the lead-up to or during an incident.
Near-miss
Sometimes, you get lucky. You noticed you ran a command and left off the --qa flag or some other footgun. In these cases, it can be helpful to record this situation as evidence to help with future investments. Hotpot isn't just about reactive stuff but proactive as well. These can be an interesting part of handoff discussion.
Page
Paging someone is expensive. You're interrupting them and bringing their attention to a problem that needs solving. Tracking how often people are paged and the impact that has is important both individually and in the context of your organization's larger operational picture.
Note that a moment type of page does not page someone. It records a page for feedback and disruption tracking! Hotpot has several mechanisms to page someone, which in turn create moments of type page.
Moment priority
Moments may also have a priority of Low, Medium, or High. This priority setting is purely informational.
Moment groups
A moment group is a collection of moments with a noise or novel designation and, generally, a common theme. You create groups during triage, refine them during handoff preparation, and discuss them during a handoff.
You can also use tags to annotate moment groups with any additional context that could be useful later.
Noise or novel
The first step in creating a group is picking one or more events and telling Hotpot if this group is noise or novel:
- Noisy moments are those that pester you and don't create actionable outcomes. These are often noisy alerts or false alarms. Capturing this information can help you and your team learn what needs tuning or more investment.
- Novel alerts are those that spur action and generate insight for the team. This might be a novel alert delivered at the right time, an incident that impacted customers, or a near-miss that could pay off with some investment.
Groups as themes
Groups are something you might want to talk about in Handoff. When you create a moment group, you're flagging that combination of moments for later discussion or at least for bookkeeping!
Moments can be in more than one group, and each group is meant to capture a theme. Hotpot tries to help with this by making some basic assumptions, like a group with an incident is probably about that incident.
Noisy groups with alerts are likely about, well, a noisy alert! Over time Hotpot's capabilities in groups will expand but in terms of data we can infer as well as actions we can suggest based on the data therein.