Snapshot Sentinel
Compares a before snapshot of the fabric with an after snapshot and tells you what is missing, what is new and what changed. Two ways to take the snapshots - pick the one that fits:
| Automated snapshot | Manual snapshot | |
|---|---|---|
| How the tables are captured | Snapshot pulls them from Orchestrator for you - double-click, two buttons | You export each table from Orchestrator by hand, as the Before and After checklists describe |
| Needs | An API key (Read-Only) and a PC that can reach Orchestrator | Nothing - one HTML file, works with no network |
| Sees | Everything the hand export has, plus the primary / backup route tag and flow counts per appliance | What the Orchestrator export contains |
| Effort per snapshot | One click, about a minute | A dozen exports and saves |
| Compare with | Snapshot Sentinel (included) | Snapshot Sentinel |
Both produce the same files with the same names, so a hand export and an automated capture can be compared against each other.
Download before the maintenance window, into the folder you will keep the snapshots in. Snapshot Sentinel works with no network connection. The automated snapshot needs to reach Orchestrator - for a cloud Orchestrator that means internet access from the PC running it.
Automated snapshot - double-click, two buttons
Download Automated snapshot (zip) above and unzip it into the folder you will keep the snapshots in. Then:
| Step | Do |
|---|---|
| 1 | Double-click Snapshot.cmd. A black window opens and stays open; your browser opens the Snapshot page |
| 2 | First time only: type the Orchestrator address and paste an API key from Orchestrator > Users & Authentication > API Keys (Read-Only is enough, give it an expiry, it is shown once). Both are remembered on that PC. On a large fabric, pick Hubs only or Hubs plus these for the live reads - all appliances on a 5,000-appliance fabric takes a while |
| 3 | Click Take BEFORE snapshot. The page lists each table as it lands and ends with a green line |
| 4 | Do the maintenance work |
| 5 | Click Take AFTER snapshot, then Compare BEFORE and AFTER. Every table is judged on one page - MATCH rows can be ignored, flagged rows have a Show button |
Close the black window when you are finished. Nothing typed, no commands - the page runs only on that PC, and only the calls to your Orchestrator leave it. For a cloud Orchestrator the PC needs internet access; for on-prem, reach to the Orchestrator and the self-signed-certificate box ticked.
Make a fresh API key for each maintenance window and delete it in Orchestrator afterwards. The key is saved on that PC in snapshot-config.json, next to Snapshot.cmd, in plain text - delete that file when the window is done. A Read-Only key with an expiry is the safeguard while it exists.
Each snapshot writes every table the checklist asks for, with the checklist's filenames, plus snapshot-manifest.txt saying which appliances Orchestrator could reach and which tables it could not fetch. An appliance that was unreachable at capture is written as a row saying so, not left out - otherwise the Sentinel would report all of its routes as missing.
What the snapshot sees that the hand export does not: the routes table carries Peer status (P/B), Admin distance and Learned from, so a route that flipped from primary to backup during the window shows as CHANGED. That is the blind spot GMS-46509 leaves in the CSV export, closed. The BGP peers table carries prefixes received and sent per peer, shown before → after - a peer that is still Established but receives a different number of routes is a route map or redistribution change, and no Peer state column can see it. It also records flow counts per appliance, which is the flows number that means something after a reboot.
Three tables have no hand-export equivalent at all: Interfaces (link, negotiated speed and duplex per port in use - the port that came back at 100 Mb half duplex), IP SLA (each monitor up or down, and how many times it flipped during the window), and System (uptime, so you can see which appliances actually rebooted and what release they came back on; on an EdgeHA pair, which is Primary and which is Backup).
Works on the PowerShell every Windows PC already has, and on PowerShell 7 on macOS (run pwsh snapshot.ps1 there). Keep snapshot-config.json with the snapshots, not on a shared drive - it holds the API key. Verified against Orchestrator 9.7.1; the API differs between releases, so on another release read the manifest for any table it could not fetch.
Manual snapshot, and comparing
Export the tables as the Before and After checklists describe, into a before and an after folder. Then compare:
| Step | Do |
|---|---|
| 1 | Open snapshot-sentinel.html in any browser (double-click it) |
| 2 | Drop the before files on the left and the after files on the right - the whole folder at once is fine. Each table is paired with its twin and judged; one summary row per table, Show for the detail. Any CSV straight from Orchestrator, or any list one item per line |
| 3 | Read the result. MATCH means nothing missing, nothing new, nothing changed. DIFFERENCES lists the counts, and the MISSING rows are the ones to read first |
| 4 | Download results (CSV) for the change record, or Copy summary for the ticket to paste into the case |
What it understands
| Export | What counts as the same row | What counts as changed | Ignored |
|---|---|---|---|
| Routes Report | Appliance, segment, subnet / mask, next hop, label, interface, type (the peer) | Zone, state, metric, additional info, and from a Snapshot capture peer status (P/B), admin distance, learned from | Comment |
| Underlay Tunnels Report | Appliance, segment, tunnel name | Overlays, admin status, status, MTU, local and remote IP:port, mode | Discovered IP:port, uptime |
| Overlay Tunnels Report | Appliance, tunnel name, overlay | Admin status, status, MTU, underlay tunnels | Uptime |
| Passthrough Tunnels Report | Appliance, segment, tunnel name | Admin status, status, IPs, mode, NAT, peer/service, max BW | Uptime |
| BGP Peers Report | Appliance, segment, peer IP, address family, route target, local interface, peer ASN | Peer state, inbound and outbound route map | Established time, peer details, prefix counters |
| BGP Summary Report | Appliance, segment | Every configuration column | BGP state details |
| OSPF Summary Report | Appliance, segment | Every configuration column | - |
| BFD Sessions Report | Appliance, segment, local and remote address, local interface | State | Up time |
| Alarms Report | Alarm ID, host name, severity, source, description | Ack, acked by | Alarm time, recommended action, acked time, comments |
| Flows Report | Not row by row - see below | Flow counts per appliance, per outbound tunnel and per application, with average delay | Everything else |
| Flows Summary Report | Appliance | Flow totals per appliance, from the appliance itself (Snapshot writes this). CHANGED only when a count moves by more than 25% | - |
| Reachability Report | Appliance | State, model, software version (Snapshot writes this) | - |
| OSPF Neighbors Report | Appliance, segment, neighbor ID and IP, interface | State, priority | Dead time |
| Peer Priority Report | Appliance and peer | Peer priority, advertise metric | - |
| Any other export with a header row | Every column, except ones that look like times, counters or rates | - | - |
| Plain text, one item per line | The whole line, exactly as typed | - | - |
Four things it does on its own so you do not have to:
- Tunnel status: Up - Active and Up - Idle are both Up. A tunnel goes idle after five minutes with no traffic, so that flip is not a change. Down is.
- "Nothing to report" rows are set aside. Orchestrator writes one row per appliance or segment with no entries - No tunnels configured., No BFD sessions available., No data available, Peer priority not configured on the appliance. They are counted, not compared, so an appliance that reports nothing is told apart from one that is missing from the export.
- Alarms read as RAISED and CLEARED, raised first - those are the ones you want. An alarm keeps its ID while it is active, so a raised alarm is genuinely new.
- Flows are judged by volume, not row by row. After a reboot every session resets, so every flow row is different and a row-by-row compare would say everything changed. Instead the Sentinel counts flows per appliance, per outbound tunnel and per application, and flags any that halved or doubled - that is where traffic moved. Where flows report a delay, the average is shown too, and a site whose average delay doubled is flagged as a slower path.
- A hand export and a Snapshot capture compare against each other. The capture has extra columns; the Sentinel compares on the columns both sides have and says which it left out.
The Advanced section under the results lets you change which columns identify a row and which are compared, if the defaults are not what you need.
Read the routes result with these in mind
The hand routes export does not carry the primary / backup tag or the admin distance the Routes page shows (GMS-46509). In a hand export a route that flipped from primary to backup looks identical and will not show here - spot-check critical prefixes in the Routes tab. A Snapshot capture carries both, and the flip shows as CHANGED.
- The same prefix legitimately appears once per peer it is learned from. A row MISSING means that copy is gone - check whether the prefix is still there from another peer before treating it as an outage.
- Export with the same scope both times. If before was taken with "All" appliances and after with a filter on, every appliance the filter left out is reported as missing. The per-appliance table says ABSENT from the AFTER export when that has happened.
- A count that moved by more than 5 on one appliance is flagged for investigation, matching the checklist.
Updated about 2 months ago