Monitoring Orchestrator and EdgeConnect appliances

Integration of HPE Aruba Networking EdgeConnect SD-WAN into various IT & OSS/BSS systems can span a number of different functional areas, depending on what type of integration, automation, or aggregation is desired. Below is a table that describes various integration use cases and the recommended integration points, based on what the API was designed for.

Integration Use-Case	Integration Options
Realtime Alarm Monitoring - Asynchronous streaming of alarm & clear notifications	Appliance: SNMP, syslog Orchestrator (recommended): Orchestrator Notification Service for aggregated alarms from all appliances and Orchestrator (syslog, HTTP/HTTPS, or Kafka)
Audit Log Monitoring - Asynchronous streaming of audit log event notifications	Appliance: syslog
Flow-level Performance Data Streaming - Real-time streaming of flow records	Appliance: IPFIX, Netflow
Firewall Logging - Logging of FW deny actions	Appliance-;eve;” syslog, IPFIX
Topology & Resource Discovery - Retrieval of SD-WAN topology resources	Orchestrator: REST APIs for appliances, tunnels & interface resources
EC Pre-configuration - Definition of appliance config before the appliance is deployed	Orchestrator: YAML pre-configuration
EC Post-deployment config - Adding incremental configuration to existing appliance	Orchestrator: YAML config & apply (recommended), REST API for specific configuration modules
EC-V Bootstrapping - Initialization of EC-V w/ account name/key and MAC	Appliance: CloudInit spcustom.yml file
Aggregated Statistics Retrieval	Orchestrator and Appliance: REST API polling for aggregate statistics
Realtime Statistics Retrieval	Orchestrator and Appliance: REST API polling for real-time statistics
User Management & RBAC	Orchestrator and Appliance: REST API
Orchestrator Configuration	Orchestrator: REST API for any Orchestrator meta-data configuration

Best practice: Retrieve minute statistics from EdgeConnect appliances

Download the "Retrieving SD-WAN Statistics" Whitepaper (Here)

The HPE Aruba Networking EdgeConnect SD-WAN solution generates volumes of statistics every minute. These statistics help us debug SD-WAN networks. Other network devices generate orders of magnitude less data per device. This disparity is partly due to the meshing capabilities of EdgeConnect and the collection of data on every tunnel. The other factor is the frequency at which data is collected. HPE Aruba Networking EdgeConnect products produce statistics every minute while the industry standard is generally every five minutes.

While the HPE Aruba Networking Orchestrator is the main consumer of this data and uses it extensively in reporting and troubleshooting, there are many situations where customers will need to access these statistics. The use cases for statistics range substantially and can span anything from integration with diagnostic tools to external reporting tools, to the development of customized health
dashboards. Orchestrator offers a REST API to provide periodic access to statistics (including real-time, time-series, and aggregated statistics). Still, for use cases that require large volumes of statistics or frequent polling of statistics (such as access to minute-granularity statistics), Orchestrator is not designed for these use cases and thus, not the right integration point.

The REST API is not suited for high-frequency polling of granular statistics or “data replication” into an external data store for two reasons: 1) the volume of data that is requested scales linearly with the number of EdgeConnect devices and the granularity of the requested statistics, and 2) Orchestrator does not have the resources for the load that frequent REST API calls would impose. Orchestrator serves multiple purposes and has limited resources tuned to do many functions, including:

Supporting 5-50 concurrent users accessing the UI
Orchestrating policies across the SD-WAN fabric with 2 to 1,000 devices
Collecting and reporting statistics from these devices for Orchestrator users
Servicing third-party API requests from external systems

Collecting and reporting statistics consumes large amounts of CPU and memory resources. Since Orchestrator is not designed to provide end applications with a copy of its statistics, HPE Aruba Networking provides REST APIs that allow customers to obtain this data directly from the appliances for use by third-party applications. This approach is required when the use case calls for access to fine-grained statistics and/or frequent access to the statistics and avoids overloading Orchestrator by distributing the load across the SD-WAN.

Pre-Requisites

Before collecting statistics using the EdgeConnect REST APIs, the following prerequisite must be met:

Appliance access: To access the EdgeConnect REST APIs directly, customers must have an up-to-date list of appliances and a method to access their REST API endpoints. The list of appliances and their configuration information can be retrieved from the Orchestrator REST API as part of inventory discovery.
Resource name or ID: Depending on which resource the customer is interested in (for example, tunnels, interfaces, flows, and so on), the identity of that resource needs to be known. This can be obtained from the Orchestrator REST API as part of the inventory discovery. The resource's ID or name is used to identify and associate the statistics in the files.
Login access to EdgeConnect gateways: File retrieval from EdgeConnect requires login access to EdgeConnect to ensure authorized access. HPE Aruba Networking recommends that customers create or orchestrate an "api-user" login on the edgeConnect appliances for use by the third-party application that is retrieving the files.

Overview of file retrieval methodology

To use EdgeConnect statistics in an external application, one can poll each appliance directly using REST APIs. The following steps outline how to do this.

Invoke the EdgeConnect REST API to determine which statistics files are available. EdgeConnect stores a fixed number of statistics files. The REST API call provides information about the time range that is currently available.
Request the stats file for the minute(s) of interest. The stats files are zipped. Information about which stats are contained in each file is described in the examples.
Store the files of interest and parse or ingest them as needed. The file formats for various statistics files are described in the examples.