Azure HDInsight is a fully managed, full-spectrum, open-source analytics service in the cloud for enterprises. The Apache Hadoop cluster type in Azure HDInsight allows you to use HDFS, YARN resource management, and a simple MapReduce programming model to process and analyze batch data in parallel.
Use the Azure public cloud integration to discover and collect metrics against the Azure service.
External reference
Azure Monitor - Platform metrics
Setup
To set up the Azure integration and discover the Azure service, go to Azure Integration Discovery Profile and select HD Insight Cluster.
Event support
- Supported
- Configurable in OpsRamp Azure Integration Discovery Profile.
Supported metrics
| OpsRamp Metric | Azure Metric | Metric Display Name | Unit | Aggregation Type | Description |
|---|---|---|---|---|---|
| azure_hdinsight_clusters_CategorizedGatewayRequests | CategorizedGatewayRequests | Categorized Gateway Requests | Count | Count | Number of gateway requests by categories (1xx/2xx/3xx/4xx/5xx) |
| azure_hdinsight_clusters_GatewayRequests | GatewayRequests | Gateway Requests | Count | Count | Number of gateway requests |
| azure_hdinsight_clusters_KafkaRestProxyConsumerRequestm1_delta | KafkaRestProxy.ConsumerRequest.m1_delta | REST proxy Consumer RequestThroughput | CountPerSecond | Total | Number of consumer requests to Kafka REST proxy |
| azure_hdinsight_clusters_KafkaRestProxyConsumerRequestFailm1_delta | KafkaRestProxy.ConsumerRequestFail.m1_delta | REST proxy Consumer Unsuccessful Requests | CountPerSecond | Total | Consumer request exceptions |
| azure_hdinsight_clusters_KafkaRestProxyConsumerRequestTimep95 | KafkaRestProxy.ConsumerRequestTime.p95 | REST proxy Consumer RequestLatency | Milliseconds | Average | Message latency in a consumer request through Kafka REST proxy |
| azure_hdinsight_clusters_KafkaRestProxyConsumerRequestWaitingInQueueTimep95 | KafkaRestProxy.ConsumerRequestWaitingInQueueTime.p95 | REST proxy Consumer Request Backlog | Milliseconds | Average | Consumer REST proxy queue length |
| azure_hdinsight_clusters_KafkaRestProxyMessagesInm1_delta | KafkaRestProxy.MessagesIn.m1_delta | REST proxy Producer MessageThroughput | CountPerSecond | Total | Number of producer messages through Kafka REST proxy |
| azure_hdinsight_clusters_KafkaRestProxyMessagesOutm1_delta | KafkaRestProxy.MessagesOut.m1_delta | REST proxy Consumer MessageThroughput | CountPerSecond | Total | Number of consumer messages through Kafka REST proxy |
| azure_hdinsight_clusters_KafkaRestProxyOpenConnections | KafkaRestProxy.OpenConnections | REST proxy ConcurrentConnections | Count | Total | Number of concurrent connections through Kafka REST proxy |
| azure_hdinsight_clusters_KafkaRestProxyProducerRequestm1_delta | KafkaRestProxy.ProducerRequest.m1_delta | REST proxy Producer RequestThroughput | CountPerSecond | Total | Number of producer requests to Kafka REST proxy |
| azure_hdinsight_clusters_KafkaRestProxyProducerRequestFailm1_delta | KafkaRestProxy.ProducerRequestFail.m1_delta | REST proxy Producer Unsuccessful Requests | CountPerSecond | Total | Producer request exceptions |
| azure_hdinsight_clusters_KafkaRestProxyProducerRequestTimep95 | KafkaRestProxy.ProducerRequestTime.p95 | REST proxy Producer RequestLatency | Milliseconds | Average | Message latency in a producer request through Kafka REST proxy |
| azure_hdinsight_clusters_KafkaRestProxyProducerRequestWaitingInQueueTimep95 | KafkaRestProxy.ProducerRequestWaitingInQueueTime.p95 | REST proxy Producer Request Backlog | Milliseconds | Average | Producer REST proxy queue length |
| azure_hdinsight_clusters_NumActiveWorkers | NumActiveWorkers | Number of Active Workers | Count | Average | Number of Active Workers |
| azure_hdinsight_clusters_PendingCPU | PendingCPU | Pending CPU | Count | Average | Pending CPU Requests in YARN |
| azure_hdinsight_clusters_PendingMemory | PendingMemory | Pending Memory | Count | Average | Pending Memory Requests in YARN |