banshee/windows-collection/windows/winrm/diskio

Disk activity of a Windows, by WinRM

Returns, for each physical disk of a Windows, the operations and the throughput of reads and writes, the mean wait of an operation, the share of time the disk was busy and the operations waiting in its queue. Four watches are proposed, each one set on the probe. The space of the volumes is the collector windows/winrm/volumes.

Versions

Windows Server 2012 and later, Windows 8 and later: Windows PowerShell 3.0, which the collector needs, ships with each of them; the WinRM service runs by default on every Windows Server from 2012 on, and winrm quickconfig starts it on a desktop Windows. Generic: no folder of version. That every release answers alike is assumed, not yet checked on a real Windows.

How it works

The collector runs one PowerShell script, by WinRM, that reads the raw counters of Win32_PerfRawData_PerfDisk_PhysicalDisk, one object per physical disk:

ConvertTo-Json -Compress -InputObject @(Get-CimInstance Win32_PerfRawData_PerfDisk_PhysicalDisk | Select-Object Name, DiskReadsPerSec, DiskWritesPerSec, DiskReadBytesPerSec, DiskWriteBytesPerSec, AvgDiskSecPerRead, AvgDiskSecPerRead_Base, AvgDiskSecPerWrite, AvgDiskSecPerWrite_Base, PercentIdleTime, CurrentDiskQueueLength, Timestamp_Sys100NS, Frequency_PerfTime)

WinRM hands the script to PowerShell itself: no shell stands in between, and the script is the one of windows/ssh/diskio, without the powershell -Command "…" that cmd.exe needs there. The default shell of an OpenSSH server does not matter here.
Each counter adds up since boot; the collector returns its rate between two runs: - operations per second, from DiskReadsPerSec and DiskWritesPerSec; - throughput, from DiskReadBytesPerSec and DiskWriteBytesPerSec, in MiB/s; - the mean wait of an operation, from AvgDiskSecPerRead (the waits added up, in ticks of Frequency_PerfTime) over AvgDiskSecPerRead_Base (the operations), in ms; the same for writes; - the busy share, 1 − idle / clock, from PercentIdleTime against the clock Timestamp_Sys100NS.
CurrentDiskQueueLength is not a counter: the operations waiting at the time of the reading.
The raw counters, not the formatted ones: a formatted rate needs two samples taken by the same process, and each run starts a new PowerShell. Their names are English whatever the language of Windows.
A disk is named as Windows names it, its number then the letters of its volumes: 0 C:, 1 D: E:. The _Total object is left out.
The first run has no interval yet: it is skipped. So is a run where every disk had a counter going backwards.
No disk at all, or an answer that is not the expected JSON, stops the collector as misconfigured: never a green light without a measure.

Nothing is written to the Windows: the script only reads.

Prerequisites

The WinRM service of Windows listening in HTTPS. The service runs by default on every Windows Server from 2012 on (on a desktop Windows, winrm quickconfig starts it), with an HTTP listener only; the HTTPS listener needs a certificate: - a certificate for server authentication in the store of the computer (Cert:\LocalMachine\My), bearing the name the poller connects to: issued by the authority of the company, or made on the server by New-SelfSignedCertificate -DnsName <name> -CertStoreLocation Cert:\LocalMachine\My from Windows Server 2012 R2 on (Windows Server 2012 does not have this command); - then the listener: winrm quickconfig -transport:https finds a certificate of an authority by itself, but refuses a self-signed one; the command that takes either, on every Windows from Server 2012 on, is winrm create winrm/config/Listener?Address=*+Transport=HTTPS @{Hostname="<name>";CertificateThumbprint="<thumbprint>"} in cmd.exe, the thumbprint being the one Windows shows for the certificate.
The Windows firewall open to the poller on TCP 5986, which creating the listener does not do: New-NetFirewallRule -DisplayName "WinRM HTTPS" -Direction Inbound -Protocol TCP -LocalPort 5986 -Action Allow in PowerShell. A listener on another port is set in protocol:winrm:port.
The certificate judged by the poller before the account is sent, by the TLS profile of the probe: issued by an authority the system of the poller knows, by an authority added in protocol:tls:authorities, or pinned in protocol:winrm:fingerprint, the way for a self-signed certificate; the pin is not needed when the Security settings trust the first WinRM fingerprint presented. The fingerprint is sha256: followed by the SHA-256 of the certificate in 64 lower-case hexadecimal characters; from a machine that reaches the port, openssl s_client -connect <name>:5986 </dev/null | openssl x509 -noout -fingerprint -sha256 prints it, in capitals, with colons to take off. A certificate whose name is not the address of the host is judged on the name set in protocol:tls:server-name.
NTLM authentication, which the WinRM service accepts as installed (Negotiate = true in winrm get winrm/config/service/auth): the password never travels, only an answer computed with it. A domain that refuses NTLM is not served.
Windows PowerShell 3.0 or later, as shipped with every Windows from Server 2012 and Windows 8 on (3.0 on Server 2012, 4.0 on 2012 R2, 5.1 from 2016 on).
An account for monitoring, without administrator rights, local or of the domain, set in protocol:winrm:user (account, DOMAIN\account or account@domain) with its password in protocol:winrm:password, member of the local group "Remote Management Users", which opens WinRM to an account that is not an administrator. Also a member of "Performance Monitor Users": the collector reads performance counters, which Windows refuses outside this group to a WinRM session, a network logon.

Outputs

Name	Label	Unit	Per instance
`read-iops`	Reads	op/s	yes, per disk
`write-iops`	Writes	op/s	yes, per disk
`read-rate`	Read throughput	MiB/s	yes, per disk
`write-rate`	Write throughput	MiB/s	yes, per disk
`read-wait`	Mean read wait	ms	yes, per disk
`write-wait`	Mean write wait	ms	yes, per disk
`busy`	Busy time	%	yes, per disk
`queue`	Operations in the queue	—	yes, per disk

Watches

Name	Title	Kind	Defaults	Checked by default
`read-latency`	Slow reads	above a threshold, on `read-wait`	warning above 50 ms, critical above 200 ms, for 5 min	yes
`write-latency`	Slow writes	above a threshold, on `write-wait`	warning above 50 ms, critical above 200 ms, for 5 min	yes
`busy`	Disk saturated	above a threshold, on `busy`	warning above 90 %, for 10 min; no critical	no
`queue-long`	Long disk queue	above a threshold, on `queue`	no threshold: to set on the probe; for 5 min	no

Limits

A certificate of the listener that expires, or a pinned one that is renewed, stops every WinRM collector of the host: the network/tls/certificate collector of banshee/web-collection, deployed on port 5986, watches it.
Values arrive from the second run on; a probe whose interval is long averages over it.
The busy share only means something for a disk that serves one operation at a time: an SSD, or a volume of a disk array, at 100 % may still serve many more.
The wait of a read or a write only exists if there was one during the interval: a disk that did not read returns no read wait.
WMI declares the wait counters on 32 bits: a counter that wraps around between two runs gives no wait for that run.
The queue is a moment, read once per run.
Not yet checked on a real Windows: that "Performance Monitor Users" is the group a WinRM session needs for the counters.