SAR Data Online Charting Practical Guide

If performance issues have occurred historically but no monitoring system has collected data, then sar might be your only option left. SAR is like the system's "health report" that can tell you what exactly happened to CPU, memory, disk, and network over a period of time.

What exactly is sar?

SAR stands for System Activity Reporter. Simply put, it records various system performance metrics every few minutes. Unlike top or htop which only show current status, sar allows you to go back in time and see what exactly happened to the system at 2 AM yesterday.

However, using sar comes with a significant cost - there are no good visualization tools to display sar data. In most cases, you can only manually browse through the data, making problem identification very costly.

ctbots.com to Bring SAR Data to Life

I later developed ctbots.com's sar online analysis tool.

Tool URL: https://ctbots.com/en/linux/graph/sar.html

It supports the following features:

• Supports plotting single-day sar data
• Supports plotting multi-day sar data after concatenation
• Supports time interval zooming - after selecting a time zoom interval, all chart time intervals synchronize zooming
• Display metric items are configured with corresponding help descriptions

Common Suspicious Points in Performance Issues

Based on my experience, these areas are most likely to expose problems:

CPU-related Issues

• %user over 80%: Application CPU-intensive operations
• %system over 30%: Possibly high volume of system calls
• %iowait over 10%: Disk I/O becoming bottleneck
• %steal over 5%: Resource contention in virtualized environment

Memory-related Issues

• %memused over 90%: Insufficient memory
• kbcommit continuously growing: Possible memory leak

Disk I/O Issues

• %util close to 100%: Disk running at full capacity
• await over 20ms: Slow disk response
• avgqu-sz over 2: Disk queue backlog

Network-related Issues

• Traffic suddenly drops to 0: Network disconnection
• Packet count and traffic mismatch: Possible small packet attacks