Wednesday, December 28, 2011

Linux process memory usage calculations

It is difficult to obtain the actual memory consumption of a process in Linux/Unix environment.We cannot always pin-point the exact memory usage of a process as most processes on Linux uses shared library.

Assume that we want to calculate memory usage for the 'ls' process. Do we count only the memory used by the executable 'ls' ? How about libc? Or all these other libs that are required to run 'ls' ? One could argue that they are shared by other processes, but 'ls' can't be run on the system without them being loaded.Also, if we need to know how much memory a process needs in order to do capacity planning, we would need to calculate how much each additional copy of the process uses

A close approximation of memory consumtion of a process is being provided by VSS(virtual Set Size) of the process.

We can obtain the VSS value from 'ps aux' command line. This is handy when we just need to view the process memory consumption once in a while.

However, the use of 'ps' command is not suitable when designing an application which reports system health. Imagine a health-check application which needs to report the memory consumtion of some resource intensive process running on the box. If the health-check app itself consumes 4-5% of memory as it needs to run say every 10sec then it adds an unnecessary overhead on the Linux/Unix host which is already resource starved. 'ps' command itself is taxing on resource memory utilisation.The same goes true for other commands e.g top, vmstat, etc

So to increase the efficiency of the program reporting the system health, we should avoid the use of 'ps' command. So what alternatives are available to obtain the VSS of a process?

VSS information can be obtained from /proc filesystem.VSS information is present in 23rd argument in /proc/$pid/stat. This is the same place from where ps command obtains its value. Its value can be obtained from /proc/$pid/status from VmSize value as well./proc/$pid/smaps also provide this information in detailed breakdown manner but is not worth the effort in most of the scenarios as the value of VSS is a close approximation to the actual memory being used by the system.

For the sake of calculating the %age memory consumtion by a process we can obtain the system memory consumtion from file /proc/meminfo. The argumenet MemTotal depicts the total memory consumption by the system.

The most efficient and easy way is to obtain it from a C/C++ program reading the /proc filesystem.C language is preferable on the consistency front as all Unix/Linux kernel supports C.

No comments:

Post a Comment