Overview

Today, I want to try to reproduce a problem, need to achieve an effect is to hit the CPU full, create a high load situation, my first idea is to use stress, the results found that actually has not been able to hit, can only hit half (50%) to the top, so I explored what the problem, by the way, record.

Operation process

First, I first looked at how many cores there are on this machine, and then to start stress with the corresponding number of cores.

  1. [root@liqiang.io]# cat /proc/cpuinfo | grep "processor" | wc -l
  2. 6

This is the number of logical cores, but the physical cores are also 6, and each core is 1, so it is also 6, no problem. Then it’s time to start stress.

  1. [root@liqiang.io]# stress -c 6

Then you see the CPU usage like this.

Figure 1: stress cpu usage

This is awesome, you can see that stress actually only takes up one core, and then the utilization rate is only one 100%, which is not what I expected, and then I was like, I’ll open two more on the line, and then I opened another terminal, and then ran another one, and it turned out to be like this

Figure 2: Multi-open stress cpu usage

Look at them, the ripples continue to even out, how exasperating. So what is the problem? Why would another open terminal also run to the same core inside it, so I’ll take a closer look at the stress documentation.

How stress works

So my suspicion is that a colleague has restricted the cgroup of the shell, so I can only use a fixed core.

  1. [root@liqiang.io]# cat /proc/5813/cgroup 
  2. 11:perf_event:/
  3. 10:cpuset:/zbs/app
  4. 9:devices:/system.slice/sshd.service
  5. 8:freezer:/
  6. 7:pids:/
  7. 6:blkio:/system.slice/sshd.service
  8. 5:net_prio,net_cls:/
  9. 4:hugetlb:/
  10. 3:cpuacct,cpu:/system.slice/sshd.service
  11. 2:memory:/system.slice/sshd.service
  12. 1:name=systemd:/system.slice/sshd.service

ok, from here you can see that the cpuset used is a cgroup entry called /zbs/app, so one more step to see exactly how it is restricted.

  1. [root@liqiang.io]# cat /etc/cgconfig.conf | grep -A 7 zbs/app
  2. group zbs/app {
  3.     cpuset {
  4.         cpuset.cpus = "4,5";
  5.         cpuset.mems = "0";
  6.         cpuset.cpu_exclusive = "0";
  7.         cpuset.mem_hardwall = "1";
  8.     }
  9. }

You can see that it is limited to cpu 4 and 5, check the top cpu usage to see if it matches.

Figure 3: Detailed cpu usage

ok, this looks like the problem, so if I want to fill up the CPU, the easiest way to deal with it is to remove this cgroup limit, but that’s not very friendly, so a more friendly way would be to filter my stress processes separately and leave the rest as is.