virsh qemu-monitor-command does not change cpu index

[sathnaga]

cde:info Mirrored with LTC bug https://bugzilla.linux.ibm.com/show_bug.cgi?id=161249 </cde:info>

libvirt-3.6.0-3.rel.gitdd9401b.el7.centos.ppc64le

cpu index is not getting changed using virsh qemu-monitor-command command

virsh qemu-monitor-command virt-tests-vm1 --cmd "info cpus" --hmp
* CPU #0: nip=0xc0000000000b715c thread_id=34297
  CPU #1: nip=0xc0000000000b715c thread_id=63839

# virsh qemu-monitor-command virt-tests-vm1 --cmd "cpu 1" --hmp

# echo $?
0

# virsh qemu-monitor-command virt-tests-vm1 --cmd "info cpus" --hmp
* CPU #0: nip=0xc0000000000b715c thread_id=34297
  CPU #1: nip=0xc0000000000b715c thread_id=63839-----------------------NOK

# virsh qemu-monitor-command virt-tests-vm1 --cmd "cpu 2" --hmp
invalid CPU index-----------OK


the same works fine with just qemu-kvm
qemu-kvm -M pseries,max-cpu-compat=power9,accel=kvm -smp 2,cores=2,threads=1,sockets=1 -m 256 -nographic -enable-kvm /var/lib/libvirt/images/workspace/runAvocadoFVTTest/avocado-fvt-wrapper/data/avocado-vt/images/hostos-3.0-ppc64le.qcow2 -monitor stdio -serial /dev/pts/5

(qemu) info cpus
* CPU #0: nip=0xc0000000000b190c thread_id=59666
  CPU #1: nip=0xc0000000000b190c thread_id=60775
(qemu) cpu 1
(qemu) info cpus
  CPU #0: nip=0xc0000000000b190c thread_id=59666
* CPU #1: nip=0xc0000000000b190c thread_id=60775-----------------OK

(qemu) cpu 2
invalid CPU index

[sathnaga]

# virsh qemu-monitor-command virt-tests-vm1 --cmd "info cpus" --hmp
* CPU #0: nip=0xc0000000000b715c thread_id=34297
  CPU #1: nip=0xc0000000000b715c thread_id=63839

# virsh qemu-monitor-command virt-tests-vm1 --cmd '{ "execute": "cpu", "arguments": { "index": 1 } }'
{"return":{},"id":"libvirt-60"}

# virsh qemu-monitor-command virt-tests-vm1 --cmd '{ "execute": "query-cpus" }' --pretty
{
  "return": [
    {
      "arch": "ppc",
      "current": true,
      "props": {
        "core-id": 0,
        "node-id": 0
      },
      "CPU": 0,
      "nip": -4611686018426637988,
      "qom_path": "/machine/unattached/device[0]/thread[0]",
      "halted": false,
      "thread_id": 34297
    },
    {
      "arch": "ppc",
      "current": false,
      "props": {
        "core-id": 1,
        "node-id": 0
      },
      "CPU": 1,
      "nip": -4611686018426637988,
      "qom_path": "/machine/peripheral/vcpu1/thread[0]",
      "halted": false,
      "thread_id": 63839
    }
  ],
  "id": "libvirt-61"
}

# virsh qemu-monitor-command virt-tests-vm1 --cmd "info cpus" --hmp
* CPU #0: nip=0xc0000000000b715c thread_id=34297
  CPU #1: nip=0xc0000000000b715c thread_id=63839

[cdeadmin]

------- Comment From [email protected] 2017-12-11 15:46:55 EDT-------
After investigating I'm tempted to close this as a NOTABUG, but first I'll ask for thoughts/opinions from the Libvirt team.

The issue here is what the 'cpu' command does and how "virsh qemu-monitor-command" executes it. The 'cpu' command sets a default CPU for the current monitor session. It is not a VM state, but a monitor state. It is possible for multiple monitors in the same VM to have different settings. In the example below I've started a VM with 2 HMP monitors, one in local port 4444 and other in local port 5555. Note that setting the default CPU in one monitor does not change the 'info cpus' output from the other:

First monitor:

(qemu) cpu 1
cpu 1
(qemu)
(qemu) info cpus
info cpus
CPU #0: pc=0xffffffff900d3d66 (halted) thread_id=29478

• CPU PCI-Multifunction hotplug support #1: pc=0xffffffff900d3d66 (halted) thread_id=29479
  CPU Snapshot-revert making the attached interface persistent in the guest #2: pc=0xffffffff900d3d66 (halted) thread_id=29480
  CPU Feature: Add support for guest smt in power9 [qemu machine property vsmt to be enabled]  #3: pc=0xffffffff900d3d66 (halted) thread_id=29481
  (qemu)

Second monitor:

(qemu)
(qemu) info cpus
info cpus

• CPU #0: pc=0xffffffff900d3d66 (halted) thread_id=29478
  CPU PCI-Multifunction hotplug support #1: pc=0xffffffff900d3d66 (halted) thread_id=29479
  CPU Snapshot-revert making the attached interface persistent in the guest #2: pc=0xffffffff900d3d66 (halted) thread_id=29480
  CPU Feature: Add support for guest smt in power9 [qemu machine property vsmt to be enabled]  #3: pc=0xffffffff900d3d66 (halted) thread_id=29481
  (qemu)
  (----- after setting "cpu 1" in the first monitor ----)
  (qemu) info cpus
  info cpus
• CPU #0: pc=0xffffffff900d3d66 (halted) thread_id=29478
  CPU PCI-Multifunction hotplug support #1: pc=0xffffffff900d3d66 (halted) thread_id=29479
  CPU Snapshot-revert making the attached interface persistent in the guest #2: pc=0xffffffff900d3d66 (halted) thread_id=29480
  CPU Feature: Add support for guest smt in power9 [qemu machine property vsmt to be enabled]  #3: pc=0xffffffff900d3d66 (halted) thread_id=29481
  (qemu)

Which bring us back to "virsh qemu-monitor-command". This virsh command makes Libvirt issue a QMP command called "human-monitor-command". This command will open up a new HMP monitor, execute the command and then exit, destroying the HMP monitor it created. There is no state written in both the VM or the current QMP monitor that Libvirt uses to execute the commands. So, this command:

virsh qemu-monitor-command virt-tests-vm1 --cmd "cpu 1" --hmp

• executes QMP 'human-monitor-command' with 'cpu 1' as the command to be execute
• QEMU creates a new HMP monitor and executes 'cpu 1' in it
• QEMU destroys the HMP monitor

And this is why you'll always see the same output from both this and the "info cpu" command - they are being executed in a fresh HMP monitor every time. In the end, this will never work as intended by the tester. This alone is enough to close this as NOTABUG.

Now, an alternative would be to use the QMP commands "cpu" and "query-cpus" to do the same thing, but then I realized that "cpu" does not have a QMP implementation and "query-cpus" will not return the intended values at it is. I'll send patches to try to get this implemented in QMP to offer an alternative for Libvirt.

Thoughts?

[cdeadmin]

------- Comment From [email protected] 2017-12-16 13:06:56 EDT-------
In the QEMU mailing list, the QMP/HMP maintainer explained that QMP is supposed to be stateless, as opposed to HMP. It was also mentioned that any QMP command that might make use of the current monitor CPU should be pass it as a parameter. This means that the community does not want 'qmp_cpu' to be implemented. In fact, I'll send a patch to put the command in the deprecated list.

As far as this bug is concerned, if Libvirt wants to use a specific CPU index to execute a command, it should use the cpu as a parameter instead of setting the active CPU. Given what I've already said in my last comment about how qemu-monitor-command --hmp works, I'm closing this bug as NOTABUG.

Thanks,

Daniel