Cloudstrack Triggering Automatic VM stop

[midhunpjos]

Hi Community,

Recently I am facing and issue in which cloudstack is stopping a recently started VM multiple times. As per logs i can see the below:

2025-01-19 15:40:25,675 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-59:ctx-479e17ca job-63513/job-64194) (logid:e7a741de) Run VM work job: com.cloud.vm.VmWorkStop for VM 12492, job origin: 63513
2025-01-19 15:40:25,679 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-59:ctx-479e17ca job-63513/job-64194 ctx-10734987) (logid:e7a741de) Execute VM work job: com.cloud.vm.VmWorkStop{"cleanup":true,"userId":1,"accountId":1,"vmId":12492,"handlerName":"VirtualMachineManagerImpl"}
2025-01-19 15:40:25,711 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-59:ctx-479e17ca job-63513/job-64194 ctx-10734987) (logid:e7a741de) VM instance {"id":12492,"instanceName":"i-279-12492-VM","type":"User","uuid":"b5c68df2-4da4-4000-a040-b0d0bd87e09f"} state transited from [Running] to [Stopping] with event [StopRequested]

Job is successful and VM is Getting stopped without user intervention

Job ID 63513 is showing below in DB:
*************************** 1. row ***************************
id: 63513
user_id: 1
account_id: 1
instance_type: Thread
instance_id: 93
job_cmd: NULL
job_cmd_info: NULL
job_cmd_ver: 0
job_status: 0
job_process_status: 0
job_result_code: 0
job_result: NULL
job_init_msid: 108462341209969
job_complete_msid: NULL
created: 2025-01-14 15:36:28
last_updated: NULL
last_polled: NULL
removed: NULL
uuid: e7a741de-f4d6-4e75-9a38-440534155b2a
related:
job_type: NULL
job_dispatcher: pseudoJobDispatcher
job_executing_msid: NULL
job_pending_signals: 1
1 row in set (0.00 sec)

Can someone guide what could be triggering a VM stop ?

[weizhouapache]

@midhunpjos
it would be good to share the logs right before it

[midhunpjos]

@midhunpjos it would be good to share the logs right before it

Adding the same as per your request

2025-01-19 15:40:25,659 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-4aef2f75) (logid:468b1806) Execute sync-queue item: SyncQueueItemVO {id:33883, queueId: 23356, contentType: AsyncJob, contentId: 64194, lastProcessMsid: 271367514881978, lastprocessNumber: 28, lastProcessTime: Sun Jan 19 15:40:25 NPT 2025, created: Sun Jan 19 15:40:24 NPT 2025}
2025-01-19 15:40:25,660 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-4aef2f75) (logid:468b1806) Schedule queued job-64194
2025-01-19 15:40:25,669 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-59:ctx-479e17ca job-63513/job-64194) (logid:ed9528eb) Add job-64194 into job monitoring
2025-01-19 15:40:25,675 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-59:ctx-479e17ca job-63513/job-64194) (logid:e7a741de) Executing AsyncJobVO: {id:64194, userId: 1, accountId: 1, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo: rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAwzHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwB, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 108462341209969, completeMsid: null, lastUpdated: null, lastPolled: null, created: Sun Jan 19 15:40:24 NPT 2025, removed: null}
2025-01-19 15:40:25,675 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-59:ctx-479e17ca job-63513/job-64194) (logid:e7a741de) Run VM work job: com.cloud.vm.VmWorkStop for VM 12492, job origin: 63513
2025-01-19 15:40:25,679 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-59:ctx-479e17ca job-63513/job-64194 ctx-10734987) (logid:e7a741de) Execute VM work job: com.cloud.vm.VmWorkStop{"cleanup":true,"userId":1,"accountId":1,"vmId":12492,"handlerName":"VirtualMachineManagerImpl"}

[weizhouapache]

@midhunpjos
I still did not get what triggered the vm stop.
can you share the logs in 30 seconds prior to it ?