2 Replies Latest reply on May 31, 2017 11:00 AM by arun7

    soft lockup and cpu stuck for 22s 

    arun7

      I installed ubuntu 16.04 and launched openstack ocata setup on it .While working on it ,often am getting error like "soft lockup cpu stuck for 22s" and /proc/sys/kernel/hung_task_timeout_secs for 120 seconds .

      I dont know whether i need to upgrade my kernel and derivers or not .please help me to fix it .

       

      Thanks in advance..!

        • Re: soft lockup and cpu stuck for 22s 
          ray_m

          Please take the time to provide the essential INFORMATION REQUIRED WHEN POSTING A QUESTION.

            • Re: soft lockup and cpu stuck for 22s 
              arun7

              Hello ray,

               

              specs of my system:

              MotherBoard       : Gigabyte AB-350-Gaming 3

              Processor            : AMD ryzen 7 1700 (8 core,voltage 1.2V),20mb cache ,3.0GHZBase/3.7GHZ precision Boost  ,wattage 65w

              Graphics Card     : Nvidia corp GK 208[GEForce GT 710B]

              OS                       : Linux os (ubuntu 14.04,16.04 with kernel version 4.10.3)

              RAM                    : 2 * 16GB

              Driver version      : default drivers available in linux kernel and several linux firmwares(http://www.nextized.net/repo/linux/ubuntu-linux-4.10.3_ryzen01_amd64_generic.tar.gz )

               

              Display Device : Samsung 50hz /230v adapter

               

              i am facing several issues related to kernel and hardware incompatibility(device drivers).

              1, Error with khugepaged ,solved it by disabling it in kernel config file .[cpu stuck for 22s]error

              2,Linux Kernel panic issue:hung_task_timeout_secs and blocked for more than 120 seconds problem

              In addition to this ,i updated kernel version to 4.10.3 from 4.4.0 (http://www.nextized.net/repo/linux/ubuntu-linux-4.10.3_ryzen01_amd64_generic.tar.gz ) .

               

              After all this ,still am getting error like ,

               

               

              May 30 21:39:07 ubuntu kernel: [ 8333.220405] INFO: rcu_sched detected stalls on CPUs/tasks:

              May 30 21:39:07 ubuntu kernel: [ 8333.222212]   9-...: (21 GPs behind) idle=943/1/0 softirq=2735/2735 fqs=7471

              May 30 21:39:07 ubuntu kernel: [ 8333.223845]   (detected by 12, t=15002 jiffies, g=82766, c=82765, q=1386)

              May 30 21:39:07 ubuntu kernel: [ 8333.225457] Sending NMI from CPU 12 to CPUs 9:

              May 30 21:39:07 ubuntu kernel: [ 8333.226452] NMI watchdog: Watchdog detected hard LOCKUP on cpu 9

              May 30 21:39:07 ubuntu kernel: [ 8333.226453] Modules linked in: snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer snd soundcore ccp input_leds shpchp i2c_piix4 edac_mce_amd edac_core kvm_amd kvm i2c_designware_platform tpm_infineon mac_hid i2c_designware_core 8250_dw irqbypass ib_iser rdma_cm iw_cm ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 raid0 multipath linear hid_generic usbhid hid usb_storage nouveau mxm_wmi video i2c_algo_bit bnx2x ttm drm_kms_helper crct10dif_pclmul crc32_pclmul ghash_clmulni_intel syscopyarea sysfillrect pcbc sysimgblt aesni_intel aes_x86_64 ptp fb_sys_fops crypto_simd glue_helper pps_core

              May 30 21:39:07 ubuntu kernel: [ 8333.226471]  cryptd drm r8169 mdio ahci libcrc32c mii libahci wmi gpio_amdpt gpio_generic

              May 30 21:39:07 ubuntu kernel: [ 8333.226474] CPU: 9 PID: 0 Comm: swapper/9 Not tainted 4.11.0 #1

              May 30 21:39:07 ubuntu kernel: [ 8333.226475] Hardware name: Gigabyte Technology Co., Ltd. Default string/AB350-Gaming 3-CF, BIOS F5 02/24/2017

              May 30 21:39:07 ubuntu kernel: [ 8333.226476] task: ffff97a53aaa2d00 task.stack: ffffbe2843264000

              May 30 21:39:07 ubuntu kernel: [ 8333.226476] RIP: 0010:native_queued_spin_lock_slowpath+0x17a/0x1a0

              May 30 21:39:07 ubuntu kernel: [ 8333.226477] RSP: 0018:ffff97a53e843a98 EFLAGS: 00000002

              May 30 21:39:07 ubuntu kernel: [ 8333.226477] RAX: 0000000000000101 RBX: 0000000000000086 RCX: 0000000000000001

              May 30 21:39:07 ubuntu kernel: [ 8333.226478] RDX: 0000000000000101 RSI: 0000000000000001 RDI: ffff97a531857d08

              May 30 21:39:07 ubuntu kernel: [ 8333.226479] RBP: ffff97a53e843a98 R08: 0000000000000101 R09: 000000000000003a

              May 30 21:39:07 ubuntu kernel: [ 8333.226479] R10: 00000000ffffffff R11: 0000000000000000 R12: ffff97a531857d08

              May 30 21:39:07 ubuntu kernel: [ 8333.226480] R13: ffff97a5321e2d00 R14: ffff97a531857c00 R15: ffff97a531857c00

              May 30 21:39:07 ubuntu kernel: [ 8333.226480] FS:  0000000000000000(0000) GS:ffff97a53e840000(0000) knlGS:0000000000000000

              May 30 21:39:07 ubuntu kernel: [ 8333.226481] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

              May 30 21:39:07 ubuntu kernel: [ 8333.226481] CR2: 00007f2909496624 CR3: 00000007f9337000 CR4: 00000000003406e0

              May 30 21:39:07 ubuntu kernel: [ 8333.226481] Call Trace:

              May 30 21:39:07 ubuntu kernel: [ 8333.226482]  <IRQ>

              May 30 21:39:07 ubuntu kernel: [ 8333.226482]  _raw_spin_lock_irqsave+0x37/0x3f

              May 30 21:39:07 ubuntu kernel: [ 8333.226483]  nvkm_fantog_update+0x43/0x110 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226483]  nvkm_fantog_set+0x38/0x40 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226483]  nvkm_fan_update+0xbf/0x200 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226484]  nvkm_therm_fan_set+0x19/0x20 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226484]  nvkm_therm_update+0x98/0x280 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226485]  nvkm_therm_alarm+0x17/0x20 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226485]  nvkm_timer_alarm_trigger+0x100/0x150 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226486]  nvkm_timer_alarm+0x7f/0xd0 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226486]  alarm_timer_callback+0xce/0xd0 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226487]  nvkm_timer_alarm_trigger+0x100/0x150 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226487]  nvkm_timer_alarm+0x7f/0xd0 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226487]  nvkm_fantog_update+0x105/0x110 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226488]  nvkm_fantog_alarm+0x1a/0x20 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226488]  nvkm_timer_alarm_trigger+0x100/0x150 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226489]  nv04_timer_intr+0x5c/0xa0 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226489]  nvkm_timer_intr+0x17/0x20 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226490]  nvkm_subdev_intr+0x17/0x20 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226490]  nvkm_mc_intr+0xe5/0x190 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226490]  nvkm_pci_intr+0x53/0x80 [nouveau]

              May 30 21:39:07 ubuntu kernel: [ 8333.226491]  __handle_irq_event_percpu+0x7e/0x1a0

              May 30 21:39:07 ubuntu kernel: [ 8333.226491]  handle_irq_event_percpu+0x32/0x80

              May 30 21:39:07 ubuntu kernel: [ 8333.226492]  handle_irq_event+0x3b/0x60

              May 30 21:39:07 ubuntu kernel: [ 8333.226492]  handle_edge_irq+0x74/0x130

              May 30 21:39:07 ubuntu kernel: [ 8333.226492]  handle_irq+0x1a/0x30

              May 30 21:39:07 ubuntu kernel: [ 8333.226493]  do_IRQ+0x46/0xd0

              May 30 21:39:07 ubuntu kernel: [ 8333.226493]  common_interrupt+0x89/0x89

              May 30 21:39:07 ubuntu kernel: [ 8333.226494] RIP: 0010:cpuidle_enter_state+0x12b/0x2d0

              May 30 21:39:07 ubuntu kernel: [ 8333.226494] RSP: 0018:ffffbe2843267e68 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff78

              May 30 21:39:07 ubuntu kernel: [ 8333.226495] RAX: 0000000000000000 RBX: 0000000000000002 RCX: 000000000000001f

              May 30 21:39:07 ubuntu kernel: [ 8333.226495] RDX: 0000000000000000 RSI: ffff97a53e8589d8 RDI: 0000000000000000

              May 30 21:39:07 ubuntu kernel: [ 8333.226496] RBP: ffffbe2843267ea0 R08: ffff97a53e8581a4 R09: 0000000000000008

              May 30 21:39:07 ubuntu kernel: [ 8333.226496] R10: ffffbe2843267e38 R11: 000000000000de3e R12: 0000000000000002

              ay 30 21:39:07 ubuntu kernel: [ 8333.226496] R10: ffffbe2843267e38 R11: 000000000000de3e R12: 0000000000000002

              May 30 21:39:07 ubuntu kernel: [ 8333.226497] R13: ffff97a53180a000 R14: ffffffff908f8d18 R15: 00000786302eb194

              May 30 21:39:07 ubuntu kernel: [ 8333.226497]  </IRQ>

              May 30 21:39:07 ubuntu kernel: [ 8333.226498]  ? cpuidle_enter_state+0x119/0x2d0

              May 30 21:39:07 ubuntu kernel: [ 8333.226498]  cpuidle_enter+0x17/0x20

              May 30 21:39:07 ubuntu kernel: [ 8333.226498]  call_cpuidle+0x23/0x40

              May 30 21:39:07 ubuntu kernel: [ 8333.226499]  do_idle+0x17f/0x1f0

              May 30 21:39:07 ubuntu kernel: [ 8333.226499]  cpu_startup_entry+0x71/0x80

              May 30 21:39:07 ubuntu kernel: [ 8333.226500]  start_secondary+0x154/0x190

              May 30 21:39:07 ubuntu kernel: [ 8333.226500]  start_cpu+0x14/0x14

              May 30 21:39:07 ubuntu kernel: [ 8333.226500] Code: 41 39 c0 74 e6 4d 85 c9 c6 07 01 74 30 41 c7 41 08 01 00 00 00 e9 51 ff ff ff 83 fa 01 0f 84 af fe ff ff 8b 07 84 c0 74 08 f3 90 <8b> 07 84 c0 75 f8 b8 01 00 00 00 66 89 07 5d c3 f3 90 4c 8b 09

              May 30 21:39:07 ubuntu kernel: [ 8333.226512] NMI backtrace for cpu 9

              May 30 21:39:07 ubuntu kernel: [ 8333.226512] CPU: 9 PID: 0 Comm: swapper/9 Not tainted 4.11.0 #1

              May 30 21:39:07 ubuntu kernel: [ 8333.226513] Hardware name: Gigabyte Technology Co., Ltd. Default string/AB350-Gaming 3-CF, BIOS F5 02/24/2017

              May 30 21:39:07 ubuntu kernel: [ 8333.226513] task: ffff97a53aaa2d00 task.stack: ffffbe2843264000

              May 30 21:39:07 ubuntu kernel: [ 8333.226514] RIP: 0010:native_queued_spin_lock_slowpath+0x17a/0x1a0

              May 30 21:39:07 ubuntu kernel: [ 8333.226514] RSP: 0018:ffff97a53e843a98 EFLAGS: 00000002

              May 30 21:39:07 ubuntu kernel: [ 8333.226515] RAX: 0000000000000101 RBX: 0000000000000086 RCX: 0000000000000001

              May 30 21:39:07 ubuntu kernel: [ 8333.226516] RDX: 0000000000000101 RSI: 0000000000000001 RDI: ffff97a531857d08

              May 30 21:39:07 ubuntu kernel: [ 8333.226516] RBP: ffff97a53e843a98 R08: 0000000000000101 R09: 000000000000003a

              May 30 21:39:07 ubuntu kernel: [ 8333.226517] R10: 00000000ffffffff R11: 0000000000000000 R12: ffff97a531857d08

              May 30 21:39:07 ubuntu kernel: [ 8333.226517] R13: ffff97a5321e2d00 R14: ffff97a531857c00 R15: ffff97a531857c00

              May 30 21:39:07 ubuntu kernel: [ 8333.226518] FS:  0000000000000000(0000) GS:ffff97a53e840000(0000) knlGS:0000000000000000

              May 30 21:39:07 ubuntu kernel: [ 8333.226518] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

              May 30 21:39:07 ubuntu kernel: [ 8333.226519] CR2: 00007f2909496624 CR3: 00000007f9337000 CR4: 00000000003406e0

              May 30 21:39:07 ubuntu kernel: [ 8333.226519] Call Trace:

              May 30 21:39:07 ubuntu kernel: [ 8333.226519]  <IRQ>