Bug 96419

Looking at the logs for APL (from bug 86767), I am noticing that this looks like same root cause than also for bug 96229 (checked logs on bug 86767 and bug 96229)
=> vblank not available on crtc 0, ret=-22
which btw appeared in bug 89108 and was fixed on previous platforms with 

commit f9b61ff6bce9a44555324b29e593fdffc9a115bc
Author: Daniel Vetter <daniel.vetter@ffwll.ch>
Date:   Wed Jan 7 13:54:39 2015 +0100

    drm/i915: Push vblank enable/disable past encoder->enable/disable

now is back.

We encountered that issue running some kms_rotation_crc igt subtests or some  kms_flip subtests

(In reply to yann from comment #1)
...
> We encountered that issue running some kms_rotation_crc igt subtests or some
> kms_flip subtests

I've tried running these tests, but they don't reproduce it for me unfortunately (on APL).  The original reporter was using 3 displays; is that the same setup you're using to trigger this with IGT?

Can you trigger this WARN() with 100% reproducibility with any specific IGT subtest or is it just something that just shows up randomly at different points while running the tests?

Can you provide the contents of /sys/kernel/debug/dri/0/i915_display_info to help clarify exactly what your display configuration is?

Direx, please attach dmesg from boot to the problem, with drm.debug=14 module parameter set. If you can't reproduce it easily, please attach the dmesg on a normal boot anyway, it'll give us some info on your system.

Created attachment 124574 [details]
dmesg with drm.debug=14

I have attached a dmesg, but unfortunately I was not able to reproduce the issue, although I am still using the same kernel. The first time the message appeared was when I tried out the KDE desktop (maybe OpenGL 3.0 desktop effects?).

I also ran the IGT tests, but this particular error did not appear (many others did, though).

Notes:
- I got two other traces instead this time (should I open a new bug for this?)
- I am getting the usual "FIFO underrun" errors, which cause flickering. You should already be aware of those errors (Bug 94605, Bug 94597, ...)

Created attachment 124575 [details]
display_info

Created attachment 124615 [details]
dmesg with drm.debug=14 and various WARN messages

I was able to reproduce it this time. It was triggered by the kms_rotation_crc test.

I can reproduce this on a SKL NUC with DP + HDMI when running the kms_plane tests on -nightly from today (2016-06-23). About half of the tests fail consistently. Running with just DP or HDMI works fine, which I find a bit odd if it's just a matter of waiting for vblank before the pipe is fully on. I'll look into this some more.

Any news on this? I just ran into this again on 4.7.0-rc6 with drm-intel-next. I was only doing desktop stuff and it made X freeze once again.

So I am pretty sure that this will hit less experienced users in the near future, if this does not get fixed.

*** Bug 96701 has been marked as a duplicate of this bug. ***

Christoph, I believe that is not a duplicate of this bug. Seems to be unrelated.

Direx, I don't think this warning is directly related to the X freeze that you're seeing. I'm seeing fifo underruns on my system so I suspect that is a more likely cause. You also have a bunch of underruns in your dmesg so that is what we should be looking at.

(In reply to Patrik Jakobsson from comment #11)
> Direx, I don't think this warning is directly related to the X freeze that
> you're seeing. I'm seeing fifo underruns on my system so I suspect that is a
> more likely cause. You also have a bunch of underruns in your dmesg so that
> is what we should be looking at.

Well, I am always getting pipe underruns on my system, accompanied by short screen flickerring:

[  423.607669] [drm:intel_cpu_fifo_underrun_irq_handler [i915]] *ERROR* CPU pipe A FIFO underrun
[10258.835183] ------------[ cut here ]------------
[10258.835207] WARNING: CPU: 1 PID: 455 at drivers/gpu/drm/drm_irq.c:1218 drm_wait_one_vblank+0x17b/0x1c0 [drm]
[10258.835209] vblank not available on crtc 0, ret=-22
[10258.835265] Modules linked in: nls_iso8859_1 nls_cp437 vfat fat ctr ccm snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic nf_conntrack_ipv6 nf_defrag_ipv6 ipt_REJECT nf_reject_ipv4 xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ip6table_filter ip6_tables iptable_filter arc4 iTCO_wdt iTCO_vendor_support iwlmvm mac80211 uvcvideo videobuf2_vmalloc videobuf2_memops x86_pkg_temp_thermal videobuf2_v4l2 intel_powerclamp coretemp snd_soc_skl snd_soc_skl_ipc videobuf2_core iwlwifi e1000e i915 kvm snd_hda_ext_core snd_soc_sst_ipc irqbypass snd_soc_sst_dsp videodev snd_soc_sst_match snd_soc_core joydev input_leds btusb media btrtl snd_hda_intel btbcm btintel snd_hda_codec bluetooth serio_raw intel_gtt psmouse ptp pcspkr snd_hwdep crc16 snd_hda_core thinkpad_acpi snd_pcm pps_core
[10258.835317]  i2c_algo_bit snd_timer cfg80211 i2c_i801 nvram drm_kms_helper evdev mousedev rtsx_pci_ms mac_hid syscopyarea sysfillrect memstick sysimgblt mei_me snd shpchp fb_sys_fops mei drm intel_pch_thermal thermal soundcore led_class wmi rfkill battery ac video tpm_crb button tpm_tis tpm sch_fq_codel ip_tables x_tables btrfs xor raid6_pq algif_skcipher af_alg hid_logitech_hidpp hid_logitech_dj usbhid dm_crypt dm_mod sd_mod rtsx_pci_sdmmc mmc_core atkbd libps2 crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ahci libahci libata rtsx_pci xhci_pci scsi_mod xhci_hcd i8042 serio hid_microsoft hid jitterentropy_rng sha256_ssse3 sha256_generic hmac drbg ansi_cprng
[10258.835322] CPU: 1 PID: 455 Comm: Xorg Not tainted 4.7.0-rc6-git+ #10
[10258.835324] Hardware name: LENOVO 20FVS01500/20FVS01500, BIOS R08ET41W (1.15 ) 05/11/2016
[10258.835330]  0000000000000286 00000000d546d760 ffff8804244e3610 ffffffff81308192
[10258.835333]  ffff8804244e3660 0000000000000000 ffff8804244e3650 ffffffff8109a8bb
[10258.835337]  000004c2244e3618 ffff88007f872000 0000000000000000 ffff88042e30c000
[10258.835338] Call Trace:
[10258.835347]  [<ffffffff81308192>] dump_stack+0x63/0x81
[10258.835352]  [<ffffffff8109a8bb>] __warn+0xcb/0xf0
[10258.835357]  [<ffffffff8109a93f>] warn_slowpath_fmt+0x5f/0x80
[10258.835370]  [<ffffffffa035f609>] ? drm_vblank_get+0x79/0xd0 [drm]
[10258.835381]  [<ffffffffa035f86b>] drm_wait_one_vblank+0x17b/0x1c0 [drm]
[10258.835418]  [<ffffffffa085eb30>] ? gen9_write16+0x3d0/0x3d0 [i915]
[10258.835440]  [<ffffffffa080eb21>] skl_update_wm+0x421/0x560 [i915]
[10258.835462]  [<ffffffffa0814ebe>] intel_update_watermarks+0x1e/0x20 [i915]
[10258.835495]  [<ffffffffa0884a92>] haswell_crtc_enable+0x7b2/0x930 [i915]
[10258.835528]  [<ffffffffa0880a3f>] intel_atomic_commit_tail+0x8ef/0x1160 [i915]
[10258.835561]  [<ffffffffa0888356>] ? intel_prepare_plane_fb+0x106/0x2b0 [i915]
[10258.835590]  [<ffffffffa0881694>] intel_atomic_commit+0x3e4/0x4f0 [i915]
[10258.835602]  [<ffffffffa0378517>] ? drm_atomic_check_only+0x187/0x610 [drm]
[10258.835613]  [<ffffffffa0378ea7>] ? drm_atomic_set_crtc_for_connector+0x97/0x100 [drm]
[10258.835624]  [<ffffffffa03789d7>] drm_atomic_commit+0x37/0x60 [drm]
[10258.835631]  [<ffffffffa03b0a3c>] restore_fbdev_mode+0x14c/0x270 [drm_kms_helper]
[10258.835638]  [<ffffffffa03b1e74>] drm_fb_helper_restore_fbdev_mode_unlocked+0x34/0x80 [drm_kms_helper]
[10258.835644]  [<ffffffffa03b1eed>] drm_fb_helper_set_par+0x2d/0x50 [drm_kms_helper]
[10258.835673]  [<ffffffffa089bd4a>] intel_fbdev_set_par+0x1a/0x60 [i915]
[10258.835680]  [<ffffffff8137f99f>] ? fb_set_var+0x30f/0x450
[10258.835685]  [<ffffffff8137f8e0>] fb_set_var+0x250/0x450
[10258.835690]  [<ffffffff810e39a6>] ? cpuacct_charge+0x86/0xa0
[10258.835693]  [<ffffffff810ce8f5>] ? update_curr+0xe5/0x1a0
[10258.835696]  [<ffffffff810cbe3d>] ? set_next_entity+0x4d/0x920
[10258.835700]  [<ffffffff810d09e4>] ? put_prev_entity+0xb4/0x8c0
[10258.835704]  [<ffffffff81038e8e>] ? __switch_to_xtra+0x12e/0x160
[10258.835709]  [<ffffffff81376493>] fbcon_blank+0x363/0x3b0
[10258.835716]  [<ffffffff813fa5a2>] do_unblank_screen+0xc2/0x190
[10258.835722]  [<ffffffff813f05a9>] complete_change_console+0x59/0xe0
[10258.835727]  [<ffffffff813f0d39>] vt_ioctl+0x709/0x12d0
[10258.835732]  [<ffffffff811c2c1c>] ? do_wp_page+0x12c/0x710
[10258.835745]  [<ffffffffa035d9ea>] ? drm_ioctl+0x18a/0x540 [drm]
[10258.835748]  [<ffffffff813e5835>] tty_ioctl+0x365/0xc70
[10258.835754]  [<ffffffff811c6871>] ? handle_mm_fault+0xa81/0x1fa0
[10258.835759]  [<ffffffff8129324b>] ? kernfs_fop_write+0x8b/0x1a0
[10258.835763]  [<ffffffff81214e87>] ? __vfs_write+0x37/0x140
[10258.835767]  [<ffffffff81228653>] do_vfs_ioctl+0xa3/0x5d0
[10258.835772]  [<ffffffff81233547>] ? __fget+0x77/0xb0
[10258.835775]  [<ffffffff81228bf9>] SyS_ioctl+0x79/0x90
[10258.835781]  [<ffffffff815f26f2>] entry_SYSCALL_64_fastpath+0x1a/0xa4
[10258.835819] ---[ end trace 542fab9ae0a0feb8 ]---

But as you can see from the stacktrace the FIFO underrun occurs much earlier. The X freeze however happens at the same time as the drm_wait_one_vblank error.

With X freeze I mean that the X server becomes unresponsive. The mouse can still be moved around, but the rest of the screen does not update any more. Whenever this happens a normal reboot helps (which means no hard kernel freezes, at least).

Yes, the underruns might not be the problem either but I suspect the warning you get is from when X has already crashed (see the vt switch). That makes me think the warning is not what is freezing X. Just that it pops up when system is trying to recover. Can you post the Xorg.log for when this happens?

removing APL since this different and tracked on original bug 96701

I do not encounter X freezing, but the "vblank not available on crtc 0" shows up on startup using drm-intel-nightly from 07/29/2016. I don't know if this is related or not, but I also encounter mouse cursor issues in Gnome: the cursor leaves trailing marks when moving to the notification bar and flickers when moving over controls. I do not observe these problems using 4.7 rc2 built from the same branch on 06/13/2016. I had the same problems with rc6 and rc7, but was hoping it may be fixed.

The cursor also disappears when mousing over chrome tab headers. This makes it fairly difficult to use.

Here's an example of what the cursor artifacts look like: https://goo.gl/photos/MbwQmsjH9fySdqPX9

Let me know what, if any other, debugging information may be useful to reproduce this.

This is on Skylake on Dell XPS 13. The cursor artifacts only show up when I connect an external monitor (Dell p2715q). I'm using DP to connect the monitor -- I don't know if that makes a difference.

I suspect this issue is similar to the "FIFO underrun" errors claimed to fixed by this patch series from Lyude:

https://lists.freedesktop.org/archives/intel-gfx/2016-August/102349.html

Has anyone tried that fix?

Assigned to me to check https://lists.freedesktop.org/archives/intel-gfx/2016-August/102349.html

Created attachment 125877 [details]
SKL-kms_rotation_crc-patch-104497-kern.log

By executing IGT test kms_rotation_crc, I'm reproducing:
- vblank not available on crtc 0, ret=-22
- ERROR CPU pipe A FIFO underrun
- a GPU hang
I confirm it needs 2 external screens to reproduce it.

I applied version 12: https://lists.freedesktop.org/archives/intel-gfx/2016-August/104497.html
I now see:
[  150.340992] [drm:verify_wm_state.isra.103 [i915]] *ERROR* mismatch in DDB state pipe A plane 2 (expected (0,0), found (430,860))
[  155.163585] WARNING: CPU: 3 PID: 3758 at drivers/gpu/drm/i915/intel_display.c:3249 intel_fb_gtt_offset+0x6d/0x80 [i915]
[  155.163588] ggtt vma for display object not found! (view=1)
[  155.194432] [drm:gen8_irq_handler [i915]] *ERROR* Fault errors on pipe A
[  164.832451] [drm] GPU crash dump saved to /sys/class/drm/card0/error

Platform: NUC6i3SYB
CPU: Intel(R) Core(TM) i3-6100U CPU @ 2.30GHZ (family 6, model 78, stepping 3)
Motherboard version: H81132-502
GPU: Intel® HD Graphics 520 - Intel Corporation Sky Lake Integrated Graphics (rev 07)
Memory: one 8GB card Kingston KVR21S15D8/8
SSD: Samsung 850 EVO M.2 120 Go

Software
Bios: SYSKLi35.86A.0045.2016.0527.1055 from https://downloadcenter.intel.com/downloads/eula/26097/BIOS-Update-SYSKLi35-86A-?httpDown=https%3A%2F%2Fdownloadmirror.intel.com%2F26097%2Feng%2FSY0045.bio
Linux distribution: Ubuntu 16.04 64 bits
Kernel: 4.8.0-rc2 e4ec788 from http://cgit.freedesktop.org/drm-intel/
  commit e4ec788166b13fc8b5b7e188c522d4cf07ba53eb
  Author: Sumit Semwal <sumit.semwal@linaro.org>
  Date:   Tue Aug 16 12:36:30 2016 +0530
  drm-intel-nightly: 2016y-08m-16d-07h-06m-09s UTC integration manifest
  with https://lists.freedesktop.org/archives/intel-gfx/2016-August/104497.html and https://patchwork.freedesktop.org/series/11157/ applied
libdrm-2.4.70-2 b214b05 from git://anongit.freedesktop.org/mesa/drm
mesa: mesa-11.2.2 3a9f628from git://anongit.freedesktop.org/mesa/mesa
cairo 1.15.2 db8a7f1 from git://anongit.freedesktop.org/cairo
xorg-server-1.18.0-525 de5291c from git://git.freedesktop.org/git/xorg/xserver
xf86-video-intel 2.99.917-695 c8fc7f5 from git://git.freedesktop.org/git/xorg/driver/xf86-video-intel
libva-1.7.0-45 b27feb9 from git://git.freedesktop.org/git/vaapi/libva 
vaapi-intel-driver: 1.7.0-82 98555d2 from git://git.freedesktop.org/git/vaapi/intel-driver
DMC 1.26 from https://01.org/sites/default/files/downloads/intelr-graphics-linux/skldmcver126.tar_1.bz2
GUC 6.1 from https://01.org/sites/default/files/downloads/intelr-graphics-linux/sklgucver61.tar.bz2
Intel-Gpu-Tools 1.15 a147ef2 from http://anongit.freedesktop.org/git/xorg/app/intel-gpu-tools.git

External screens: DP: LG 23MB35PYI, HDMI: CTL IP2152

I don't know if it's worth mentioning, but this bug has now made it's way from drm-intel-nightly into kernel 4.8-rc2 and possibly the next stable Kernel release, if it does not get fixed.

BTW: Xorg.log looks completely normal for me. No more log messages after X has completely started (except from VT switches).

Is there anybody looking into it, since it can be reproduced easily with multiple screens?

Created attachment 126665 [details]
dmesg

By executing IGT test kms_rotation_crc the FIFO underrun error is not reproduced with last setup but we have another bug:

Setup:
======
Hardware
Platform: 
CPU :  Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz (family: 6, model: 94  stepping: 3)
Software 
Linux OS : Ubuntu 16.10 64 bits
Kernel: drm-intel-nightly: 2016y-09m-19d-20h-40m-51s UTC integration manifest
        author: Daniel Vetter <daniel.vetter@ffwll.ch>
        commit: 4c518aef024daa0223692124baa2d7399f54dd97
drm: libdrm-2.4.70-14-g0659558 from http://cgit.freedesktop.org/mesa/drm/
xorg-server-1.18.99.2 from git://git.freedesktop.org/git/xorg/xserver
mesa:  mesa-12.0.0  78b061 from http://cgit.freedesktop.org/mesa/mesa/
cairo: tag 1.15.2 db8a7f1 from http://cgit.freedesktop.org/cairo
libva: libva-1.7.0-50-g7aa2dd9 from http://cgit.freedesktop.org/libva/
vaapi-intel-driver: 1.7.0-136-g36fbd81 from http://cgit.freedesktop.org/vaapi/intel-driver
intel-gpu-tool: intel-gpu-tools-1.16-22-g200237a

I cannot reproduce the issue I originally reported with 4.8-rc7, so this looks good now (for whatever reason).

The issues that Rami found should still be looked into (maybe open a new bug for this?).

New bug is tracked by https://bugs.freedesktop.org/show_bug.cgi?id=97944

Thanks Rami.
So closed