22 Replies Latest reply on Oct 25, 2017 3:27 PM by nezarn

    Driver Crash\BSOD with Vulkan

    nezarn

      Hey,

       

      I'm getting a driver crash and sometimes a BSOD when i play Hatsune Miku Project Diva F in RPCS3 emulator.

       

      Here you can read some discussion: Vulkan: Project Diva F broken graphics (AMD specific) · Issue #2129 · RPCS3/rpcs3 · GitHub  (some time ago graphics were broken in a lot of games with Vulkan on AMD GCN3 a GCN4 cards (GCN2 and older cards that supports Vulkan always worked fine) and even with a workaround its not perfect, im just mentioning this because maybe in some way its related, but in that issue you can see some discussion about this BSOD\driver crash too)

       

      This driver crash\BSOD is 100% reproducable in this game (Hatsune Miku: Project Diva F), in the song called "Black★Rock Shooter".

       

      The file where the BSOD is coming from is ATIKMDAG.SYS

       

      Windows crashdump: Dropbox - 100416-17093-01.dmp

       

      PS.: I'm using the latest driver with an RX480. (and posting this issue, since rpcs3 devs told me getting a BSOD is a driver bug\issue as you can see in that github issue link)

        • Re: Driver Crash\BSOD with Vulkan
          Zephiris

          fwiw, I'm getting a similar issue using Vulkan with the Dolphin emulator on Super Smash Brothers Brawl, 100% of the time it loads stage 2 in "classic mode", RX 480, 16.10.1, and using stock settings (no OC).

           

          Not sure how to reproduce this more easily, but even if emulators or games are sending invalid commands somehow, obviously it shouldn't crash the entire driver.

            • Re: Driver Crash\BSOD with Vulkan
              nezarn

              IDK what did they change from GCN2 to 3 and 4, because its interesting that only GCN3 and GCN4 cards are affected...

               

              It would be nice if someone from AMD would reply.

               

               

              edit: also i get 100% GPU usage always in rpcs3 in Vulkan, while for others GPU usage is fine. (this 100% GPU usage can be easily reproduced by running one of the samples that are included in rpcs3)

            • Re: Driver Crash\BSOD with Vulkan
              dwitczak

              I could look into this, granted the following:

               

              1) You can provide clear instructions on how to reproduce the issue + app source code is available.

              2) There's no need to download any third-party content.

              3) No issues are reported by the latest version of the validation layers, taken straight from the project's repository.

               

              Edit: Reason behind all of these is that it doesn't take much for a misbehaving app to TDR a driver in Vulkan, with vast majority of the cases turning out to be app-side issues.

                • Re: Driver Crash\BSOD with Vulkan
                  nezarn

                  Well you need to have a PS3 and the game (so basically you need 3rd party content), the Vulkan source code is here: rpcs3/rpcs3/Emu/RSX/VK at master · RPCS3/rpcs3 · GitHub  feel free to take a look, and if you see anything wrong\missing, post it here Vulkan: multiple issues on newer AMD (GCN3+) cards · Issue #2201 · RPCS3/rpcs3 · GitHub

                   

                  But if you have a PS3 and the game, here's how to make the game run and reproduce the BSOD\driver crash:

                   

                  1. Download latest RPCS3 from here: AppVeyor

                  2. You need to get some firmware modules from your PS3 (or if you can find a pup file extractor, you can get them from the official PS3 firmware updates)

                  3. Use these settings: http://i.imgur.com/f4KO3ii.png (+ in graphics select Vulkan, and a resolution, 1280x720 or 1920x1080, frame limit auto, in audio Xaudio2, at input if you have an Xinput gamepad, then select Xinput)

                  4. Here is a savegame so the required song is unlocked Dropbox - NPUB31241_SYSTEM00.ZIP (unzip it into rpcs3\dev_hdd0\home\00000001\savedata\)

                  5. Run the game (if you have the disc version, select Boot->Boot Game and browse to the game's folder (you need to rip your disc), if you have the PSN version you need to back it up into a pkg and use Boot->Install Game and you need to place the .rap file (which is basically the license that you have the game) to rpcs3\dev_hdd0\home\00000001\exdata\ and then the game will be in the list, just simply doubleclick on it

                  6. In game select Load Game, then Play->Rhythm Game and select the song called "Black★Rock Shooter", then you can either play with start, or watch video, both will crash\BSOD at the same place after a while (and always)

                   

                   

                  edit: sometime later when i have time, i can post some renderdoc\perfstudio stuff, so at least you could help getting rid of the graphical issues that are present on GCN3+ cards

                    • Re: Driver Crash\BSOD with Vulkan
                      dwitczak

                      Sorry, no go, we won't be able to help if we are required to use third-party content to reproduce the issue.

                       

                      On the other hand, a RenderDoc-based repro is fine, that's something I can definitely try helping you out with.

                        • Re: Driver Crash\BSOD with Vulkan
                          nezarn

                          Here is a renderdoc capture from a game's main menu, on the bottom you can clearly see the corruption that happens (used 0.31 renderdoc)

                          Dropbox - graphicalissue.rdc

                           

                          Here is a video how it looks like: 2016 10 15 23 05 37 - YouTube  (if link doesn't contain the timestamp, go to 0:25, there you can see it)

                           

                          And btw i mentioned that the emulator always uses 100% gpu on newer AMD cards, you can easily reproduce that, just download latest rpcs3 version from my comment above, then after setting up the emulator to use Vulkan, you can boot an included sample from rpcs3\dev_hdd0\game\TEST12345\USRDIR\ (so you don't even need to have a PS3, since samples doesn't need any files from firmware)

                           

                          For example when running gs_gcm_hello_world.elf : RX480: http://i.imgur.com/Kg0hxul.png R9 290: http://i.imgur.com/wxEM8bU.png

                            • Re: Driver Crash\BSOD with Vulkan
                              dwitczak

                              It appears that the glitches are introduced in colour passes #7-#11, where the app appears to be doing a blur.

                               

                              The biggest problem that's jarring from the trace you provided is that your application is executing a lot of renderpasses without defining external dependencies. This can lead to corruptions as the ones we're seeing because of the fact the GPU is free to run the commands in an overlapping manner which may lead to RAW hazards.

                               

                              For performance reasons, also please consider coalescing the huge number of renderpasses your application is using right now, so that the draw calls are embedded in subpasses.

                              1 of 1 people found this helpful
                                • Re: Driver Crash\BSOD with Vulkan
                                  kd-11

                                  Hi, I added the following dependency to try and finish previous writes before they are read as textures

                                   

                                    dependencies[1].srcSubpass = VK_SUBPASS_EXTERNAL;

                                    dependencies[1].srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT;

                                    dependencies[1].srcStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT;

                                    dependencies[1].dstSubpass = 0;

                                    dependencies[1].dstAccessMask = VK_ACCESS_SHADER_READ_BIT;

                                    dependencies[1].dstStageMask = VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;

                                    dependencies[1].dependencyFlags = VK_DEPENDENCY_BY_REGION_BIT;

                                   

                                  It didn't help; my fault for not understanding the spec on the fields there I'm sure. There's no way for the renderer to know what the next draw command will be like or if the render target will even change and the current one used as input since it is an emulator, so I've been using one subpass and setting src to external.

                                  What's the correct way to set up this dependency?

                                    • Re: Driver Crash\BSOD with Vulkan
                                      dwitczak

                                      The BY_REGION flag doesn't make much sense for a single-subpass renderpass, please remove it. Also, if your application needs to sample DS image views, you should consider ORing srcStageMask with VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT.

                                       

                                      In general, please use validation layers to confirm your Vulkan back-end does everything OK.

                                  • Re: Driver Crash\BSOD with Vulkan
                                    dwitczak

                                    Oh, and reg. 100% GPU utilization: assuming you do not use any kind of a CPU-side-based frame limiting solution, that's absolutely fine. After all, wasn't the idea behind Vulkan to squeeze as much juice from the GPU as it's only possible?