[vk, renderdoc] fixed zero-sized streambuffer bug on old heap-absent GPUs (#4052)

This fix a bug in which eden crashes when renderdoc is attached to vulkan.
that kept me away from renderdoc for around a year now.

the bug:

in video_core\renderer_vulkan\vk_staging_buffer_pool.cpp
in GetStreamBufferSize(...)
if device.HasDebuggingToolAttached()
but heap is empty/unavailable <-- Case in my old nvidia kepler gpu

the original method returns size 0, right?
the change honors same original behavior, while covers my case properly, returning MAX_STREAM_BUFFER_SIZE.

addl some log tip and some minimal doc. fully safe. let it rip.

Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/4052
Reviewed-by: MaranBr <maranbr@eden-emu.dev>
Reviewed-by: Lizzie <lizzie@eden-emu.dev>

docs/Debug.md

@@ -69,29 +69,6 @@ Expressions can be `variable_names` or `1234` (numbers) or `*var` (dereference o
 
 For more information type `info gdb` and read [the man page](https://man7.org/linux/man-pages/man1/gdb.1.html).
 
-## Simple checklist for debugging black screens using Renderdoc
-
-Renderdoc is a free, cross platform, multi-graphics API debugger. It is an invaluable tool for diagnosing issues with graphics applications, and includes support for Vulkan. Get it at [renderdoc.org](https://renderdoc.org).
-
-Before using renderdoc to diagnose issues, it is always good to make sure there are no validation errors. Any errors means the behavior of the application is undefined. That said, renderdoc can help debug validation errors if you do have them.
-
-When debugging a black screen, there are many ways the application could have setup Vulkan wrong.
-Here is a short checklist of items to look at to make sure are appropriate:
-
-- Draw call counts are correct (aka not zero, or if rendering many triangles, not 3)
-- Vertex buffers are bound
-- vertex attributes are correct - Make sure the size & offset of each attribute matches what should it should be
-- Any bound push constants and descriptors have the right data - including:
-  - Matrices have correct values - double check the model, view, & projection matrices are uploaded correctly
-- Pipeline state is correct
-  - viewport range is correct - x,y are 0,0; width & height are screen dimensions, minDepth is 0, maxDepth is 1, NDCDepthRange is 0,1
-  - Fill mode matches expected - usually solid
-  - Culling mode makes sense - commonly back or none
-  - The winding direction is correct - typically CCW (counter clockwise)
-  - Scissor region is correct - usually same as viewport's x,y,width, &height
-- Blend state is correct
-- Depth state is correct - typically enabled with Function set to Less than or Equal
-- Swapchain images are bound when rendering to the swapchain
-- Image being rendered to is the same as the one being presented when rendering to the swapchain
-
-Alternatively, a [RenderDoc Extension](https://github.com/baldurk/renderdoc-contrib/tree/main/baldurk/whereismydraw) ([Archive](https://web.archive.org/web/20250000000000*/https://github.com/baldurk/renderdoc-contrib/tree/main/baldurk/whereismydraw)) exists which automates doing a lot of these manual steps.
+# RenderDoc (Graphic Debugging Tool)
+
+Guidelines for graphical debugging using RenderDoc: **[RenderDoc usage](./RenderDoc.md)**

docs/README.md

@@ -12,6 +12,7 @@ This contains documentation created by developers. This contains build instructi
 - **[Development Guidelines](./Development.md)**
 - **[Dependencies](./Deps.md)**
 - **[Debug Guidelines](./Debug.md)**
+- **[RenderDoc usage](./RenderDoc.md)**
 - **[CPM - CMake Package Manager](./CPMUtil)**
 - **[Platform-Specific Caveats](./Caveats.md)**
 - **[The NVIDIA SM86 (Maxwell) GPU](./NvidiaGpu.md)**

docs/RenderDoc.md

@@ -0,0 +1,52 @@
+# RenderDoc
+
+Renderdoc is a free, cross platform, multi-graphics API debugger. It is an invaluable tool for diagnosing issues with graphics applications, and includes support for Vulkan. Get it at [renderdoc.org](https://renderdoc.org).
+
+RenderDoc can capture Eden's Vulkan output when its Vulkan layer is loaded before Eden creates the Vulkan device. Before using renderdoc to diagnose issues, it is always good to make sure there are no validation errors. Any errors means the behavior of the application is undefined. That said, renderdoc can help debug validation errors if you do have them.
+
+## Usage on Windows 
+
+You can either use RenderDoc UI to launch eden, or you can make eden attach it internally:
+
+On Windows PowerShell:
+```powershell
+$env:ENABLE_VULKAN_RENDERDOC_CAPTURE='1'
+.\eden.exe
+```
+When RenderDoc is attached, Eden logs the default Windows capture folder:
+```text
+%LOCALAPPDATA%\Temp\RenderDoc
+```
+
+Press RenderDoc's capture hotkey, usually `F12`, to capture a frame. To stop using RenderDoc, close Eden and launch it again without `ENABLE_VULKAN_RENDERDOC_CAPTURE`.
+
+## Eden Hotkey
+
+Eden also has a separate `Toggle Renderdoc Capture` hotkey behind the debug setting `renderdoc_hotkey`.
+That hotkey does not load or unload RenderDoc. It only toggles Eden's own manual capture through RenderDoc's API:
+
+- first press: starts a capture
+- second press: ends that capture
+
+## Simple checklist for debugging black screens using Renderdoc
+
+When debugging a black screen, there are many ways the application could have setup Vulkan wrong.
+Here is a short checklist of items to look at to make sure are appropriate:
+
+- Draw call counts are correct (aka not zero, or if rendering many triangles, not 3)
+- Vertex buffers are bound
+- vertex attributes are correct - Make sure the size & offset of each attribute matches what should it should be
+- Any bound push constants and descriptors have the right data - including:
+  - Matrices have correct values - double check the model, view, & projection matrices are uploaded correctly
+- Pipeline state is correct
+  - viewport range is correct - x,y are 0,0; width & height are screen dimensions, minDepth is 0, maxDepth is 1, NDCDepthRange is 0,1
+  - Fill mode matches expected - usually solid
+  - Culling mode makes sense - commonly back or none
+  - The winding direction is correct - typically CCW (counter clockwise)
+  - Scissor region is correct - usually same as viewport's x,y,width, &height
+- Blend state is correct
+- Depth state is correct - typically enabled with Function set to Less than or Equal
+- Swapchain images are bound when rendering to the swapchain
+- Image being rendered to is the same as the one being presented when rendering to the swapchain
+
+Alternatively, a [RenderDoc Extension](https://github.com/baldurk/renderdoc-contrib/tree/main/baldurk/whereismydraw) ([Archive](https://web.archive.org/web/20250000000000*/https://github.com/baldurk/renderdoc-contrib/tree/main/baldurk/whereismydraw)) exists which automates doing a lot of these manual steps.

src/video_core/renderer_vulkan/vk_staging_buffer_pool.cpp

@@ -1,4 +1,4 @@
-// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
+// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
 // SPDX-License-Identifier: GPL-3.0-or-later
 
 // SPDX-FileCopyrightText: Copyright 2022 yuzu Emulator Project
@@ -31,11 +31,18 @@ constexpr VkDeviceSize MAX_ALIGNMENT = 256;
 constexpr VkDeviceSize MAX_STREAM_BUFFER_SIZE = 128_MiB;
 
 size_t GetStreamBufferSize(const Device& device) {
+    if (!device.HasDebuggingToolAttached()) {
+        return MAX_STREAM_BUFFER_SIZE;
+    }
+
     VkDeviceSize size{0};
-    if (device.HasDebuggingToolAttached()) {
-        ForEachDeviceLocalHostVisibleHeap(device, [&size](size_t index, VkMemoryHeap& heap) {
-            size = (std::max)(size, heap.size);
-        });
+    bool has_device_local_host_visible_heap{};
+    ForEachDeviceLocalHostVisibleHeap(device, [&size, &has_device_local_host_visible_heap](
+                                                  size_t index, VkMemoryHeap& heap) {
+        has_device_local_host_visible_heap = true;
+        size = (std::max)(size, heap.size);
+    });
+    if (has_device_local_host_visible_heap) {
         // If rebar is not supported, cut the max heap size to 40%. This will allow 2 captures to be
         // loaded at the same time in RenderDoc. If rebar is supported, this shouldn't be an issue
         // as the heap will be much larger.

src/video_core/vulkan_common/vulkan_device.cpp

@@ -1481,6 +1481,12 @@ void Device::CollectToolingInfo() {
         has_nsight_graphics = has_nsight_graphics || name == "NVIDIA Nsight Graphics";
         has_radeon_gpu_profiler = has_radeon_gpu_profiler || name == "Radeon GPU Profiler";
     }
+#ifdef _WIN32
+    if (has_renderdoc) {
+        LOG_INFO(Render_Vulkan,
+                 "Windows default RenderDoc output folder: %LOCALAPPDATA%\\Temp\\RenderDoc");
+    }
+#endif
 }
 
 std::vector<VkDeviceQueueCreateInfo> Device::GetDeviceQueueCreateInfos() const {