gfx/vulkan: fix VkDeviceMemory leak + spec violations in create/destroy paths

Four issues found in the same audit pass over gfx/drivers/vulkan.c, all
in the create-with-old / destroy-buffer ownership protocol.  None of
them are the trigger for the iOS materialui content-close abort that
prompted the audit (the font teardown does not go through the patched
paths), but they are real and worth fixing on their own.

* vulkan_create_texture: VkDeviceMemory leak on alloc failure
  (gfx/drivers/vulkan.c)

  When called with a non-NULL `old`, the function destroys old's
  view/image/buffer at the top of the `if (old)` block, then either
  pilfers old->memory or allocates fresh memory for tex.memory.  The
  trailing `if (old)` cleanup is responsible for freeing old->memory in
  the no-pilfer path (skipped by the pilfer branch via
  old->memory = VK_NULL_HANDLE).

  The vkAllocateMemory failure branch returns early before reaching
  that cleanup.  At return, old->view/image/buffer are dangling but
  zeroed by the caller's `*old = tex` overwrite, while old->memory
  still holds the original live VkDeviceMemory and is now unreachable.
  Pure leak per failed call; reachable from every call site that
  passes a real `old` (vk_per_frame texture realloc on resize, MUI/RGUI
  set_texture, readback staging realloc).

  Inline an equivalent cleanup before the early return: unmap if
  mapped, free old->memory, zero old.  Mirrors the unmap-before-free
  the pilfer branch already does on line 1277.

* vulkan_create_texture: free-while-mapped on no-pilfer cleanup
  (gfx/drivers/vulkan.c)

  The trailing `if (old)` cleanup was calling vkFreeMemory on
  old->memory without unmapping first, even though the pilfer branch a
  few lines up does unmap.  vkFreeMemory implicitly unmaps so this is
  not a spec violation, but MoltenVK has historically handled
  free-while-mapped poorly (KhronosGroup/MoltenVK#957) and the
  asymmetry with the pilfer branch is gratuitous.

  Add the `if (old->mapped) vkUnmapMemory(...)` guard before the free.

* vulkan_destroy_buffer: VUID-vkFreeMemory-memory-00677 violation
  (gfx/drivers/vulkan.c)

  Order was: vkUnmapMemory (unconditional) -> vkFreeMemory ->
  vkDestroyBuffer.  Two problems.  First, the memory is freed while
  buffer->buffer is still bound to it, which the spec forbids per
  VUID-vkFreeMemory-memory-00677 -- vulkan_destroy_texture (line 974)
  gets this right; this function was the outlier.  Second,
  vkUnmapMemory was unconditional, but vulkan_create_buffer leaves
  buffer->mapped == NULL when its initial vkMapMemory fails, so a
  destroy on a never-mapped buffer would unmap memory that is not
  currently mapped.

  Fix the order to unmap (gated on buffer->mapped) -> destroy buffer
  -> free memory, with NULL-handle guards on the destroys for symmetry
  with vulkan_destroy_texture.

* vulkan_init_quad_ibo: same VUID violation in vkMapMemory failure path
  (gfx/drivers/vulkan.c)

  Identical wrong order in the IBO-init map-failure cleanup -- frees
  vk->quad_ibo.memory before destroying vk->quad_ibo.buffer that is
  still bound to it.  Swap the two calls.  vulkan_deinit_quad_ibo
  itself was already correct.

Author: LibretroAdmin

gfx/drivers/vulkan.c

@@ -1289,6 +1289,20 @@ static struct vk_texture vulkan_create_texture(vk_t *vk,
             vkDestroyImage(device, tex.image, NULL);
          if (tex.buffer)
             vkDestroyBuffer(device, tex.buffer, NULL);
+         /* The `if (old)` cleanup further below is skipped by this
+          * early return, but old->view/image/buffer were already
+          * destroyed at the top of the `if (old)` block above and
+          * old->memory still owns a live VkDeviceMemory.  Free it
+          * here, otherwise it leaks across the call.  Mirrors the
+          * unmap-before-free done in the pilfer branch. */
+         if (old)
+         {
+            if (old->mapped)
+               vkUnmapMemory(device, old->memory);
+            if (old->memory != VK_NULL_HANDLE)
+               vkFreeMemory(device, old->memory, NULL);
+            memset(old, 0, sizeof(*old));
+         }
          memset(&tex, 0, sizeof(tex));
          return tex;
       }
@@ -1299,8 +1313,18 @@ static struct vk_texture vulkan_create_texture(vk_t *vk,
 
    if (old)
    {
+      /* old->view/image/buffer were already destroyed at the top of
+       * the `if (old)` block above.  In the no-pilfer path we did not
+       * take ownership of old->memory, so it still owns a live
+       * VkDeviceMemory; free it here.  Unmap first to mirror the
+       * pilfer branch and to avoid free-while-mapped, which the spec
+       * permits but MoltenVK has historically handled poorly. */
       if (old->memory != VK_NULL_HANDLE)
+      {
+         if (old->mapped)
+            vkUnmapMemory(device, old->memory);
          vkFreeMemory(device, old->memory, NULL);
+      }
       memset(old, 0, sizeof(*old));
    }
 
@@ -3867,10 +3891,20 @@ static void vulkan_init_samplers(vk_t *vk)
 
 static void vulkan_destroy_buffer(VkDevice device, struct vk_buffer *buffer)
 {
-   vkUnmapMemory(device, buffer->memory);
-   vkFreeMemory(device, buffer->memory, NULL);
+   /* Order: unmap (only if mapped) -> destroy buffer -> free memory.
+    * vkFreeMemory must not be called while a VkBuffer is still bound
+    * to the memory (VUID-vkFreeMemory-memory-00677), and vkUnmapMemory
+    * must not be called on memory that is not currently mapped.
+    * vulkan_create_buffer leaves buffer->mapped == NULL when the
+    * initial vkMapMemory fails, so the mapped check is required. */
+   if (buffer->mapped)
+      vkUnmapMemory(device, buffer->memory);
 
-   vkDestroyBuffer(device, buffer->buffer, NULL);
+   if (buffer->buffer != VK_NULL_HANDLE)
+      vkDestroyBuffer(device, buffer->buffer, NULL);
+
+   if (buffer->memory != VK_NULL_HANDLE)
+      vkFreeMemory(device, buffer->memory, NULL);
 
    memset(buffer, 0, sizeof(*buffer));
 }
@@ -4483,8 +4517,10 @@ static void vulkan_init_quad_ibo(vk_t *vk, unsigned max_quads)
    if (res != VK_SUCCESS || !mapped)
    {
       RARCH_ERR("[Vulkan] Failed to map quad IBO memory (VkResult: %d).\n", res);
-      vkFreeMemory(device, vk->quad_ibo.memory, NULL);
+      /* Destroy the buffer before freeing the memory it is bound to
+       * (VUID-vkFreeMemory-memory-00677). */
       vkDestroyBuffer(device, vk->quad_ibo.buffer, NULL);
+      vkFreeMemory(device, vk->quad_ibo.memory, NULL);
       vk->quad_ibo.buffer    = VK_NULL_HANDLE;
       vk->quad_ibo.memory    = VK_NULL_HANDLE;
       vk->quad_ibo.num_quads = 0;