6bd801
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
6bd801
From: Daniel Axtens <dja@axtens.net>
6bd801
Date: Wed, 15 Apr 2020 23:28:29 +1000
6bd801
Subject: [PATCH] ieee1275: claim more memory
6bd801
6bd801
On powerpc-ieee1275, we are running out of memory trying to verify
6bd801
anything. This is because:
6bd801
6bd801
 - we have to load an entire file into memory to verify it. This is
6bd801
   extremely difficult to change with appended signatures.
6bd801
 - We only have 32MB of heap.
6bd801
 - Distro kernels are now often around 30MB.
6bd801
6bd801
So we want to claim more memory from OpenFirmware for our heap.
6bd801
6bd801
There are some complications:
6bd801
6bd801
 - The grub mm code isn't the only thing that will make claims on
6bd801
   memory from OpenFirmware:
6bd801
6bd801
    * PFW/SLOF will have claimed some for their own use.
6bd801
6bd801
    * The ieee1275 loader will try to find other bits of memory that we
6bd801
      haven't claimed to place the kernel and initrd when we go to boot.
6bd801
6bd801
    * Once we load Linux, it will also try to claim memory. It claims
6bd801
      memory without any reference to /memory/available, it just starts
6bd801
      at min(top of RMO, 768MB) and works down. So we need to avoid this
6bd801
      area. See arch/powerpc/kernel/prom_init.c as of v5.11.
6bd801
6bd801
 - The smallest amount of memory a ppc64 KVM guest can have is 256MB.
6bd801
   It doesn't work with distro kernels but can work with custom kernels.
6bd801
   We should maintain support for that. (ppc32 can boot with even less,
6bd801
   and we shouldn't break that either.)
6bd801
6bd801
 - Even if a VM has more memory, the memory OpenFirmware makes available
6bd801
   as Real Memory Area can be restricted. A freshly created LPAR on a
6bd801
   PowerVM machine is likely to have only 256MB available to OpenFirmware
6bd801
   even if it has many gigabytes of memory allocated.
6bd801
6bd801
EFI systems will attempt to allocate 1/4th of the available memory,
6bd801
clamped to between 1M and 1600M. That seems like a good sort of
6bd801
approach, we just need to figure out if 1/4 is the right fraction
6bd801
for us.
6bd801
6bd801
We don't know in advance how big the kernel and initrd are going to be,
6bd801
which makes figuring out how much memory we can take a bit tricky.
6bd801
6bd801
To figure out how much memory we should leave unused, I looked at:
6bd801
6bd801
 - an Ubuntu 20.04.1 ppc64le pseries KVM guest:
6bd801
    vmlinux: ~30MB
6bd801
    initrd:  ~50MB
6bd801
6bd801
 - a RHEL8.2 ppc64le pseries KVM guest:
6bd801
    vmlinux: ~30MB
6bd801
    initrd:  ~30MB
6bd801
6bd801
Ubuntu VMs struggle to boot with just 256MB under SLOF.
6bd801
RHEL likewise has a higher minimum supported memory figure.
6bd801
So lets first consider a distro kernel and 512MB of addressible memory.
6bd801
(This is the default case for anything booting under PFW.) Say we lose
6bd801
131MB to PFW (based on some tests). This leaves us 381MB. 1/4 of 381MB
6bd801
is ~95MB. That should be enough to verify a 30MB vmlinux and should
6bd801
leave plenty of space to load Linux and the initrd.
6bd801
6bd801
If we consider 256MB of RMA under PFW, we have just 125MB remaining. 1/4
6bd801
of that is a smidge under 32MB, which gives us very poor odds of verifying
6bd801
a distro-sized kernel. However, if we need 80MB just to put the kernel
6bd801
and initrd in memory, we can't claim any more than 45MB anyway. So 1/4
6bd801
will do. We'll come back to this later.
6bd801
6bd801
grub is always built as a 32-bit binary, even if it's loading a ppc64
6bd801
kernel. So we can't address memory beyond 4GB. This gives a natural cap
6bd801
of 1GB for powerpc-ieee1275.
6bd801
6bd801
Also apply this 1/4 approach to i386-ieee1275, but keep the 32MB cap.
6bd801
6bd801
make check still works for both i386 and powerpc and I've booted
6bd801
powerpc grub with this change under SLOF and PFW.
6bd801
6bd801
Signed-off-by: Daniel Axtens <dja@axtens.net>
6bd801
---
6bd801
 grub-core/kern/ieee1275/init.c | 81 +++++++++++++++++++++++++++++++++---------
6bd801
 docs/grub-dev.texi             |  6 ++--
6bd801
 2 files changed, 69 insertions(+), 18 deletions(-)
6bd801
6bd801
diff --git a/grub-core/kern/ieee1275/init.c b/grub-core/kern/ieee1275/init.c
b71686
index ee97d761d..a6e169bd0 100644
6bd801
--- a/grub-core/kern/ieee1275/init.c
6bd801
+++ b/grub-core/kern/ieee1275/init.c
6bd801
@@ -42,11 +42,12 @@
6bd801
 #include <grub/machine/kernel.h>
6bd801
 #endif
6bd801
 
6bd801
-/* The maximum heap size we're going to claim */
6bd801
+/* The maximum heap size we're going to claim. Not used by sparc.
6bd801
+   We allocate 1/4 of the available memory under 4G, up to this limit. */
6bd801
 #ifdef __i386__
6bd801
 #define HEAP_MAX_SIZE		(unsigned long) (64 * 1024 * 1024)
6bd801
-#else
6bd801
-#define HEAP_MAX_SIZE		(unsigned long) (32 * 1024 * 1024)
6bd801
+#else // __powerpc__
6bd801
+#define HEAP_MAX_SIZE		(unsigned long) (1 * 1024 * 1024 * 1024)
6bd801
 #endif
6bd801
 
6bd801
 extern char _end[];
6bd801
@@ -143,16 +144,45 @@ grub_claim_heap (void)
6bd801
 				 + GRUB_KERNEL_MACHINE_STACK_SIZE), 0x200000);
6bd801
 }
6bd801
 #else
6bd801
-/* Helper for grub_claim_heap.  */
6bd801
+/* Helper for grub_claim_heap on powerpc. */
6bd801
+static int
6bd801
+heap_size (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type,
6bd801
+	   void *data)
6bd801
+{
6bd801
+  grub_uint32_t total = *(grub_uint32_t *)data;
6bd801
+
6bd801
+  if (type != GRUB_MEMORY_AVAILABLE)
6bd801
+    return 0;
6bd801
+
6bd801
+  /* Do not consider memory beyond 4GB */
6bd801
+  if (addr > 0xffffffffUL)
6bd801
+    return 0;
6bd801
+
6bd801
+  if (addr + len > 0xffffffffUL)
6bd801
+    len = 0xffffffffUL - addr;
6bd801
+
6bd801
+  total += len;
6bd801
+  *(grub_uint32_t *)data = total;
6bd801
+
6bd801
+  return 0;
6bd801
+}
6bd801
+
6bd801
 static int
6bd801
 heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type,
6bd801
 	   void *data)
6bd801
 {
6bd801
-  unsigned long *total = data;
6bd801
+  grub_uint32_t total = *(grub_uint32_t *)data;
6bd801
 
6bd801
   if (type != GRUB_MEMORY_AVAILABLE)
6bd801
     return 0;
6bd801
 
6bd801
+  /* Do not consider memory beyond 4GB */
6bd801
+  if (addr > 0xffffffffUL)
6bd801
+    return 0;
6bd801
+
6bd801
+  if (addr + len > 0xffffffffUL)
6bd801
+    len = 0xffffffffUL - addr;
6bd801
+
6bd801
   if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_NO_PRE1_5M_CLAIM))
6bd801
     {
6bd801
       if (addr + len <= 0x180000)
6bd801
@@ -166,10 +196,6 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type,
6bd801
     }
6bd801
   len -= 1; /* Required for some firmware.  */
6bd801
 
6bd801
-  /* Never exceed HEAP_MAX_SIZE  */
6bd801
-  if (*total + len > HEAP_MAX_SIZE)
6bd801
-    len = HEAP_MAX_SIZE - *total;
6bd801
-
6bd801
   /* In theory, firmware should already prevent this from happening by not
6bd801
      listing our own image in /memory/available.  The check below is intended
6bd801
      as a safeguard in case that doesn't happen.  However, it doesn't protect
6bd801
@@ -181,6 +207,18 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type,
6bd801
       len = 0;
6bd801
     }
6bd801
 
6bd801
+  /* If this block contains 0x30000000 (768MB), do not claim below that.
6bd801
+     Linux likes to claim memory at min(RMO top, 768MB) and works down
6bd801
+     without reference to /memory/available. */
6bd801
+  if ((addr < 0x30000000) && ((addr + len) > 0x30000000))
6bd801
+    {
6bd801
+      len = len - (0x30000000 - addr);
6bd801
+      addr = 0x30000000;
6bd801
+    }
6bd801
+
6bd801
+  if (len > total)
6bd801
+    len = total;
6bd801
+
6bd801
   if (len)
6bd801
     {
6bd801
       grub_err_t err;
6bd801
@@ -189,10 +227,12 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type,
6bd801
       if (err)
6bd801
 	return err;
6bd801
       grub_mm_init_region ((void *) (grub_addr_t) addr, len);
6bd801
+      total -= len;
6bd801
     }
6bd801
 
6bd801
-  *total += len;
6bd801
-  if (*total >= HEAP_MAX_SIZE)
6bd801
+  *(grub_uint32_t *)data = total;
6bd801
+
6bd801
+  if (total == 0)
6bd801
     return 1;
6bd801
 
6bd801
   return 0;
6bd801
@@ -201,13 +241,22 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type,
6bd801
 static void 
6bd801
 grub_claim_heap (void)
6bd801
 {
6bd801
-  unsigned long total = 0;
6bd801
+  grub_uint32_t total = 0;
6bd801
 
6bd801
   if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_FORCE_CLAIM))
6bd801
-    heap_init (GRUB_IEEE1275_STATIC_HEAP_START, GRUB_IEEE1275_STATIC_HEAP_LEN,
6bd801
-	       1, &total);
6bd801
-  else
6bd801
-    grub_machine_mmap_iterate (heap_init, &total);
6bd801
+    {
6bd801
+      heap_init (GRUB_IEEE1275_STATIC_HEAP_START, GRUB_IEEE1275_STATIC_HEAP_LEN,
6bd801
+		 1, &total);
6bd801
+      return;
6bd801
+    }
6bd801
+
6bd801
+  grub_machine_mmap_iterate (heap_size, &total);
6bd801
+
6bd801
+  total = total / 4;
6bd801
+  if (total > HEAP_MAX_SIZE)
6bd801
+    total = HEAP_MAX_SIZE;
6bd801
+
6bd801
+  grub_machine_mmap_iterate (heap_init, &total);
6bd801
 }
6bd801
 #endif
6bd801
 
6bd801
diff --git a/docs/grub-dev.texi b/docs/grub-dev.texi
b71686
index 421dd410e..03d53498c 100644
6bd801
--- a/docs/grub-dev.texi
6bd801
+++ b/docs/grub-dev.texi
6bd801
@@ -930,7 +930,9 @@ space is limited to 4GiB. GRUB allocates pages from EFI for its heap, at most
6bd801
 1.6 GiB.
6bd801
 
6bd801
 On i386-ieee1275 and powerpc-ieee1275 GRUB uses same stack as IEEE1275.
6bd801
-It allocates at most 32MiB for its heap.
6bd801
+
6bd801
+On i386-ieee1275, GRUB allocates at most 32MiB for its heap. On
6bd801
+powerpc-ieee1275, GRUB allocates up to 1GiB.
6bd801
 
6bd801
 On sparc64-ieee1275 stack is 256KiB and heap is 2MiB.
6bd801
 
6bd801
@@ -958,7 +960,7 @@ In short:
6bd801
 @item i386-qemu               @tab 60 KiB  @tab < 4 GiB
6bd801
 @item *-efi                   @tab ?       @tab < 1.6 GiB
6bd801
 @item i386-ieee1275           @tab ?       @tab < 32 MiB
6bd801
-@item powerpc-ieee1275        @tab ?       @tab < 32 MiB
6bd801
+@item powerpc-ieee1275        @tab ?       @tab < 1 GiB
6bd801
 @item sparc64-ieee1275        @tab 256KiB  @tab 2 MiB
6bd801
 @item arm-uboot               @tab 256KiB  @tab 2 MiB
6bd801
 @item mips(el)-qemu_mips      @tab 2MiB    @tab 253 MiB