Blame SOURCES/0199-ieee1275-claim-more-memory.patch

5593c8
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
5593c8
From: Daniel Axtens <dja@axtens.net>
5593c8
Date: Wed, 15 Apr 2020 23:28:29 +1000
5593c8
Subject: [PATCH] ieee1275: claim more memory
5593c8
5593c8
On powerpc-ieee1275, we are running out of memory trying to verify
5593c8
anything. This is because:
5593c8
5593c8
 - we have to load an entire file into memory to verify it. This is
5593c8
   extremely difficult to change with appended signatures.
5593c8
 - We only have 32MB of heap.
5593c8
 - Distro kernels are now often around 30MB.
5593c8
5593c8
So we want to claim more memory from OpenFirmware for our heap.
5593c8
5593c8
There are some complications:
5593c8
5593c8
 - The grub mm code isn't the only thing that will make claims on
5593c8
   memory from OpenFirmware:
5593c8
5593c8
    * PFW/SLOF will have claimed some for their own use.
5593c8
5593c8
    * The ieee1275 loader will try to find other bits of memory that we
5593c8
      haven't claimed to place the kernel and initrd when we go to boot.
5593c8
5593c8
    * Once we load Linux, it will also try to claim memory. It claims
5593c8
      memory without any reference to /memory/available, it just starts
5593c8
      at min(top of RMO, 768MB) and works down. So we need to avoid this
5593c8
      area. See arch/powerpc/kernel/prom_init.c as of v5.11.
5593c8
5593c8
 - The smallest amount of memory a ppc64 KVM guest can have is 256MB.
5593c8
   It doesn't work with distro kernels but can work with custom kernels.
5593c8
   We should maintain support for that. (ppc32 can boot with even less,
5593c8
   and we shouldn't break that either.)
5593c8
5593c8
 - Even if a VM has more memory, the memory OpenFirmware makes available
5593c8
   as Real Memory Area can be restricted. A freshly created LPAR on a
5593c8
   PowerVM machine is likely to have only 256MB available to OpenFirmware
5593c8
   even if it has many gigabytes of memory allocated.
5593c8
5593c8
EFI systems will attempt to allocate 1/4th of the available memory,
5593c8
clamped to between 1M and 1600M. That seems like a good sort of
5593c8
approach, we just need to figure out if 1/4 is the right fraction
5593c8
for us.
5593c8
5593c8
We don't know in advance how big the kernel and initrd are going to be,
5593c8
which makes figuring out how much memory we can take a bit tricky.
5593c8
5593c8
To figure out how much memory we should leave unused, I looked at:
5593c8
5593c8
 - an Ubuntu 20.04.1 ppc64le pseries KVM guest:
5593c8
    vmlinux: ~30MB
5593c8
    initrd:  ~50MB
5593c8
5593c8
 - a RHEL8.2 ppc64le pseries KVM guest:
5593c8
    vmlinux: ~30MB
5593c8
    initrd:  ~30MB
5593c8
5593c8
Ubuntu VMs struggle to boot with just 256MB under SLOF.
5593c8
RHEL likewise has a higher minimum supported memory figure.
5593c8
So lets first consider a distro kernel and 512MB of addressible memory.
5593c8
(This is the default case for anything booting under PFW.) Say we lose
5593c8
131MB to PFW (based on some tests). This leaves us 381MB. 1/4 of 381MB
5593c8
is ~95MB. That should be enough to verify a 30MB vmlinux and should
5593c8
leave plenty of space to load Linux and the initrd.
5593c8
5593c8
If we consider 256MB of RMA under PFW, we have just 125MB remaining. 1/4
5593c8
of that is a smidge under 32MB, which gives us very poor odds of verifying
5593c8
a distro-sized kernel. However, if we need 80MB just to put the kernel
5593c8
and initrd in memory, we can't claim any more than 45MB anyway. So 1/4
5593c8
will do. We'll come back to this later.
5593c8
5593c8
grub is always built as a 32-bit binary, even if it's loading a ppc64
5593c8
kernel. So we can't address memory beyond 4GB. This gives a natural cap
5593c8
of 1GB for powerpc-ieee1275.
5593c8
5593c8
Also apply this 1/4 approach to i386-ieee1275, but keep the 32MB cap.
5593c8
5593c8
make check still works for both i386 and powerpc and I've booted
5593c8
powerpc grub with this change under SLOF and PFW.
5593c8
5593c8
Signed-off-by: Daniel Axtens <dja@axtens.net>
5593c8
---
5593c8
 grub-core/kern/ieee1275/init.c | 81 +++++++++++++++++++++++++++++++++---------
5593c8
 docs/grub-dev.texi             |  6 ++--
5593c8
 2 files changed, 69 insertions(+), 18 deletions(-)
5593c8
5593c8
diff --git a/grub-core/kern/ieee1275/init.c b/grub-core/kern/ieee1275/init.c
d3c3ab
index 0dcd114ce54..c61d91a0285 100644
5593c8
--- a/grub-core/kern/ieee1275/init.c
5593c8
+++ b/grub-core/kern/ieee1275/init.c
5593c8
@@ -46,11 +46,12 @@
5593c8
 #endif
5593c8
 #include <grub/lockdown.h>
5593c8
 
5593c8
-/* The maximum heap size we're going to claim */
5593c8
+/* The maximum heap size we're going to claim. Not used by sparc.
5593c8
+   We allocate 1/4 of the available memory under 4G, up to this limit. */
5593c8
 #ifdef __i386__
5593c8
 #define HEAP_MAX_SIZE		(unsigned long) (64 * 1024 * 1024)
5593c8
-#else
5593c8
-#define HEAP_MAX_SIZE		(unsigned long) (32 * 1024 * 1024)
5593c8
+#else // __powerpc__
5593c8
+#define HEAP_MAX_SIZE		(unsigned long) (1 * 1024 * 1024 * 1024)
5593c8
 #endif
5593c8
 
5593c8
 extern char _end[];
5593c8
@@ -147,16 +148,45 @@ grub_claim_heap (void)
5593c8
 				 + GRUB_KERNEL_MACHINE_STACK_SIZE), 0x200000);
5593c8
 }
5593c8
 #else
5593c8
-/* Helper for grub_claim_heap.  */
5593c8
+/* Helper for grub_claim_heap on powerpc. */
5593c8
+static int
5593c8
+heap_size (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type,
5593c8
+	   void *data)
5593c8
+{
5593c8
+  grub_uint32_t total = *(grub_uint32_t *)data;
5593c8
+
5593c8
+  if (type != GRUB_MEMORY_AVAILABLE)
5593c8
+    return 0;
5593c8
+
5593c8
+  /* Do not consider memory beyond 4GB */
5593c8
+  if (addr > 0xffffffffUL)
5593c8
+    return 0;
5593c8
+
5593c8
+  if (addr + len > 0xffffffffUL)
5593c8
+    len = 0xffffffffUL - addr;
5593c8
+
5593c8
+  total += len;
5593c8
+  *(grub_uint32_t *)data = total;
5593c8
+
5593c8
+  return 0;
5593c8
+}
5593c8
+
5593c8
 static int
5593c8
 heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type,
5593c8
 	   void *data)
5593c8
 {
5593c8
-  unsigned long *total = data;
5593c8
+  grub_uint32_t total = *(grub_uint32_t *)data;
5593c8
 
5593c8
   if (type != GRUB_MEMORY_AVAILABLE)
5593c8
     return 0;
5593c8
 
5593c8
+  /* Do not consider memory beyond 4GB */
5593c8
+  if (addr > 0xffffffffUL)
5593c8
+    return 0;
5593c8
+
5593c8
+  if (addr + len > 0xffffffffUL)
5593c8
+    len = 0xffffffffUL - addr;
5593c8
+
5593c8
   if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_NO_PRE1_5M_CLAIM))
5593c8
     {
5593c8
       if (addr + len <= 0x180000)
5593c8
@@ -170,10 +200,6 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type,
5593c8
     }
5593c8
   len -= 1; /* Required for some firmware.  */
5593c8
 
5593c8
-  /* Never exceed HEAP_MAX_SIZE  */
5593c8
-  if (*total + len > HEAP_MAX_SIZE)
5593c8
-    len = HEAP_MAX_SIZE - *total;
5593c8
-
5593c8
   /* In theory, firmware should already prevent this from happening by not
5593c8
      listing our own image in /memory/available.  The check below is intended
5593c8
      as a safeguard in case that doesn't happen.  However, it doesn't protect
5593c8
@@ -185,6 +211,18 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type,
5593c8
       len = 0;
5593c8
     }
5593c8
 
5593c8
+  /* If this block contains 0x30000000 (768MB), do not claim below that.
5593c8
+     Linux likes to claim memory at min(RMO top, 768MB) and works down
5593c8
+     without reference to /memory/available. */
5593c8
+  if ((addr < 0x30000000) && ((addr + len) > 0x30000000))
5593c8
+    {
5593c8
+      len = len - (0x30000000 - addr);
5593c8
+      addr = 0x30000000;
5593c8
+    }
5593c8
+
5593c8
+  if (len > total)
5593c8
+    len = total;
5593c8
+
5593c8
   if (len)
5593c8
     {
5593c8
       grub_err_t err;
5593c8
@@ -193,10 +231,12 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type,
5593c8
       if (err)
5593c8
 	return err;
5593c8
       grub_mm_init_region ((void *) (grub_addr_t) addr, len);
5593c8
+      total -= len;
5593c8
     }
5593c8
 
5593c8
-  *total += len;
5593c8
-  if (*total >= HEAP_MAX_SIZE)
5593c8
+  *(grub_uint32_t *)data = total;
5593c8
+
5593c8
+  if (total == 0)
5593c8
     return 1;
5593c8
 
5593c8
   return 0;
5593c8
@@ -205,13 +245,22 @@ heap_init (grub_uint64_t addr, grub_uint64_t len, grub_memory_type_t type,
5593c8
 static void 
5593c8
 grub_claim_heap (void)
5593c8
 {
5593c8
-  unsigned long total = 0;
5593c8
+  grub_uint32_t total = 0;
5593c8
 
5593c8
   if (grub_ieee1275_test_flag (GRUB_IEEE1275_FLAG_FORCE_CLAIM))
5593c8
-    heap_init (GRUB_IEEE1275_STATIC_HEAP_START, GRUB_IEEE1275_STATIC_HEAP_LEN,
5593c8
-	       1, &total);
5593c8
-  else
5593c8
-    grub_machine_mmap_iterate (heap_init, &total);
5593c8
+    {
5593c8
+      heap_init (GRUB_IEEE1275_STATIC_HEAP_START, GRUB_IEEE1275_STATIC_HEAP_LEN,
5593c8
+		 1, &total);
5593c8
+      return;
5593c8
+    }
5593c8
+
5593c8
+  grub_machine_mmap_iterate (heap_size, &total);
5593c8
+
5593c8
+  total = total / 4;
5593c8
+  if (total > HEAP_MAX_SIZE)
5593c8
+    total = HEAP_MAX_SIZE;
5593c8
+
5593c8
+  grub_machine_mmap_iterate (heap_init, &total);
5593c8
 }
5593c8
 #endif
5593c8
 
5593c8
diff --git a/docs/grub-dev.texi b/docs/grub-dev.texi
d3c3ab
index 19f708ee662..90083772c8a 100644
5593c8
--- a/docs/grub-dev.texi
5593c8
+++ b/docs/grub-dev.texi
5593c8
@@ -1047,7 +1047,9 @@ space is limited to 4GiB. GRUB allocates pages from EFI for its heap, at most
5593c8
 1.6 GiB.
5593c8
 
5593c8
 On i386-ieee1275 and powerpc-ieee1275 GRUB uses same stack as IEEE1275.
5593c8
-It allocates at most 32MiB for its heap.
5593c8
+
5593c8
+On i386-ieee1275, GRUB allocates at most 32MiB for its heap. On
5593c8
+powerpc-ieee1275, GRUB allocates up to 1GiB.
5593c8
 
5593c8
 On sparc64-ieee1275 stack is 256KiB and heap is 2MiB.
5593c8
 
5593c8
@@ -1075,7 +1077,7 @@ In short:
5593c8
 @item i386-qemu               @tab 60 KiB  @tab < 4 GiB
5593c8
 @item *-efi                   @tab ?       @tab < 1.6 GiB
5593c8
 @item i386-ieee1275           @tab ?       @tab < 32 MiB
5593c8
-@item powerpc-ieee1275        @tab ?       @tab < 32 MiB
5593c8
+@item powerpc-ieee1275        @tab ?       @tab < 1 GiB
5593c8
 @item sparc64-ieee1275        @tab 256KiB  @tab 2 MiB
5593c8
 @item arm-uboot               @tab 256KiB  @tab 2 MiB
5593c8
 @item mips(el)-qemu_mips      @tab 2MiB    @tab 253 MiB