How to configure huge pages in Linux?

Huge pages in Linux are a memory management feature that allows the operating system to use larger memory pages than the default 4KB, typically 2MB or 1GB. Configuring huge pages can significantly improve performance for applications that handle large amounts of memory, such as databases and high-performance computing workloads, by reducing Translation Lookaside Buffer (TLB) miss rates.

Understanding Huge Pages

Standard memory pages are 4KB in size. When an application requests memory, the CPU's Memory Management Unit (MMU) translates virtual addresses to physical addresses using a lookup table cached in the Translation Lookaside Buffer (TLB). For applications using large amounts of memory, a 4KB page size can lead to a large number of TLB entries and frequent TLB misses, causing performance overhead.

Huge pages address this by allowing applications to use much larger page sizes (e.g., 2MB or 1GB). This reduces the number of required TLB entries and, consequently, the number of TLB misses, leading to more efficient memory access and improved application performance.

How to Configure Huge Pages in Linux

Configuring huge pages involves several steps, from verifying kernel support to setting user limits and allocating the pages system-wide.

1. Check Kernel Support for Huge Pages

Before configuring huge pages, it's essential to confirm that your Linux kernel supports them. Most modern Linux distributions have huge page support compiled in by default.

You can verify this by checking for the hugetlbfs filesystem type:

grep HugePages /proc/meminfo

You should see output similar to this, indicating the total and free huge pages:

AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

The presence of Hugepagesize and other HugePages_ entries indicates kernel support.

2. Determine the Number of Huge Pages to Allocate

The number of huge pages to allocate depends on your application's memory requirements. For database systems like Oracle, this often corresponds to a portion of the System Global Area (SGA). It's crucial to allocate enough huge pages to cover the maximum memory requirement without excessively reducing memory available for other system processes.

To calculate the required number of huge pages, divide the target memory size (e.g., SGA size for a database) by the huge page size. For example, if your Hugepagesize is 2048 kB (2MB) and you need to allocate 8GB (8192 MB) for huge pages:

Number of Huge Pages = 8192 MB / 2 MB = 4096 huge pages

3. Allocate Huge Pages System-Wide

Once you've determined the number of huge pages, you need to configure the kernel to reserve them at boot time. This is done by modifying the vm.nr_hugepages kernel parameter.

To set huge pages temporarily (until next reboot):

sudo sysctl -w vm.nr_hugepages=<number_of_huge_pages>

Replace <number_of_huge_pages> with the calculated value.

To set huge pages persistently (recommended):

Edit the /etc/sysctl.conf file and add or modify the following line:

vm.nr_hugepages = <number_of_huge_pages>

After modifying /etc/sysctl.conf, apply the changes without rebooting:

sudo sysctl -p

4. Configure Memory Lock Limit (memlock) for Users

Applications using huge pages must be able to lock memory into RAM to prevent it from being swapped out. This requires setting the memlock limit for the user account that will run the application.

Edit the /etc/security/limits.conf file. Add or modify the following lines, replacing <user> with the specific user (e.g., oracle for an Oracle database installation):

<user>           soft    memlock         unlimited
<user>           hard    memlock         unlimited

Alternatively, you can set a specific memory limit in kilobytes if you prefer not to use unlimited. This value should be greater than or equal to the total size of huge pages the user's processes will allocate.

Example for an Oracle user:

oracle           soft    memlock         unlimited
oracle           hard    memlock         unlimited

After saving the file, the user needs to log out and log back in for the changes to take effect.

5. Verify the Memory Lock Setting

After updating /etc/security/limits.conf and re-logging in as the specified user, you can verify the new memlock setting:

ulimit -l

The output should show the memlock limit, which should reflect the unlimited setting or the specific kilobyte value you configured. For unlimited, it often shows "unlimited" or a very large number.

6. Mount the Hugetlbfs Filesystem (Optional but Common)

While vm.nr_hugepages reserves the huge pages, some applications, especially older ones, might explicitly expect huge pages to be mounted via hugetlbfs. This is a pseudo-filesystem specifically for huge pages.

To mount temporarily:

sudo mkdir /dev/hugepages
sudo mount -t hugetlbfs none /dev/hugepages

To mount persistently (recommended):

Add the following line to /etc/fstab:

none            /dev/hugepages          hugetlbfs       rw      0 0

Then, mount it:

sudo mount /dev/hugepages

Summary of Configuration Steps

Here's a concise overview of the steps:

By following these steps, you can successfully configure huge pages on your Linux system, optimizing memory usage and improving the performance of memory-intensive applications.