From patchwork Fri May 1 20:59:25 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shamil Abdulaev X-Patchwork-Id: 134345 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [127.0.0.1]) by sourceware.org (Postfix) with ESMTP id C87E74374200 for ; Fri, 1 May 2026 21:00:03 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C87E74374200 Authentication-Results: sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20251104 header.b=pDL23jJr X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com [IPv6:2a00:1450:4864:20::232]) by sourceware.org (Postfix) with ESMTPS id A49184374200 for ; Fri, 1 May 2026 20:59:33 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org A49184374200 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org A49184374200 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::232 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1777669173; cv=none; b=T+5TrDFcve5WxfD0DMlH4xSE9SrVaCN+ZvtCAPQ2XAXaxyBgDRE/0r3+kQEg8/octoi4aI00fcCjwHARYeYmUWLMRFuvYOOBA3xEbN4qDIrRlWXjeRpjw3hAzAXwm1UARhO3+lj2+03OP8nI6ETJ7wKrJUkJBbJGtbLrO5W6ne0= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1777669173; c=relaxed/simple; bh=f9M2g8YqZJq6UcUpdwmuVscojp+96Pzhjkq7KtWZfmQ=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=DyI4SoVcsCsKlDzoQnYq6kP3MFHrPreUnxxV2URwab7KRzYh+5YPyCClXwTZ4OM8+SEGJrWWyOEVNWskzGFn8Wi+o9PZ3/Vtyp8nEZEo/ugk7VBLjjmctLqFxNkkdgL2GjGrw9a8x7+KUqhOog6wsajI1xP8NIcoYOYjWTonDDc= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A49184374200 Received: by mail-lj1-x232.google.com with SMTP id 38308e7fff4ca-38ea6a5a0b3so22807141fa.3 for ; Fri, 01 May 2026 13:59:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1777669172; x=1778273972; darn=sourceware.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=bQWpTtnWerBdgkfUIneF0afBpzfDHmVCLJRJs9ZWVQ0=; b=pDL23jJrdrCAmsawCR3BnNrWDekjeiofEpqGkDEWDlTKBOJLmm5uT+bBjRn+UvX4xe Rt8fxz9eG5KyGHmjZoyV8UxB7LxOw/xIWXUbXmtSxcdZmugMFqDlDd/UTVlXb3pGFGxH lnqBy65fx0rgEiY2FBpo/ucWpKtlQwDmjonZIWiTeGae1J7PlPXfIaNF5/2o4cU3NBeZ QPtzS5dzFP4QuNnXMl/uLpRU1S01AdWbLAi0r5gLi92mxWrgOjAWQ/UN7yA4tMX8RqnN k/25WXghmaCfTLmZSWBlRqya+44a4S/ekMiqymlmFreJzX6/CFFqB+TiJXTTn/Jlsrks 6dJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777669172; x=1778273972; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=bQWpTtnWerBdgkfUIneF0afBpzfDHmVCLJRJs9ZWVQ0=; b=mrx5zg9is6+5uMSY83KjosBKrKz+CdWcsstCCzqeAAUOgBjlqiNyHj/BtyRw6RH5mf hDzxctyNoit2kWdxPwSuhQu7eAuJgMlVjItFb2uD4zYkHvOkGQ/srovBPmevQmwURiNJ eTXYk9s4WshNXW+zjfOOeSmhLQ5t+jyni2s5ZDOwUZgM8L8BYQ9KM3FdDK0gdbHzUhkb zop0jJQQwqfl9l9b+z2Ui3EOzyV7j5jKcfOi3nMoXAv0XbnVmI6wb9sSTM3PQnM69psD R/NxZbPkOS5RmzCSdFreaRQwT30OXXL1vRObWbYcn+iQwE3NU9Fo9Q598qboEzTncEBi onCQ== X-Gm-Message-State: AOJu0YzbKfLxF+dsODhN7Sq73HW+JDu+tqQiRRm+YAYNlN8X0vb0vbp1 FM5PtKKTtyZtLwR9qBZ//0Ops30ko5z0ZSbLjPp09txCY+vJXXggOyo6wcNmN0f5GOw= X-Gm-Gg: AeBDiesWNh0PyITjPtBaH77aAkBYt/4a08vohAeTbKXqpr1fcye8YhCMVOL9aFKz1S+ SDRazzGoOwlnWEGSCWXrS8H2Nyj2qGxUUKNjvE0qF6IUw2jP7rLIGr/ra08jIKh6URKs86AGWgm VYkW8lzGTX+B7u21GTc8wavfgnGzY3OfxzlEJOgTK7zxX5KfTBGj8DaCBf7g634/uUSMHKaWlYx rQu44mIiJMvHF9uGWxaqPPMI2Zp9YK3rTS0HbDRfQhzjnpOBJ0wL+3PLcviVaHsHZeQ3DwuqKOA XBr39rP5qim2kRDXPswc/L0yhwEK1DCLB8KmYfrBTsks5PqORrT1suwCEjTTzDNIuDgIOmCJOzN lRmV121xq2hhDdHmq0cKsPNrWa2I0rEO6ReXoOrdnZ1FbvFGiGsbqy0QP90CufpwCDVjJUePD3O Z1pYIF86FK6ANCQw8oAtE+Xsj0K3GvfNNtkMZkBMNgnUO0y7rr4Y60/17c3SqrQCo7WQ46bz1mS 3Qoy6sSJO/EjirJ8BlQ95YAQsp2rs/UZr3JwRS6BGc= X-Received: by 2002:a2e:bc02:0:b0:38e:a842:52f1 with SMTP id 38308e7fff4ca-393784ad0b5mr1644461fa.11.1777669171754; Fri, 01 May 2026 13:59:31 -0700 (PDT) Received: from oops-lab.cluster.local ([62.152.34.188]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-393610d6d2dsm10027731fa.17.2026.05.01.13.59.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 May 2026 13:59:30 -0700 (PDT) From: Shamil Abdulaev To: libc-alpha@sourceware.org Cc: hjl.tools@gmail.com Subject: [PATCH] hugepages: cache THP mode and page size [BZ #34083] Date: Fri, 1 May 2026 23:59:25 +0300 Message-ID: X-Mailer: git-send-email 2.54.0 MIME-Version: 1.0 X-Spam-Status: No, score=-9.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FORGED_GMAIL_RCVD, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM, GB_FREEMAIL_NUM, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org __get_thp_mode and __get_thp_size read /sys files and are used by both the dynamic linker and malloc. Cache their results in rtld_global so the expensive THP probes are performed once per process. Tested on x86_64-linux-gnu. No new test failures. Signed-off-by: Shamil Abdulaev --- elf/dl-support.c | 6 +++ sysdeps/generic/hugepages.h | 7 ++++ sysdeps/generic/ldsodefs.h | 6 +++ sysdeps/unix/sysv/linux/hugepages.c | 58 ++++++++++++++++++++++++----- 4 files changed, 68 insertions(+), 9 deletions(-) diff --git a/elf/dl-support.c b/elf/dl-support.c index 0508d6113b..ec77cb3e7a 100644 --- a/elf/dl-support.c +++ b/elf/dl-support.c @@ -144,6 +144,12 @@ int _dl_inhibit_cache; /* All known directories in sorted order. */ struct r_search_path_elem *_dl_all_dirs; +/* Cached Transparent Huge Page mode and page size. */ +int _dl_thp_mode; +unsigned long int _dl_thp_pagesize; +int _dl_thp_mode_state; +int _dl_thp_pagesize_state; + /* All directories after startup. */ struct r_search_path_elem *_dl_init_all_dirs; diff --git a/sysdeps/generic/hugepages.h b/sysdeps/generic/hugepages.h index 5fc9b5c8de..68be2ee242 100644 --- a/sysdeps/generic/hugepages.h +++ b/sysdeps/generic/hugepages.h @@ -32,6 +32,13 @@ enum thp_mode_t thp_mode_not_supported }; +enum thp_cache_state_t +{ + thp_cache_uninitialized, + thp_cache_initializing, + thp_cache_initialized +}; + enum thp_mode_t __get_thp_mode (void) attribute_hidden; /* Return the supported huge page size from the REQUESTED sizes on PAGESIZE diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h index 15c4659853..4b2e337a81 100644 --- a/sysdeps/generic/ldsodefs.h +++ b/sysdeps/generic/ldsodefs.h @@ -395,6 +395,12 @@ struct rtld_global /* List of search directories. */ EXTERN struct r_search_path_elem *_dl_all_dirs; + /* Cached Transparent Huge Page mode and page size. */ + EXTERN int _dl_thp_mode; + EXTERN unsigned long int _dl_thp_pagesize; + EXTERN int _dl_thp_mode_state; + EXTERN int _dl_thp_pagesize_state; + /* Get architecture specific definitions. */ #define PROCINFO_DECL #ifndef PROCINFO_CLASS diff --git a/sysdeps/unix/sysv/linux/hugepages.c b/sysdeps/unix/sysv/linux/hugepages.c index 46e05151a3..c32127f6a4 100644 --- a/sysdeps/unix/sysv/linux/hugepages.c +++ b/sysdeps/unix/sysv/linux/hugepages.c @@ -16,6 +16,8 @@ License along with the GNU C Library; see the file COPYING.LIB. If not, see . */ +#include +#include #include #include #include @@ -25,18 +27,32 @@ unsigned long int __get_thp_size (void) { + if (atomic_load_acquire (&GL (dl_thp_pagesize_state)) + == thp_cache_initialized) + return GL (dl_thp_pagesize); + + int state = thp_cache_uninitialized; + if (!atomic_compare_exchange_acquire (&GL (dl_thp_pagesize_state), + &state, thp_cache_initializing)) + { + while (atomic_load_acquire (&GL (dl_thp_pagesize_state)) + != thp_cache_initialized) + atomic_spin_nop (); + return GL (dl_thp_pagesize); + } + + unsigned long int r = 0; int fd = __open64_nocancel ( "/sys/kernel/mm/transparent_hugepage/hpage_pmd_size", O_RDONLY); if (fd == -1) - return 0; + goto out; char str[INT_BUFSIZE_BOUND (unsigned long int)]; ssize_t s = __read_nocancel (fd, str, sizeof (str)); __close_nocancel (fd); if (s < 0) - return 0; + goto out; - unsigned long int r = 0; for (ssize_t i = 0; i < s; i++) { if (str[i] == '\n') @@ -44,16 +60,36 @@ __get_thp_size (void) r *= 10; r += str[i] - '0'; } + +out: + GL (dl_thp_pagesize) = r; + atomic_store_release (&GL (dl_thp_pagesize_state), + thp_cache_initialized); return r; } enum thp_mode_t __get_thp_mode (void) { + if (atomic_load_acquire (&GL (dl_thp_mode_state)) + == thp_cache_initialized) + return (enum thp_mode_t) GL (dl_thp_mode); + + int state = thp_cache_uninitialized; + if (!atomic_compare_exchange_acquire (&GL (dl_thp_mode_state), + &state, thp_cache_initializing)) + { + while (atomic_load_acquire (&GL (dl_thp_mode_state)) + != thp_cache_initialized) + atomic_spin_nop (); + return (enum thp_mode_t) GL (dl_thp_mode); + } + + enum thp_mode_t mode = thp_mode_not_supported; int fd = __open64_nocancel ("/sys/kernel/mm/transparent_hugepage/enabled", O_RDONLY); if (fd == -1) - return thp_mode_not_supported; + goto out; static const char mode_always[] = "[always] madvise never\n"; static const char mode_madvise[] = "always [madvise] never\n"; @@ -63,19 +99,23 @@ __get_thp_mode (void) ssize_t s = __read_nocancel (fd, str, sizeof (str)); __close_nocancel (fd); if (s >= sizeof str || s < 0) - return thp_mode_not_supported; + goto out; str[s] = '\0'; if (s == sizeof (mode_always) - 1) { if (strcmp (str, mode_always) == 0) - return thp_mode_always; + mode = thp_mode_always; else if (strcmp (str, mode_madvise) == 0) - return thp_mode_madvise; + mode = thp_mode_madvise; else if (strcmp (str, mode_never) == 0) - return thp_mode_never; + mode = thp_mode_never; } - return thp_mode_not_supported; + +out: + GL (dl_thp_mode) = mode; + atomic_store_release (&GL (dl_thp_mode_state), thp_cache_initialized); + return mode; } #if !IS_IN (rtld)