From patchwork Tue Jun 16 22:58:21 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 137173 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9B37A4A9F1F5 for ; Tue, 16 Jun 2026 22:59:05 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 9B37A4A9F1F5 Authentication-Results: sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20251104 header.b=P0ASieyG X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-ot1-x32a.google.com (mail-ot1-x32a.google.com [IPv6:2607:f8b0:4864:20::32a]) by sourceware.org (Postfix) with ESMTPS id 53D634A9F1F2 for ; Tue, 16 Jun 2026 22:58:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 53D634A9F1F2 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 53D634A9F1F2 Authentication-Results: sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::32a ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1781650714; cv=none; b=jwAMxjFmM5ZULwRoB7rbjKY2B19tqSd8n0PcsYBx9d38zujyqseB9ASXAa2jvScPFjDkzacpuHjpi9uUrt2pmB0wK76GM3JQxZ0fODe4sPLYOc0xUErzFYZ3IvvuSbm2QGc9sy4XYcjYV/VwXvhKaJyrOXgwRNPMOPAapaTdvco= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1781650714; c=relaxed/simple; bh=MhwefC294Ju02rB2nfrZyFqcGLx1iMkzCuYcAnfxesc=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=LB0jVuAdQDgViI3OIYpwkLBRSYbqcdLV+Ef+9WtWSI1TvWWTApICntXqIGXcF03od2jM+T7iT3zFQ5lQsSpp3ZMrE/vY64hNRv59xhjJaA8qFmYu1I1iw4MWeAwejSigQpZMiIyWCap9qdzlXQ4GTSbUyHcttjq/HlEPXt1uwfc= ARC-Authentication-Results: i=1; sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20251104 header.b=P0ASieyG DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 53D634A9F1F2 Received: by mail-ot1-x32a.google.com with SMTP id 46e09a7af769-7e6dcad6018so4417287a34.3 for ; Tue, 16 Jun 2026 15:58:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781650713; x=1782255513; darn=sourceware.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=jt24ckkrBTuuE62rIeTMZfJupnQkPpIrZS1j7r/Ib88=; b=P0ASieyG5VpA4VUp1kf2td2PmaR03OZ0aFgEOEoBKXO9D6/vxjUhEJaNOy0YbsLEna usAWkmWtN4dd+cyQzEvR0qUfEVQlNDeCXNEgpSTBUTTp6bqIabMGi0xnp7MVpR1z0Wdn 6MYhYJMmqGh5CZPsOah0GX6pC1y4HxrSDAra7RZlmKoKSzCnnU6qyjamBfD2k3pqCJBt e4CoYTgOg+8eMDH+75yC/mbNOqqxgJ1jsjLWzVZQWh51/goBxUnzs7i4sJZ47x97tvlo 6Z6Sr+zRxFs+5b1pQVP8ygu70q4jCROFekAgl0C1NkRBtskVUK082gSNsc1+Lo3hXWyw 0TAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781650713; x=1782255513; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=jt24ckkrBTuuE62rIeTMZfJupnQkPpIrZS1j7r/Ib88=; b=J5iR1V0ctqHO+NJ4vMdO7wo+Qvl/sSeiqo4sDS2vzhL4TFZaJlxQpWw1v6d5ayRKhe wZtH86S/HmPhnYnw8e9/GAfUbl4o4ElLjiWMZs4wUzedJpe79eudGm/N9LFrwnDOy52V 8RAcZFfOG0yfv5+yIaUhIAHToIYBJACzdI4HGHjG2d8rKtPAkRo0kfeIlDfo5wEcCm5d S7ivklsmRwV3XZ1n2nMIlTTiaXUOcYhojEl9B7yp2+0jN1ofC+7FGqwHCNeyoGYNOn38 2lJRq+9gBYOWXly4KQ9CtGtOSqPPRkJhXAWcQRD2RNvqP1V3vH6LOiP/0GycHTdMfDji xIUw== X-Gm-Message-State: AOJu0YxGKuNqdSyPbCWBjLytI14R5PQ4OSQ0UAH4456Ql+xbydMujKOF KSU858jRnwKUeAilWPaT5KwZuayCWPkJ4fEs9bELs8U60hURMgQFPgiv X-Gm-Gg: Acq92OH1oAE4yiZiENDdKLouCGxjYilTDCRUi6FJ8Tk+cDUcby/Duv9uG3QI499mAgy q3i7gNQSw8uqW5DsRA3xVbXvrWCGA+36mg+4WoP9Wc2IJ5BPigdJaB2V964PZ2QNe3/59ldzBz0 RujwWVcWzDxQEi0ZnTSZG4Pxd/MrOl9VBxqZKArP3llWUc0FakNiQMjJybJfwYJiqF7rBmvhVk9 OWAVBPpDI9Pn0izwoEsJ1KNlX4EaIYCxBgiAmQH7JnxNjrUPZxG3vjamdyL3IfUvFq+n/ujUr4h k902ftk1HvWufVPhwUqsr2NCCm6udtt+W1PjzqLMWQ8uz2sB3effnKGMnAHwLoGTrApPaVtTPe+ E7R4UKK/S65jtR93Sim85GNMoOAcWGl84WoV2nM6Dh8ZQf5cb/z8WV6XZ1c1CPki5t6PKkm+LgU WPaDqT4H9o0BMEjnLXliXjbUNgtnlzNyJLWyx2W7HexaVcf4HqogTgQsxqxw== X-Received: by 2002:a05:6830:2783:b0:7e7:352:2f74 with SMTP id 46e09a7af769-7e90b43fae5mr1384821a34.13.1781650713476; Tue, 16 Jun 2026 15:58:33 -0700 (PDT) Received: from gnu-tgl-3.localdomain ([172.58.181.94]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7e79f6b7e6csm7561620a34.17.2026.06.16.15.58.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Jun 2026 15:58:33 -0700 (PDT) Received: from gnu-tgl-3.localdomain (localhost [127.0.0.1]) by gnu-tgl-3.localdomain (Postfix) with ESMTP id 78A09C01F8; Wed, 17 Jun 2026 06:58:29 +0800 (CST) From: "H.J. Lu" To: libc-alpha@sourceware.org Cc: adhemerval.zanella@linaro.org, dj@redhat.com, carlos@redhat.com, Wilco.Dijkstra@arm.com Subject: [PATCH] malloc: Don't call __get_thp_mode/__get_thp_size twice Date: Wed, 17 Jun 2026 06:58:21 +0800 Message-ID: <20260616225821.152017-1-hjl.tools@gmail.com> X-Mailer: git-send-email 2.54.0 MIME-Version: 1.0 X-Spam-Status: No, score=-3008.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_ABUSEAT, RCVD_IN_DNSWL_NONE, RCVD_IN_PBL, RCVD_IN_SBL_CSS, RCVD_IN_XBL, SPF_HELO_NONE, SPF_PASS, TXREP shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org This patch is on top of commit 8c84793bab32868e557615c2e74b65ea06c0cd85 Author: H.J. Lu Date: Mon Apr 13 08:23:05 2026 +0800 elf: Support THP segment load with madvise enabled THP https://patchwork.sourceware.org/project/glibc/list/?series=62321 --- Both ld.so and malloc track kernel THP mode and THP page size when THP in ld.so is enabled by GLIBC_TUNABLES=glibc.elf.thp=1 and THP in malloc is enabled by GLIBC_TUNABLES=glibc.malloc.hugetlb=1 But DL_MAP_DEFAULT_THP_PAGESIZE and MALLOC_DEFAULT_THP_PAGESIZE may be different when they are defined in . If THP in ld.so is enabled, change malloc to use kernel THP mode from ld.so, instead of calling __get_thp_mode, and use THP page size from ld.so if it came from __get_thp_size. This avoids calling __get_thp_mode and __get_thp_size again, which are quite expensive: 1. Initialize mp_.thp_mode and GL(dl_thp_mode) to thp_mode_unknown. 2. Set mp_.thp_mode to GL(dl_thp_mode) if GL(dl_thp_mode) isn't thp_mode_unknown. Otherwise call __get_thp_mode to set mp_.thp_mode. 3. GL(dl_elf_thp_pagesize) is set to DL_MAP_DEFAULT_THP_PAGESIZE without calling __get_thp_size and THP page size for malloc may be different from THP page size for ld.so. Set mp_.thp_pagesize to GL(dl_elf_thp_pagesize) if DL_MAP_DEFAULT_THP_PAGESIZE is defined. Otherwise call __get_thp_size to set mp_.thp_pagesize. This fixes BZ #34083. Signed-off-by: H.J. Lu --- malloc/malloc.c | 56 +++++++++++++++++++++++++++++++++---- sysdeps/generic/hugepages.h | 3 +- 2 files changed, 53 insertions(+), 6 deletions(-) diff --git a/malloc/malloc.c b/malloc/malloc.c index a354576aca..f0ed95213f 100644 --- a/malloc/malloc.c +++ b/malloc/malloc.c @@ -1529,7 +1529,7 @@ static struct malloc_par mp_ = .mmap_threshold = DEFAULT_MMAP_THRESHOLD, .trim_threshold = DEFAULT_TRIM_THRESHOLD, .arena_test = sizeof (long) == 4 ? 2 : 8, - .thp_mode = thp_mode_not_supported + .thp_mode = thp_mode_unknown #if USE_TCACHE , .tcache_count = TCACHE_FILL_COUNT, @@ -4657,13 +4657,41 @@ do_set_mxfast (size_t value) return 1; } +#ifdef HAVE_THP +static __always_inline enum thp_mode_t +get_dl_thp_mode (void) +{ + return GL(dl_thp_mode); +} + +static __always_inline unsigned long int +get_dl_elf_thp_pagesize (void) +{ + return GL(dl_elf_thp_pagesize); +} +#else +# define get_dl_thp_mode() __get_thp_mode () +# define get_dl_elf_thp_pagesize() __get_thp_size () +#endif + static __always_inline int do_set_hugetlb (size_t value) { + /* If __get_thp_mode and __get_thp_size have been called during + startup, don't call them again here. */ + enum thp_mode_t thp_mode = get_dl_thp_mode (); + /* Enable THP if MALLOC_DEFAULT_THP_PAGESIZE is non-zero. */ if (MALLOC_DEFAULT_THP_PAGESIZE > 0) { - mp_.thp_mode = thp_mode_madvise; + /* If thp_mode is unknown, THP segment load is disabled by + GLIBC_TUNABLES=glibc.elf.thp=0. In this case, set + mp_.thp_mode to madvise. Otherwise, set it to thp_mode to + keep mp_.thp_mode in sync with GL(dl_thp_mode). */ + if (thp_mode == thp_mode_unknown) + mp_.thp_mode = thp_mode_madvise; + else + mp_.thp_mode = thp_mode; mp_.thp_pagesize = MALLOC_DEFAULT_THP_PAGESIZE; } @@ -4680,9 +4708,27 @@ do_set_hugetlb (size_t value) if (MALLOC_DEFAULT_THP_PAGESIZE > 0) return 0; - mp_.thp_mode = __get_thp_mode (); - if (mp_.thp_mode == thp_mode_madvise || mp_.thp_mode == thp_mode_always) - mp_.thp_pagesize = __get_thp_size (); + if (thp_mode == thp_mode_unknown) + { + /* Call __get_thp_mode and __get_thp_size when THP segment load + is disabled. */ + mp_.thp_mode = __get_thp_mode (); + if (mp_.thp_mode == thp_mode_madvise + || mp_.thp_mode == thp_mode_always) + mp_.thp_pagesize = __get_thp_size (); + } + else + { + /* THP segment load is enabled. GL(dl_elf_thp_pagesize) is + set to DL_MAP_DEFAULT_THP_PAGESIZE if it isn't zero . In + this case, call __get_thp_size () instead of using + DL_MAP_DEFAULT_THP_PAGESIZE for malloc. */ + mp_.thp_mode = thp_mode; + if (DL_MAP_DEFAULT_THP_PAGESIZE != 0) + mp_.thp_pagesize = __get_thp_size (); + else + mp_.thp_pagesize = get_dl_elf_thp_pagesize (); + } } else if (value >= 2) __get_hugepage_config (value == 2 ? 0 : value, &mp_.hp_pagesize, diff --git a/sysdeps/generic/hugepages.h b/sysdeps/generic/hugepages.h index 8e54661527..49251d94ab 100644 --- a/sysdeps/generic/hugepages.h +++ b/sysdeps/generic/hugepages.h @@ -37,7 +37,8 @@ unsigned long int __get_thp_size (void) attribute_hidden; enum thp_mode_t { - thp_mode_not_supported = 0, + thp_mode_unknown = 0, + thp_mode_not_supported, thp_mode_always, thp_mode_madvise, thp_mode_never