From patchwork Fri May 22 17:08:12 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella X-Patchwork-Id: 135471 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 1103748F342F for ; Fri, 22 May 2026 17:11:47 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 1103748F342F Authentication-Results: sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=FkjviKiQ X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-vk1-xa34.google.com (mail-vk1-xa34.google.com [IPv6:2607:f8b0:4864:20::a34]) by sourceware.org (Postfix) with ESMTPS id 78CE748F5278 for ; Fri, 22 May 2026 17:11:06 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 78CE748F5278 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 78CE748F5278 Authentication-Results: sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::a34 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1779469866; cv=none; b=ne0YkQn/uqSVMo1SAQ5sYKL8sulmvY+Z+yLLD5C6Rs+3WMsDvNThrJmdm167y0BTLh5M538yNXsMCbZQxlIgFg/6erXqFoqGmABpJm+Z3SkES7bG0J2NBgxo8czL2ENwjd29fZBWMuhxv6oAohM3jgpvNI4LVTKH++Lh6Mj9xu0= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1779469866; c=relaxed/simple; bh=DwpCaJVyXY++O1861q3HiQ7LkxJLgsR3To9is1i+0po=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=iSiCIR/eJBtRNqkT5aE4e1oDmLEQEh9TcLJdnkLu/qfsoY1b7QTvhTHS+FSRXBuixj8/CpGTlciOgzLCEBfxB4grcxnk4hovNNX0PQuwwXBtvhFOUXdq7kjOX2GKCsZ2mVGuHp6NqbOufsIHdn4xcrmPN+LikryQ/WizD1SU51A= ARC-Authentication-Results: i=1; sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=FkjviKiQ DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 78CE748F5278 Received: by mail-vk1-xa34.google.com with SMTP id 71dfb90a1353d-575171b1ce7so2470829e0c.1 for ; Fri, 22 May 2026 10:11:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1779469866; x=1780074666; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IXB0aJ2zym3Jzq/UhVXHoiWe11B9Osg69a3RhR0EP1Y=; b=FkjviKiQl3L0pxI3aIdRsZc45S0h/deMi5GE9hiG+NNKmk8GhAFVVP7ENLDz1Vnj3+ T4W7hpXCThxTO1TTzkpdJ0WYXf44oxwKRthRj+AAKUiCI/DcgQ56/tdpDJU+mEVff0Y2 QmqcdjVHDEZ49qz7nDPT9StRdZYSZy3TqYviL69WSOXKgmuvNPeTh3z99euUNEPeqEED P0p3lH/vlHHy4ttpAqR75mIp8pbBo5GsOna4hgV7l8MyxCvhvq4TxHp3L0i8fTKlb+qZ mtysqcVFuz7cpcH5ObkVXi/kdYQ+sEqfWfqYGhIM36GmV/Mdup63jnToEAKkahPcxwGU DgxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779469866; x=1780074666; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=IXB0aJ2zym3Jzq/UhVXHoiWe11B9Osg69a3RhR0EP1Y=; b=oT/r2LQcCvaMGeRE4BF+fQX+A00YLFnLUW88lBXDMkOaLkaFOZ8L5ZxVM+f8bQKd5S RAl0pVe2bBWxVQel9IU6Es8sdCtQ9ac2l7E3Hzpp9lfBqRfA2PiMWaW+L1ykYCsF/TaA CUWPRXI73jORHiFlvhPEes1ozQLOjIvzzNd8OliiL4rz4dam6RxmlNc04iNyJsyrHB1B 8ZhxNm0u+5ms57bbpwQLkfHNwY5l1YWfTD0YYR2tu9OWccjdmdHLlEgBnW41Ojc14kz0 8doUYGMzvp7Xz8BcZbKr/q8uVczNxlpETDblLktShVCllUySTgHuvSXV7UA6xSGB6l95 k4lA== X-Gm-Message-State: AOJu0YxY3TI8+pxIc2ny8ZGIspjXpiNwU7cRRN370bI3g9de8vsjFrmb uw77Vs1ocYvxXquz7MhTvnE58JHRGtjO27+9wd2DZoHV25cLyMvQkh5iCu1w0Gu59ZUGUghnkya 4svwP X-Gm-Gg: Acq92OG/4NkowjCAg7lSSStz99zhpsqmpMX8OBAkdHOh1RmOlwY7798EyDESnVkY9ly /DJsPt3wDinQ1WWM1ZhIZS4eIEq8/zzjrPgCYyd51H3JEQk6z7eZJUMdg52M240cjmY9fuR6wOv wu11w9kRx+ltJx7cAYST6KvRDBqCM3UNXCgX2p63ceSNIjg82Grb4Y3YdcizCTB4wWZQ5nFWdLn VsdYtKYv4Qw4XT7geCy5MA6yvKOlBmn2FrnI5HsfwGxLyoJEPEAiHlMbPaKXXFHaADU67/6hngl 1e7j7gO0cXAd13OXuYMvvReFSf5C/KpB3+o9aLxbuHrMqQ1h1G2B6grSIK6E4YV145Zo3Z0RabO eAI+rwhvY36iWWYpSVD32LHlQYtzKxeU8syD1cuqKAXD403BkAY1H1X5aS7CvyBi+v0aUxB4b53 sljAKNeCCBshdXnQMOw7HUKIDJH1T0QvKaXf0= X-Received: by 2002:a05:6122:a5cf:10b0:588:c9a6:2807 with SMTP id 71dfb90a1353d-588c9a6446bmr508680e0c.10.1779469865564; Fri, 22 May 2026 10:11:05 -0700 (PDT) Received: from mandiga.. ([2804:1b3:a7c2:8e05:f627:cadc:3dc9:b8bf]) by smtp.gmail.com with ESMTPSA id a1e0cc1a2514c-96173842ed1sm2235743241.6.2026.05.22.10.11.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 May 2026 10:11:05 -0700 (PDT) From: Adhemerval Zanella To: libc-alpha@sourceware.org Cc: "H . J . Lu" Subject: [PATCH 1/2] elf: Initialize static TLS before relocation processing (BZ 34164) Date: Fri, 22 May 2026 14:08:12 -0300 Message-ID: <20260522171058.1681922-2-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260522171058.1681922-1-adhemerval.zanella@linaro.org> References: <20260522171058.1681922-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, URIBL_BLOCKED shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org An IFUNC resolver firing during dynamic linker relocation reads its DSO's __thread storage from a zero-filled slot: init_tls() allocates the static TLS block zero-filled, but .tdata is not copied in until the trailing _dl_allocate_tls_init at the end of dl_main, long after the per-object phase 2 resolvers from commit 63b31c05a8a901 have run. A resolver that *writes* TLS is even worse off -- the write is clobbered by that same trailing copy. dl_main (elf/rtld.c): populate the DTV slotinfo, bump dl_tls_generation, and call _dl_allocate_tls_init right after init_tls(), before the relocation loop. _dl_try_allocate_static_tls (elf/dl-reloc.c): drop the "defer-if-not-relocated" branch and always run _dl_init_static_tls inline, so a CHECK_STATIC_TLS allocation triggered mid-relocation initialises the slot before the same object's phase 2 fires. The new tests cheks some scenarios: elf/tst-ifunc-tls-init resolver reads its DSO's IE TLS. elf/tst-ifunc-tls-init-dlopen same, via dlopen. elf/tst-ifunc-tls-write resolver write to TLS must survive to main. Checked on aarch64-linux-gnu, x86_64-linux-gnu, and i686-linux-gnu. --- elf/Makefile | 10 ++++ elf/dl-reloc.c | 27 +++++---- elf/rtld.c | 31 ++++++---- elf/tst-ifunc-tls-init-dlopen-lib.c | 55 ++++++++++++++++++ elf/tst-ifunc-tls-init-lib-skeleton.c | 57 +++++++++++++++++++ elf/tst-ifunc-tls-init-lib1.c | 1 + elf/tst-ifunc-tls-init-lib2.c | 1 + elf/tst-ifunc-tls-init.c | 81 +++++++++++++++++++++++++++ elf/tst-ifunc-tls-write-lib.c | 54 ++++++++++++++++++ elf/tst-ifunc-tls-write.c | 34 +++++++++++ 10 files changed, 327 insertions(+), 24 deletions(-) create mode 100644 elf/tst-ifunc-tls-init-dlopen-lib.c create mode 100644 elf/tst-ifunc-tls-init-lib-skeleton.c create mode 100644 elf/tst-ifunc-tls-init-lib1.c create mode 100644 elf/tst-ifunc-tls-init-lib2.c create mode 100644 elf/tst-ifunc-tls-init.c create mode 100644 elf/tst-ifunc-tls-write-lib.c create mode 100644 elf/tst-ifunc-tls-write.c diff --git a/elf/Makefile b/elf/Makefile index aef13b73ca5..ff42eb32d41 100644 --- a/elf/Makefile +++ b/elf/Makefile @@ -1268,6 +1268,8 @@ tests += \ tst-ifunc-plt-bindnow \ tst-ifunc-plt-dlopen \ tst-ifunc-plt-dlopen-bindnow \ + tst-ifunc-tls-init \ + tst-ifunc-tls-write \ # tests # Note: sysdeps/x86_64/ifuncmain8.c uses ifuncmain8. tests-internal += \ @@ -1332,6 +1334,9 @@ modules-names += \ ifuncmod6 \ tst-ifunc-plt-dep \ tst-ifunc-plt-lib \ + tst-ifunc-tls-init-lib1 \ + tst-ifunc-tls-init-lib2 \ + tst-ifunc-tls-write-lib \ # modules-names ifeq (no,$(with-lld)) modules-names += ifuncmod5 @@ -2468,6 +2473,11 @@ $(objpfx)tst-ifunc-plt-dlopen.out: \ $(objpfx)tst-ifunc-plt-dlopen-bindnow.out: \ $(objpfx)tst-ifunc-plt-lib.so $(objpfx)tst-ifunc-plt-dep.so +$(objpfx)tst-ifunc-tls-init: $(objpfx)tst-ifunc-tls-init-lib1.so +$(objpfx)tst-ifunc-tls-init.out: \ + $(objpfx)tst-ifunc-tls-init-lib2.so +$(objpfx)tst-ifunc-tls-write: $(objpfx)tst-ifunc-tls-write-lib.so + $(objpfx)tst-unique1.out: $(objpfx)tst-unique1mod1.so \ $(objpfx)tst-unique1mod2.so diff --git a/elf/dl-reloc.c b/elf/dl-reloc.c index f1a432ac099..67299400cad 100644 --- a/elf/dl-reloc.c +++ b/elf/dl-reloc.c @@ -107,22 +107,21 @@ _dl_try_allocate_static_tls (struct link_map *map, bool optional) # error "Either TLS_TCB_AT_TP or TLS_DTV_AT_TP must be defined" #endif - /* If the object is not yet relocated we cannot initialize the - static TLS region. Delay it. */ - if (map->l_real->l_relocated) - { + /* Initialise the static TLS region, the map may not yet be l_relocated (a + TLS reloc inside the relocation loop triggered the allocation), but + _dl_init_static_tls only writes .tdata into the static TLS slot, which is + independent of relocation state. + Doing this inline ensures any IFUNC resolver that fires laterin the same + object's relocation pass sees an initialised TLS slot, and the + post-relocation TLS init loop in dl_open_worker_begin becomes a no-op for + this map. */ #ifdef SHARED - /* Update the DTV of the current thread. Note: GL(dl_load_tls_lock) - is held here so normal load of the generation counter is valid. */ - if (__builtin_expect (THREAD_DTV()[0].counter != GL(dl_tls_generation), - 0)) - (void) _dl_update_slotinfo (map->l_tls_modid, GL(dl_tls_generation)); + /* Update the DTV of the current thread. Note: GL(dl_load_tls_lock) + is held here so normal load of the generation counter is valid. */ + if (__glibc_unlikely (THREAD_DTV()[0].counter != GL(dl_tls_generation))) + _dl_update_slotinfo (map->l_tls_modid, GL(dl_tls_generation)); #endif - - _dl_init_static_tls (map); - } - else - map->l_need_tls_init = 1; + _dl_init_static_tls (map); return 0; } diff --git a/elf/rtld.c b/elf/rtld.c index e926ec73e49..12e1b4dd71f 100644 --- a/elf/rtld.c +++ b/elf/rtld.c @@ -2263,6 +2263,26 @@ dl_main (const ElfW(Phdr) *phdr, rtld_timer_accum (&relocate_time, start); } + /* Populate the DTV slotinfo and copy each TLS module's into thestatic TLS + block *before* the relocation loop. IFUNC resolvers fired during phase 2 + of the per-object two-phase scheme therefore observe initialised TLS. */ + if (__rtld_tls_init_tp_called) + { + unsigned int i = main_map->l_searchlist.r_nlist; + while (i-- > 0) + { + struct link_map *l = main_map->l_initfini[i]; + if (l->l_tls_blocksize != 0) + _dl_add_to_slotinfo (l, true); + } + /* _dl_add_to_slotinfo records gen = dl_tls_generation + 1, and + _dl_allocate_tls_init asserts gen <= dl_tls_generation, so bump + the generation before init. */ + if (GL(dl_tls_max_dtv_idx) > 0) + ++GL(dl_tls_generation); + _dl_allocate_tls_init (tcbp, true); + } + RTLD_TIMING_VAR (start); rtld_timer_start (&start); { @@ -2286,10 +2306,6 @@ dl_main (const ElfW(Phdr) *phdr, _dl_relocate_object (l, l->l_scope, GLRO(dl_lazy) ? RTLD_LAZY : 0, consider_profiling); - - /* Add object to slot information data if necessary. */ - if (l->l_tls_blocksize != 0 && __rtld_tls_init_tp_called) - _dl_add_to_slotinfo (l, true); } } rtld_timer_stop (&relocate_time, start); @@ -2310,12 +2326,7 @@ dl_main (const ElfW(Phdr) *phdr, || count_modids != _dl_count_modids ()) ++GL(dl_tls_generation); - /* Now that we have completed relocation, the initializer data - for the TLS blocks has its final values and we can copy them - into the main thread's TLS area, which we allocated above. - Note: thread-local variables must only be accessed after completing - the next step. */ - _dl_allocate_tls_init (tcbp, true); + /* TLS .tdata copy moved before the relocation loop above. */ /* And finally install it for the main thread. */ if (! __rtld_tls_init_tp_called) diff --git a/elf/tst-ifunc-tls-init-dlopen-lib.c b/elf/tst-ifunc-tls-init-dlopen-lib.c new file mode 100644 index 00000000000..652e220b3f0 --- /dev/null +++ b/elf/tst-ifunc-tls-init-dlopen-lib.c @@ -0,0 +1,55 @@ +/* Shared library for tst-ifunc-tls-init-dlopen. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Same as tst-ifunc-tls-init-lib.c, but exercises the dlopen path. */ + +#define SENTINEL 0x5A5A1234 + +static volatile __thread int sentinel + __attribute__ ((tls_model ("initial-exec"))) = SENTINEL; +static volatile int last_seen_sentinel; + +static int +impl_ok (void) +{ + return SENTINEL; +} + +static int +impl_bad (void) +{ + return 0; +} + +int +get_last_seen_sentinel (void) +{ + return last_seen_sentinel; +} + +static int (*resolver (void)) (void) +{ + int s = sentinel; + last_seen_sentinel = s; + return s == SENTINEL ? impl_ok : impl_bad; +} + +int ifunc_tls (void) __attribute__ ((ifunc ("resolver"))); + +/* Force a non-PLT relocation against the IFUNC symbol. */ +int (*fptr) (void) = ifunc_tls; diff --git a/elf/tst-ifunc-tls-init-lib-skeleton.c b/elf/tst-ifunc-tls-init-lib-skeleton.c new file mode 100644 index 00000000000..d8b2a4f0f10 --- /dev/null +++ b/elf/tst-ifunc-tls-init-lib-skeleton.c @@ -0,0 +1,57 @@ +/* Test that static-TLS initialisation works correctly with IFUNC resolvers. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* The initial-exec TLS model keeps the access path to a single TP-relative + load, so the test is sensitive to whether the static TLS block has been + populated rather than to any __tls_get_addr / DTV-update timing. */ + +#define SENTINEL 0x5A5A1234 + +/* The 'volatile' avoids constant fold optimization in impl_ok. */ +static volatile __thread int sentinel + __attribute__ ((tls_model ("initial-exec"))) = SENTINEL; +static volatile int last_seen_sentinel; + +static int +impl_ok (void) +{ + return SENTINEL; +} + +static int +impl_bad (void) +{ + return 0; +} + +int +get_last_seen_sentinel (void) +{ + return last_seen_sentinel; +} + +static int (*resolver (void)) (void) +{ + int s = sentinel; + last_seen_sentinel = s; + return s == SENTINEL ? impl_ok : impl_bad; +} +int ifunc_tls (void) __attribute__ ((ifunc ("resolver"))); + +/* Force a non-PLT relocation against the IFUNC symbol. */ +int (*fptr) (void) = ifunc_tls; diff --git a/elf/tst-ifunc-tls-init-lib1.c b/elf/tst-ifunc-tls-init-lib1.c new file mode 100644 index 00000000000..ed9db110b1a --- /dev/null +++ b/elf/tst-ifunc-tls-init-lib1.c @@ -0,0 +1 @@ +#include "tst-ifunc-tls-init-lib-skeleton.c" diff --git a/elf/tst-ifunc-tls-init-lib2.c b/elf/tst-ifunc-tls-init-lib2.c new file mode 100644 index 00000000000..ed9db110b1a --- /dev/null +++ b/elf/tst-ifunc-tls-init-lib2.c @@ -0,0 +1 @@ +#include "tst-ifunc-tls-init-lib-skeleton.c" diff --git a/elf/tst-ifunc-tls-init.c b/elf/tst-ifunc-tls-init.c new file mode 100644 index 00000000000..8b996b7c60c --- /dev/null +++ b/elf/tst-ifunc-tls-init.c @@ -0,0 +1,81 @@ +/* Check if static-TLS variables are correctly intialized in IFUNC resolvers. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Checks if a IFUNC resolver sees if a TLS blocker is properly initialized. + The tst-ifunc-tls-init-lib.so carries: + + - a thread-local 'sentinel' initialised to SENTINEL. + - an IFUNC 'ifunc_tls' whose resolver picks impl_ok when sentinel reads + as SENTINEL and impl_bad (returning 0) otherwise. + - an IFUNC-backed function-pointer global 'fptr' whose initialiser + produces a non-PLT relocation. */ + +#include +#include + +#define SENTINEL 0x5A5A1234 + +extern int ifunc_tls (void); +extern int (*fptr) (void); +extern int get_last_seen_sentinel (void); + +static void +test_tls_ifunc (int (*check_sentinel)(void), + int (*check_fptr)(void), + int (*check_ifunc_tls)(void)) +{ + /* Primary check: 'get_last_seen_sentinel' returns the value of the DSO's + thread-local 'sentinel' as observed by the resolver at the moment it ran + for the IFUNC reloc that initialised fptr. The getter is a regular + PLT-resolved function in the DSO, so the read of the diagnostic global + does NOT go through a COPY relocation that could overwrite the resolver's + write. */ + TEST_COMPARE (check_sentinel (), SENTINEL); + + /* Secondary check: fptr is set during IFUNC resolver call, then copied into + the exe's. Returns SENTINEL only if the resolver picked impl_ok. */ + TEST_VERIFY (check_fptr != NULL); + TEST_COMPARE (check_fptr (), SENTINEL); + + /* Sanity check: issue the ifunc. */ + TEST_COMPARE (check_ifunc_tls (), SENTINEL); +} + +static int +do_test (void) +{ + test_tls_ifunc (get_last_seen_sentinel, fptr, ifunc_tls); + + /* Same as before, but check the dlopen path. */ + void *handle = xdlopen ("tst-ifunc-tls-init-lib2.so", + RTLD_LAZY | RTLD_LOCAL); + + int (*get_last_seen_sentinel_dlopen) (void) + = xdlsym (handle, "get_last_seen_sentinel"); + int (**fptr_dlopen) (void) = xdlsym (handle, "fptr"); + int (*ifunc_tls_dlopen) (void) = xdlsym (handle, "ifunc_tls"); + + test_tls_ifunc (get_last_seen_sentinel_dlopen, *fptr_dlopen, + ifunc_tls_dlopen); + + xdlclose (handle); + + return 0; +} + +#include diff --git a/elf/tst-ifunc-tls-write-lib.c b/elf/tst-ifunc-tls-write-lib.c new file mode 100644 index 00000000000..ef1dff90e01 --- /dev/null +++ b/elf/tst-ifunc-tls-write-lib.c @@ -0,0 +1,54 @@ +/* Shared library for tst-ifunc-tls-write. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* The DSO defines an initial-exec thread-local 'counter' initialised to + SENTINEL, the IFUNC resolver (via the non-PLT IFUNC reloc on 'fptr') + overwrites counter with MARKER, and the test then reads counter back + through a getter. */ + +#define SENTINEL 0x5A5A1234u +#define MARKER 0x32125A5Au + +/* The 'volatile' avoids constant fold optimization in impl_ok. */ +static volatile __thread unsigned int counter + __attribute__ ((tls_model ("initial-exec"))) = SENTINEL; + +static unsigned int +impl (void) +{ + return 0; +} + +static unsigned int (*resolver (void)) (void) +{ + counter = MARKER; + return impl; +} +unsigned int ifunc_write (void) __attribute__ ((ifunc ("resolver"))); + +/* Force the resolver rather than lazy bind on first call, which would + re-write counter after the test reads it). Using a COPY'd fptr also lets + the test verify the resolver ran without making a PLT call that would + itself fire the resolver again. */ +unsigned int (*fptr) (void) = ifunc_write; + +unsigned int +get_counter (void) +{ + return counter; +} diff --git a/elf/tst-ifunc-tls-write.c b/elf/tst-ifunc-tls-write.c new file mode 100644 index 00000000000..8c3711c161c --- /dev/null +++ b/elf/tst-ifunc-tls-write.c @@ -0,0 +1,34 @@ +/* Check if static-TLS variables are correctly intialized in IFUNC resolvers. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#define MARKER 0x32125A5Au + +extern unsigned int (*fptr) (void); +extern unsigned int get_counter (void); + +static int +do_test (void) +{ + TEST_VERIFY (fptr != NULL); + TEST_COMPARE (get_counter (), MARKER); + return 0; +} + +#include From patchwork Fri May 22 17:08:13 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella X-Patchwork-Id: 135472 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 785444BABF18 for ; Fri, 22 May 2026 17:13:36 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 785444BABF18 Authentication-Results: sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=sArm6hP5 X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-ua1-x933.google.com (mail-ua1-x933.google.com [IPv6:2607:f8b0:4864:20::933]) by sourceware.org (Postfix) with ESMTPS id E7F0D48F8A53 for ; Fri, 22 May 2026 17:11:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E7F0D48F8A53 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org E7F0D48F8A53 Authentication-Results: sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::933 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1779469869; cv=none; b=OrjCTjjY+Lz21Y+bTGDiQ9IEgstmpP+X8+a3sf+aDvRCad1bxaeNHRsq6QKHv9YqAZdB+KaiRUyZnKtqS85wtIzUmSTkkoi7TrJzUeE54Hrdqckz+MCM3VpdL1uotjfgjaEdiLFhlPMiwyPvRmz2Dd9ZAxOJnafJZ5wQAIZd/UA= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1779469869; c=relaxed/simple; bh=QPqqUNyQ4ubQh2BTAHoJ3aLw9JtXF0MqWX51dZRYZjU=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=wUhMs6tFiWih4CgdJCqKXw2XJThlsXiVwjlUoFFhNfFpg4z+stNrBmwHVrvvLMyKCxJdUB2TgLU3sKCOIoTawHtLRmIf/WMJVbYpi/ZeVwpV1TwEFliIHmnJ0HdGjIu7q3fByoiDWRWLCvEFyZhe6NP5QAqRup80G98UUcUgWXk= ARC-Authentication-Results: i=1; sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=sArm6hP5 DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org E7F0D48F8A53 Received: by mail-ua1-x933.google.com with SMTP id a1e0cc1a2514c-95cc96eac66so5124887241.1 for ; Fri, 22 May 2026 10:11:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1779469868; x=1780074668; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LFJAN/iQ0DL+q1ZZ/JCEXwM/azPaw7U7LJXdG1RgGK0=; b=sArm6hP5r/v7T2/Cg1jsOtkXMtqA9hfLYrRIag1LXKZn2O/PwJaV7tMR5i6T3Fsr16 F5Gb4BhsgZktXDEm8tDN23m66cJEOaF7gBnQDDztn+h0wgvdET1f+7GQHEwuTGv6t7Vt MKPMTVE88KHyyODlUl1N9Kv0tb3AQRUHDxgdjHWMO0EppIm5KQcJc0kxuesoF5vjtgVa 8e6MUq4N0Nnyp1KIrFmAuP3rLI7z3C3SY0Wrc+bs0R3Z+ebN34r9f6kcS0cZLUpqDVcZ bM8ptYMYqU+0o0mqUYA3onbH7rTqfFDvsm2l0pH+QMaWQHJDo2RJQhmOneLZNDbczfsz IU5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779469868; x=1780074668; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=LFJAN/iQ0DL+q1ZZ/JCEXwM/azPaw7U7LJXdG1RgGK0=; b=PHf6BQR2lD2gxLI3b6kh0K8R5QgtX/dCX6XRS60FtF/JTTB2ZScNgfktZR9jz0QHni 2/cOdcVIxMBGWIBiPkr21hA/GMrlWPUyxXus/YM4SUCDGkQkGiJTjHH4zY8o1FB2Tmtp N6vPLFLxZCcYhD2nCvXi2gX9qAjIsZq2TPBqAoAzOP3GSCkKCbBJfOq8kL8c05kHlJSF VBwvNf4fmd1usKUitGNmHbbqpRnylsrOWJnBc+MnyQAK0VOmDJCr2z5LKh9fAva41g+6 RNZsRYpLZ+AceOK9sb5FjjP3Ceulr+uwd31gbxdhp738IXHoA25m0uouwGAvvFW2EpUS oHTQ== X-Gm-Message-State: AOJu0Yypj3Ztd9mmBPuqtra/sX7z3sZXcM0Y/NushO9nx9A6pSK3kklm 7pKu4wzChHhcwjDYiPLk53xP/KHYLXdHJbxqVXkIOJYdpX1X3sntWheY4e+y0hzMx+NXXCJ5yR+ DnkdF X-Gm-Gg: Acq92OHV108IQi3et/Xkf0ZV9DaDFzK72j8s7Wry2CNiGaPYtMMHq3VURKksyy4ZCXX kGnVZaRTXV3fhDPoPLTJUflhKq7PoJu6AtJBAf4GYAapB+1CSk9GLyN1UKZzLsSJhq3Y2qu+QAK jxyvxtq6zA1YMRIYRYF3H+c1FfVorl+cFsXOwXB5UsKdOyHcLJJBulKV4TYT6sSDp4sw6q6eaBR 8Io8uGK8PZLqKF2jnrMU0mqsoFuzd1mnhAEhh2BU/lQJrkqywi9PNK6D6pASDsrjw4N5kAeMzo7 /+E4qc4g6jqKckGK2jBSO09pxGr2smp6Ucny5fI8QKxYeo2QvPwEmax0Lkgh3xqEa50dmd83ffZ ri5dWoZcLuL7hn53rbkNv2BVWswP094phMgsGz5ARFzPWlhQW2K5rUasoTqehYWj2zwY5gie63H 7WzQN1W5ii4wZA7uVWLxcIwliArUOLqqoI8mMWvb0qAkbJTw== X-Received: by 2002:a05:6122:4213:b0:577:51e4:ef20 with SMTP id 71dfb90a1353d-58662da7b33mr2670414e0c.10.1779469867664; Fri, 22 May 2026 10:11:07 -0700 (PDT) Received: from mandiga.. ([2804:1b3:a7c2:8e05:f627:cadc:3dc9:b8bf]) by smtp.gmail.com with ESMTPSA id a1e0cc1a2514c-96173842ed1sm2235743241.6.2026.05.22.10.11.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 May 2026 10:11:07 -0700 (PDT) From: Adhemerval Zanella To: libc-alpha@sourceware.org Cc: "H . J . Lu" Subject: [PATCH 2/2] elf: Initialize TCB and stack-protector before static IFUNC resolvers (BZ 20680, BZ 27582, BZ 28817) Date: Fri, 22 May 2026 14:08:13 -0300 Message-ID: <20260522171058.1681922-3-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260522171058.1681922-1-adhemerval.zanella@linaro.org> References: <20260522171058.1681922-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, URIBL_BLOCKED shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org In static linking the IFUNC IPLT (apply_irel for non-PIE, the IRELATIVE phase inside _dl_relocate_static_pie for static-pie) ran before __libc_setup_tls and before _dl_setup_stack_chk_guard. When a resolver is compiled with -fstack-protector(-all) its prologue loads the canary from the TCB (TCB-canary ABIs: x86_64, i386, powerpc, s390) or from __stack_chk_guard (global-var ABIs). On the former the resolver crashed reading an unmapped TCB; on the latter it loaded a zero canary (no crash, but the check is ineffective). The same applies to a resolver that reads any thread-local: it crashes on TCB-canary ABIs and observes a zero-filled slot on the others (BZ 20680). The pointer guard has the same problem (e.g. resolvers that register an atexit handler). Reorder csu/libc-start.c so that ARCH_SETUP_TLS, the stack-protector canary and the pointer guard are set up before any IFUNC resolver runs. For static-pie this requires splitting the existing _dl_relocate_static_pie into two phases so the TCB/canary setup can be interleaved between the non-IRELATIVE and IRELATIVE passes. The historical ARCH_SETUP_IREL / ARCH_APPLY_IREL split (introduced for powerpc so its IFUNC resolvers could read TCB fields like hwcap and at_platform) is no longer required: TLS is now set up before either macro runs. ARCH_APPLY_IREL is removed, ARCH_SETUP_IREL does the work uniformly on every arch, and the powerpc-specific libc-start.h becomes redundant. __libc_setup_tls reaches memcpy / mempcpy via _dl_allocate_tls_init in elf/dl-tls.c, so it requires update ABI specific dl-symbol-redir-ifunc.h with memcpy/memmove. Tests added (each fails pre-fix on TCB-canary ABIs with SIGSEGV; the static-protector variants additionally fail on global-var ABIs with a "resolver_canary != main_canary" diagnostic): elf/tst-ifunc-bz28817 static-pie + TLS in resolver (BZ 28817) elf/tst-ifunc-resolver-protector dynamic elf/tst-ifunc-resolver-protector-static static-pie elf/tst-ifunc-resolver-protector-static-non-pie non-PIE static Checked on aarch64-linux-gnu, arm-linux-gnueabihf, x86_64-linux-gnu, and i686-linux-gnu I also ran the ELF tests on qemu system for loongarch64-linux-gnuf64, powerpc-linux-gnu, powerpc-linux-gnu-power4, powerpc-linux-gnu-soft, powerpc64-linux-gnu, powerpc64le-linux-gnu, riscv64-linux-gnu, and s390x-linux-gnu. --- csu/libc-start.c | 40 +++++---- csu/static-reloc.c | 5 ++ elf/Makefile | 43 +++++++++- elf/dl-reloc-static-pie.c | 24 ++++-- elf/dynamic-link.h | 84 +++++++++++++++---- elf/tst-ifunc-bz28817.c | 60 +++++++++++++ elf/tst-ifunc-resolver-protector-mod.c | 61 ++++++++++++++ elf/tst-ifunc-resolver-protector-static-mod.c | 68 +++++++++++++++ ...nc-resolver-protector-static-non-pie-mod.c | 2 + ...-ifunc-resolver-protector-static-non-pie.c | 5 ++ elf/tst-ifunc-resolver-protector-static.c | 61 ++++++++++++++ elf/tst-ifunc-resolver-protector.c | 42 ++++++++++ .../aarch64/multiarch/dl-symbol-redir-ifunc.h | 4 + sysdeps/generic/ldsodefs.h | 7 +- sysdeps/generic/libc-start.h | 9 +- .../lp64/multiarch/dl-symbol-redir-ifunc.h | 2 + sysdeps/unix/sysv/linux/aarch64/libc-start.h | 1 - sysdeps/unix/sysv/linux/powerpc/libc-start.h | 30 ------- sysdeps/x86_64/libc-start.h | 1 - .../x86_64/multiarch/dl-symbol-redir-ifunc.h | 19 +++++ 20 files changed, 490 insertions(+), 78 deletions(-) create mode 100644 elf/tst-ifunc-bz28817.c create mode 100644 elf/tst-ifunc-resolver-protector-mod.c create mode 100644 elf/tst-ifunc-resolver-protector-static-mod.c create mode 100644 elf/tst-ifunc-resolver-protector-static-non-pie-mod.c create mode 100644 elf/tst-ifunc-resolver-protector-static-non-pie.c create mode 100644 elf/tst-ifunc-resolver-protector-static.c create mode 100644 elf/tst-ifunc-resolver-protector.c delete mode 100644 sysdeps/unix/sysv/linux/powerpc/libc-start.h diff --git a/csu/libc-start.c b/csu/libc-start.c index 1c58561bce0..03d770ef157 100644 --- a/csu/libc-start.c +++ b/csu/libc-start.c @@ -268,23 +268,22 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL), ARCH_INIT_CPU_FEATURES (); - /* Do static pie self relocation after tunables and cpu features - are setup for ifunc resolvers. Before this point relocations - must be avoided. */ + /* Do static-pie self relocation for the non-IRELATIVE part after tunables + and cpu features are set up. IFUNC entries are deferred until after the + TCB and the stack-protector canary are usable, so that an instrumented + resolver does not fault. Before this point relocations must be + avoided. */ _dl_relocate_static_pie (); - /* Perform IREL{,A} relocations. */ - ARCH_SETUP_IREL (); - - /* The stack guard goes into the TCB, so initialize it early. */ + /* Set up the TCB so that the IFUNC pass below can fire resolvers + compiled with stack protection, and so that resolvers reading TLS + (errno, __thread variables, powerpc's hwcap / at_platform in the + TCB) observe an initialised slot. */ ARCH_SETUP_TLS (); - /* In some architectures, IREL{,A} relocations happen after TLS setup in - order to let IFUNC resolvers benefit from TCB information, e.g. powerpc's - hwcap and platform fields available in the TCB. */ - ARCH_APPLY_IREL (); - - /* Set up the stack checker's canary. */ + /* Set up the stack checker's canary. Must happen before any IFUNC resolver + runs so a resolver compiled with stack protection loads a defined + canary. */ uintptr_t stack_chk_guard = _dl_setup_stack_chk_guard (_dl_random); # ifdef THREAD_SET_STACK_GUARD THREAD_SET_STACK_GUARD (stack_chk_guard); @@ -292,10 +291,6 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL), __stack_chk_guard = stack_chk_guard; # endif - /* Initialize libpthread if linked in. */ - if (__pthread_initialize_minimal != NULL) - __pthread_initialize_minimal (); - /* Set up the pointer guard value. */ uintptr_t pointer_chk_guard = _dl_setup_pointer_guard (_dl_random, stack_chk_guard); @@ -305,6 +300,17 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL), __pointer_chk_guard_local = pointer_chk_guard; # endif + /* Now that the TCB, canary, and pointer guard are in place, run the + deferred IFUNC relocations. For non-PIE static binaries this is + ARCH_SETUP_IREL (apply_irel); for static-pie it is the IRELATIVE + phase of _dl_relocate_static_pie above. */ + _dl_relocate_static_pie_ifunc (); + ARCH_SETUP_IREL (); + + /* Initialize libpthread if linked in. */ + if (__pthread_initialize_minimal != NULL) + __pthread_initialize_minimal (); + #endif /* !SHARED */ /* Register the destructor of the dynamic linker if there is any. */ diff --git a/csu/static-reloc.c b/csu/static-reloc.c index 2044ca34864..4c3bc7b3270 100644 --- a/csu/static-reloc.c +++ b/csu/static-reloc.c @@ -23,4 +23,9 @@ void _dl_relocate_static_pie (void) { } + +void +_dl_relocate_static_pie_ifunc (void) +{ +} #endif diff --git a/elf/Makefile b/elf/Makefile index ff42eb32d41..d7bab52cd97 100644 --- a/elf/Makefile +++ b/elf/Makefile @@ -289,8 +289,17 @@ tests-static-internal := \ tst-tunables-enable_secure \ # tests-static-internal +ifeq (yes,$(have-gcc-ifunc)) +tests-static-internal += \ + tst-ifunc-resolver-protector-static \ + tst-ifunc-resolver-protector-static-non-pie \ + # tests-static-internal +endif + CRT-tst-tls1-static-non-pie := $(csu-objpfx)crt1.o tst-tls1-static-non-pie-no-pie = yes +CRT-tst-ifunc-resolver-protector-static-non-pie := $(csu-objpfx)crt1.o +tst-ifunc-resolver-protector-static-non-pie-no-pie = yes tests-container := \ tst-dl-cache-long-path \ @@ -744,6 +753,19 @@ test-extras += \ tst-tlsmod17a \ tst-tlsmod18a \ # test-extras + +ifeq (yes,$(have-gcc-ifunc)) +# The resolver helper needs for the TCB-canary STACK_CHK_GUARD +# macro, so it must be compiled with MODULE_NAME=testsuite_internal. +extra-test-objs += \ + tst-ifunc-resolver-protector-static-mod.o \ + tst-ifunc-resolver-protector-static-non-pie-mod.o \ + # extra-test-objs +test-internal-extras += \ + tst-ifunc-resolver-protector-static-mod \ + tst-ifunc-resolver-protector-static-non-pie-mod \ + # test-internal-extras +endif modules-names += \ circlemod1 \ circlemod1a \ @@ -1256,7 +1278,11 @@ tests-ifuncstatic := \ ifuncmain7static \ # tests-ifuncstatic ifeq (yes,$(have-gcc-ifunc)) -tests-ifuncstatic += ifuncmain9static ifuncmain9picstatic +tests-ifuncstatic += \ + ifuncmain9picstatic \ + ifuncmain9static \ + tst-ifunc-bz28817 \ + # tests-ifuncstatic endif tests-static += $(tests-ifuncstatic) tests-internal += $(tests-ifuncstatic) @@ -1268,6 +1294,7 @@ tests += \ tst-ifunc-plt-bindnow \ tst-ifunc-plt-dlopen \ tst-ifunc-plt-dlopen-bindnow \ + tst-ifunc-resolver-protector \ tst-ifunc-tls-init \ tst-ifunc-tls-write \ # tests @@ -1334,6 +1361,7 @@ modules-names += \ ifuncmod6 \ tst-ifunc-plt-dep \ tst-ifunc-plt-lib \ + tst-ifunc-resolver-protector-mod \ tst-ifunc-tls-init-lib1 \ tst-ifunc-tls-init-lib2 \ tst-ifunc-tls-write-lib \ @@ -1744,6 +1772,19 @@ $(objpfx)tst-nodelete-opened.out: $(objpfx)tst-nodelete-opened-lib.so $(objpfx)tst-tlsalign-extern: $(objpfx)tst-tlsalign-vars.o $(objpfx)tst-tlsalign-extern-static: $(objpfx)tst-tlsalign-vars.o +# The resolver translation unit must always be compiled with +# -fstack-protector-all so the canary code is emitted regardless of the +# default. +CFLAGS-tst-ifunc-resolver-protector-static-mod.c = -fstack-protector-all +CFLAGS-tst-ifunc-resolver-protector-static-non-pie-mod.c = -fstack-protector-all +CFLAGS-tst-ifunc-resolver-protector-mod.c = -fstack-protector-all +$(objpfx)tst-ifunc-resolver-protector-static: \ + $(objpfx)tst-ifunc-resolver-protector-static-mod.o +$(objpfx)tst-ifunc-resolver-protector-static-non-pie: \ + $(objpfx)tst-ifunc-resolver-protector-static-non-pie-mod.o +$(objpfx)tst-ifunc-resolver-protector: \ + $(objpfx)tst-ifunc-resolver-protector-mod.so + tst-null-argv-ENV = LD_DEBUG=all LD_DEBUG_OUTPUT=$(objpfx)tst-null-argv.debug.out LDFLAGS-nodel2mod3.so = -Wl,--no-as-needed LDFLAGS-reldepmod5.so = -Wl,--no-as-needed diff --git a/elf/dl-reloc-static-pie.c b/elf/dl-reloc-static-pie.c index 63ce609024e..8463e46147d 100644 --- a/elf/dl-reloc-static-pie.c +++ b/elf/dl-reloc-static-pie.c @@ -31,8 +31,10 @@ #include "dynamic-link.h" #include "get-dynamic-info.h" -/* Relocate static executable with PIE. */ - +/* Phase 1: relocate static PIE - non-IRELATIVE pass. IFUNC resolvers are + deferred to _dl_relocate_static_pie_ifunc so that csu/libc-start.c can + initialise the TCB (and write the stack-protector canary into it) between + the two passes. */ void _dl_relocate_static_pie (void) { @@ -76,10 +78,9 @@ _dl_relocate_static_pie (void) ELF_MACHINE_BEFORE_RTLD_RELOC (main_map, main_map->l_info); # endif - /* Relocate ourselves so we can do normal function calls and - data access using the global offset table. */ - ELF_DYNAMIC_RELOCATE (main_map, NULL, 0, 0, 0); - main_map->l_relocated = 1; + /* Relocate ourselves so we can do normal function calls and data access + using the global offset table. IRELATIVE entries are deferred. */ + ELF_DYNAMIC_RELOCATE_NOIFUNC (main_map, NULL, 0, 0); /* Initialize _r_debug_extended. */ struct r_debug *r = _dl_debug_initialize (0, LM_ID_BASE); @@ -89,4 +90,15 @@ _dl_relocate_static_pie (void) time. */ elf_setup_debug_entry (main_map, r); } + +/* Phase 2: run the deferred IRELATIVE entries for the static-pie main map. + Must be called after the TCB is set up and the stack-protector canary is + written, so that an instrumented IFUNC resolver does not fault. */ +void +_dl_relocate_static_pie_ifunc (void) +{ + struct link_map *main_map = _dl_get_dl_main_map (); + ELF_DYNAMIC_RELOCATE_IFUNC (main_map, NULL, 0, 0); + main_map->l_relocated = 1; +} #endif diff --git a/elf/dynamic-link.h b/elf/dynamic-link.h index 334e241f63c..35141acec4b 100644 --- a/elf/dynamic-link.h +++ b/elf/dynamic-link.h @@ -78,8 +78,18 @@ elf_machine_lazy_rel (struct link_map *map, struct r_scope_elem *scope[], consumes precisely the very end of the DT_REL*, or DT_JMPREL and DT_REL* are completely separate and there is a gap between them. */ +/* This controls which sub-passes _ELF_DYNAMIC_DO_RELOC runs. Used to + interleave TLS / stack-protector setup between the two passes so IFUNC + resolvers see a fully-initialised TCB. */ +enum elf_dynamic_reloc_phase +{ + DL_RELOC_BOTH = 0, /* Non-IRELATIVE pass then IRELATIVE pass. */ + DL_RELOC_NOIFUNC = 1, /* Non-IRELATIVE pass only. */ + DL_RELOC_IFUNC = 2, /* IRELATIVE pass only. */ +}; + # define _ELF_DYNAMIC_DO_RELOC(RELOC, reloc, map, scope, do_lazy, skip_ifunc, \ - test_rel) \ + test_rel, phase) \ do { \ struct { ElfW(Addr) start, size; \ __typeof (((ElfW(Dyn) *) 0)->d_un.d_val) nrelative; int lazy; } \ @@ -126,19 +136,21 @@ elf_machine_lazy_rel (struct link_map *map, struct r_scope_elem *scope[], by the linker. */ \ if (!DO_RTLD_BOOTSTRAP) \ { \ - for (int ranges_index = 0; ranges_index < 2; ++ranges_index) \ - elf_dynamic_do_##reloc ((map), scope, \ - ranges[ranges_index].start, \ - ranges[ranges_index].size, \ - ranges[ranges_index].nrelative, \ - ranges[ranges_index].lazy); \ - for (int ranges_index = 0; ranges_index < 2; ++ranges_index) \ - elf_dynamic_do_##reloc##_irelative ((map), scope, \ - ranges[ranges_index].start, \ - ranges[ranges_index].size, \ - ranges[ranges_index].nrelative,\ - ranges[ranges_index].lazy, \ - skip_ifunc); \ + if ((phase) != DL_RELOC_IFUNC) \ + for (int ranges_index = 0; ranges_index < 2; ++ranges_index) \ + elf_dynamic_do_##reloc ((map), scope, \ + ranges[ranges_index].start, \ + ranges[ranges_index].size, \ + ranges[ranges_index].nrelative, \ + ranges[ranges_index].lazy); \ + if ((phase) != DL_RELOC_NOIFUNC) \ + for (int ranges_index = 0; ranges_index < 2; ++ranges_index) \ + elf_dynamic_do_##reloc##_irelative ((map), scope, \ + ranges[ranges_index].start, \ + ranges[ranges_index].size, \ + ranges[ranges_index].nrelative,\ + ranges[ranges_index].lazy, \ + skip_ifunc); \ } \ else \ for (int ranges_index = 0; ranges_index < 2; ++ranges_index) \ @@ -158,18 +170,36 @@ elf_machine_lazy_rel (struct link_map *map, struct r_scope_elem *scope[], # if ! ELF_MACHINE_NO_REL # include "do-rel.h" # define ELF_DYNAMIC_DO_REL(map, scope, lazy, skip_ifunc) \ - _ELF_DYNAMIC_DO_RELOC (REL, Rel, map, scope, lazy, skip_ifunc, _ELF_CHECK_REL) + _ELF_DYNAMIC_DO_RELOC (REL, Rel, map, scope, lazy, skip_ifunc, \ + _ELF_CHECK_REL, DL_RELOC_BOTH) +# define ELF_DYNAMIC_DO_REL_NOIFUNC(map, scope, lazy) \ + _ELF_DYNAMIC_DO_RELOC (REL, Rel, map, scope, lazy, 0, \ + _ELF_CHECK_REL, DL_RELOC_NOIFUNC) +# define ELF_DYNAMIC_DO_REL_IFUNCONLY(map, scope, lazy, skip_ifunc) \ + _ELF_DYNAMIC_DO_RELOC (REL, Rel, map, scope, lazy, skip_ifunc, \ + _ELF_CHECK_REL, DL_RELOC_IFUNC) # else # define ELF_DYNAMIC_DO_REL(map, scope, lazy, skip_ifunc) /* Nothing to do. */ +# define ELF_DYNAMIC_DO_REL_NOIFUNC(map, scope, lazy) /* Nothing to do. */ +# define ELF_DYNAMIC_DO_REL_IFUNCONLY(map, scope, lazy, skip_ifunc) /* Nothing. */ # endif # if ! ELF_MACHINE_NO_RELA # define DO_RELA # include "do-rel.h" # define ELF_DYNAMIC_DO_RELA(map, scope, lazy, skip_ifunc) \ - _ELF_DYNAMIC_DO_RELOC (RELA, Rela, map, scope, lazy, skip_ifunc, _ELF_CHECK_REL) + _ELF_DYNAMIC_DO_RELOC (RELA, Rela, map, scope, lazy, skip_ifunc, \ + _ELF_CHECK_REL, DL_RELOC_BOTH) +# define ELF_DYNAMIC_DO_RELA_NOIFUNC(map, scope, lazy) \ + _ELF_DYNAMIC_DO_RELOC (RELA, Rela, map, scope, lazy, 0, \ + _ELF_CHECK_REL, DL_RELOC_NOIFUNC) +# define ELF_DYNAMIC_DO_RELA_IFUNCONLY(map, scope, lazy, skip_ifunc) \ + _ELF_DYNAMIC_DO_RELOC (RELA, Rela, map, scope, lazy, skip_ifunc, \ + _ELF_CHECK_REL, DL_RELOC_IFUNC) # else # define ELF_DYNAMIC_DO_RELA(map, scope, lazy, skip_ifunc) /* Nothing to do. */ +# define ELF_DYNAMIC_DO_RELA_NOIFUNC(map, scope, lazy) /* Nothing to do. */ +# define ELF_DYNAMIC_DO_RELA_IFUNCONLY(map, scope, lazy, skip_ifunc) /* Nothing. */ # endif # define ELF_DYNAMIC_DO_RELR(map) \ @@ -221,4 +251,26 @@ elf_machine_lazy_rel (struct link_map *map, struct r_scope_elem *scope[], ELF_DYNAMIC_AFTER_RELOC ((map), (edr_lazy)); \ } while (0) +/* Like ELF_DYNAMIC_RELOCATE but only processes the non-IRELATIVE pass. + The IRELATIVE pass must be completed later via ELF_DYNAMIC_RELOCATE_IFUNC. + Used by the static-pie startup so the TCB and stack-protector canary can + be initialised between the two passes. */ +# define ELF_DYNAMIC_RELOCATE_NOIFUNC(map, scope, lazy, consider_profile) \ + do { \ + int edr_lazy = elf_machine_runtime_setup ((map), (scope), (lazy), \ + (consider_profile)); \ + if (!is_rtld_link_map (map) || DO_RTLD_BOOTSTRAP) \ + ELF_DYNAMIC_DO_RELR (map); \ + ELF_DYNAMIC_DO_REL_NOIFUNC ((map), (scope), edr_lazy); \ + ELF_DYNAMIC_DO_RELA_NOIFUNC ((map), (scope), edr_lazy); \ + ELF_DYNAMIC_AFTER_RELOC ((map), (edr_lazy)); \ + } while (0) + +/* IRELATIVE-only companion to ELF_DYNAMIC_RELOCATE_NOIFUNC. */ +# define ELF_DYNAMIC_RELOCATE_IFUNC(map, scope, lazy, skip_ifunc) \ + do { \ + ELF_DYNAMIC_DO_REL_IFUNCONLY ((map), (scope), (lazy), skip_ifunc); \ + ELF_DYNAMIC_DO_RELA_IFUNCONLY ((map), (scope), (lazy), skip_ifunc); \ + } while (0) + #endif diff --git a/elf/tst-ifunc-bz28817.c b/elf/tst-ifunc-bz28817.c new file mode 100644 index 00000000000..fe6f400f4ea --- /dev/null +++ b/elf/tst-ifunc-bz28817.c @@ -0,0 +1,60 @@ +/* BZ 28817: TLS read from a static-pie IFUNC resolver. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include + +__thread int bar; +extern __thread int bar_gd asm ("bar") + __attribute__ ((tls_model("global-dynamic"))); +static int *bar_ptr; + +int foo (void); + +static void +init_foo (void) +{ + bar_ptr = &bar_gd; +} + +static int +my_foo (void) +{ + return bar_ptr != NULL; +} + +static __typeof (foo) * +inhibit_stack_protector +foo_ifunc (void) +{ + init_foo (); + __typeof (foo) *res = my_foo; + return res; +}; +__typeof (foo) foo __attribute__ ((ifunc ("foo" "_ifunc"))); + +static int +do_test (void) +{ + TEST_VERIFY (foo ()); + TEST_VERIFY (&bar == bar_ptr); + return 0; +} + +#include diff --git a/elf/tst-ifunc-resolver-protector-mod.c b/elf/tst-ifunc-resolver-protector-mod.c new file mode 100644 index 00000000000..36d11f04def --- /dev/null +++ b/elf/tst-ifunc-resolver-protector-mod.c @@ -0,0 +1,61 @@ +/* Stack-protector-instrumented IFUNC resolver in a shared library, used + by tst-ifunc-resolver-protector. BZ #27582. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Built with -fstack-protector-all so the resolver's prologue/epilogue + carries the full canary check. Reaching the return statement means the + canary load did not fault (TCB-canary ABIs) and the canary compare did not + call __stack_chk_fail. The dynamic linker calls security_init() before the + main relocation loop, so resolvers fired from startup or dlopen should + always observe an initialised canary; this test guards against any future + reordering that would break that invariant. */ + +#define SENTINEL 0x5A5A1234 + +static volatile int resolver_ran; + +int +get_resolver_ran (void) +{ + return resolver_ran; +} + +static int +impl_ok (int x) +{ + return x + SENTINEL; +} + +typedef int (*fn_t) (int); + +static fn_t +resolver (void) +{ + /* Buffer + zero-fill force -fstack-protector-all canary code. */ + volatile char buf[32]; + for (unsigned i = 0; i < sizeof (buf); ++i) + buf[i] = 0; + resolver_ran = 1; + return impl_ok; +} + +int compute (int) __attribute__ ((ifunc ("resolver"))); + +/* Address taken in DSO data to force an R_*_IRELATIVE in .rela.dyn (a + non-PLT relocation against the IFUNC). */ +int (*fptr) (int) = compute; diff --git a/elf/tst-ifunc-resolver-protector-static-mod.c b/elf/tst-ifunc-resolver-protector-static-mod.c new file mode 100644 index 00000000000..b1cb704854e --- /dev/null +++ b/elf/tst-ifunc-resolver-protector-static-mod.c @@ -0,0 +1,68 @@ +/* Stack-protector-instrumented IFUNC resolver used by + tst-ifunc-resolver-protector-static. BZ #34164. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* This translation unit is built with -fstack-protector-all so that the + compiler instruments the resolver below. The resolver's prologue loads + the canary from the TCB (TCB-canary ABIs) or from __stack_chk_guard + (global-var ABIs); the epilogue compares it against the on-stack copy. + Pre-fix the prologue load either faulted (TCB-canary) or loaded zero + (global-var). + + The resolver also records the canary value via STACK_CHK_GUARD so the + test can verify it matches what main observes. */ + +#include +#include +#include + +#define SENTINEL 0x5A5A1234 + +static volatile uintptr_t resolver_canary; + +uintptr_t +get_resolver_canary (void) +{ + return resolver_canary; +} + +static int +impl_ok (int x) +{ + return x + SENTINEL; +} + +typedef int (*fn_t) (int); + +static fn_t +resolver (void) +{ + /* Buffer forces -fstack-protector-all to emit canary code even with + no other reason to. */ + volatile char buf[32]; + for (unsigned i = 0; i < sizeof (buf); ++i) + buf[i] = 0; + + resolver_canary = STACK_CHK_GUARD; + + return impl_ok; +} + +int compute (int) __attribute__ ((ifunc ("resolver"))); + +int (*fptr) (int) = compute; diff --git a/elf/tst-ifunc-resolver-protector-static-non-pie-mod.c b/elf/tst-ifunc-resolver-protector-static-non-pie-mod.c new file mode 100644 index 00000000000..a5d200c0662 --- /dev/null +++ b/elf/tst-ifunc-resolver-protector-static-non-pie-mod.c @@ -0,0 +1,2 @@ +/* Companion module for tst-ifunc-resolver-protector-static-non-pie. */ +#include "tst-ifunc-resolver-protector-static-mod.c" diff --git a/elf/tst-ifunc-resolver-protector-static-non-pie.c b/elf/tst-ifunc-resolver-protector-static-non-pie.c new file mode 100644 index 00000000000..2f42e91f3b6 --- /dev/null +++ b/elf/tst-ifunc-resolver-protector-static-non-pie.c @@ -0,0 +1,5 @@ +/* Same coverage as tst-ifunc-resolver-protector-static, but linked as non-PIE + static. Exercises the apply_irel / __rela_iplt_start path instead of the + static-pie _dl_relocate_static_pie_ifunc path. */ + +#include "tst-ifunc-resolver-protector-static.c" diff --git a/elf/tst-ifunc-resolver-protector-static.c b/elf/tst-ifunc-resolver-protector-static.c new file mode 100644 index 00000000000..e594e1c2653 --- /dev/null +++ b/elf/tst-ifunc-resolver-protector-static.c @@ -0,0 +1,61 @@ +/* Check that a stack-protector-instrumented IFUNC resolver works in a + static binary. BZ #34164. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* When a static binary's IFUNC resolver is compiled with stack protector, the + resolver prologue loads the canary from the TCB (TCB-canary ABIs: x86_64, + i386, powerpc, s390) or from the global __stack_chk_guard (other ABIs). + The test checks if TCB-canary or the global-var is properly initialized: + + 1. The resolver runs without SIGSEGV (TCB-canary) or SIGABRT (canary + mismatch). + + 2. The resolver records the canary value it observed via STACK_CHK_GUARD, + and main asserts it equals the value visible from user code -- catching + the "silent zero canary" variant on global-var ABIs. */ + +#include +#include +#include +#include + +#define SENTINEL 0x5A5A1234 + +extern int compute (int); +extern uintptr_t get_resolver_canary (void); + +static int +do_test (void) +{ + /* compute() returns its argument + SENTINEL iff the resolver picked + impl_ok, which it does whenever the canary check at the resolver + prologue / epilogue did not abort. */ + TEST_COMPARE (compute (1), 1 + SENTINEL); + + /* Silent-variant check: the canary the resolver loaded must match the + canary do_test reads. On global-var ABIs, it checks if the stack + protector cookie is properly initialised. */ + uintptr_t resolver_canary = get_resolver_canary (); + uintptr_t main_canary = STACK_CHK_GUARD; + TEST_VERIFY (resolver_canary != 0); + TEST_COMPARE (resolver_canary, main_canary); + + return 0; +} + +#include diff --git a/elf/tst-ifunc-resolver-protector.c b/elf/tst-ifunc-resolver-protector.c new file mode 100644 index 00000000000..3e085e55091 --- /dev/null +++ b/elf/tst-ifunc-resolver-protector.c @@ -0,0 +1,42 @@ +/* Check that a stack-protector-instrumented IFUNC resolver works in a + dynamically-linked binary. BZ #27582. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* Dynamic counterpart to tst-ifunc-resolver-protector-static. + + The dynamic linker calls security_init() before the main relocation + loop, so a stack-protected resolver fired from startup should observe + an initialised canary -- but having a regression test guards against + any future reordering that would break that invariant. */ + +#include + +#define SENTINEL 0x5A5A1234 + +extern int compute (int); +extern int get_resolver_ran (void); + +static int +do_test (void) +{ + TEST_COMPARE (compute (1), 1 + SENTINEL); + TEST_VERIFY (get_resolver_ran () != 0); + return 0; +} + +#include diff --git a/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h b/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h index 9f772614bf6..26fcd1977ea 100644 --- a/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h +++ b/sysdeps/aarch64/multiarch/dl-symbol-redir-ifunc.h @@ -21,5 +21,9 @@ asm ("memset = __memset_generic"); asm ("strlen = __strlen_generic"); +#ifndef SHARED +asm ("memcpy = __memcpy_generic"); +asm ("memmove = __memmove_generic"); +#endif #endif diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h index 15c46598539..24529db8a1e 100644 --- a/sysdeps/generic/ldsodefs.h +++ b/sysdeps/generic/ldsodefs.h @@ -1197,10 +1197,15 @@ void __tls_init_tp (void) attribute_hidden; void __libc_setup_tls (void); # if ENABLE_STATIC_PIE -/* Relocate static executable with PIE. */ +/* _dl_relocate_static_pie runs every relocation except IRELATIVE. The + second entry point _dl_relocate_static_pie_ifunc must be invoked + afterwards -- but only once the TCB and the stack-protector canary + are usable -- to fire the IFUNC resolvers. */ extern void _dl_relocate_static_pie (void) attribute_hidden; +extern void _dl_relocate_static_pie_ifunc (void) attribute_hidden; # else # define _dl_relocate_static_pie() +# define _dl_relocate_static_pie_ifunc() # endif #endif diff --git a/sysdeps/generic/libc-start.h b/sysdeps/generic/libc-start.h index a36f1fa71d5..8896198fb2e 100644 --- a/sysdeps/generic/libc-start.h +++ b/sysdeps/generic/libc-start.h @@ -20,12 +20,11 @@ #define _LIBC_START_H #ifndef SHARED -/* By default we perform STT_GNU_IFUNC resolution *before* TLS - initialization, and this means you cannot, without machine - knowledge, access TLS from an IFUNC resolver. */ +/* Static startup runs ARCH_SETUP_TLS (and writes the stack-protector + canary) before ARCH_SETUP_IREL, so IFUNC resolvers can read TLS and + stack-protector instrumented resolvers do not fault. */ +#define ARCH_SETUP_TLS() __libc_setup_tls () #define ARCH_SETUP_IREL() apply_irel () -#define ARCH_SETUP_TLS() __libc_setup_tls () -#define ARCH_APPLY_IREL() #endif /* ! SHARED */ #endif /* _LIBC_START_H */ diff --git a/sysdeps/loongarch/lp64/multiarch/dl-symbol-redir-ifunc.h b/sysdeps/loongarch/lp64/multiarch/dl-symbol-redir-ifunc.h index 8af00d2846b..1cd204c2f69 100644 --- a/sysdeps/loongarch/lp64/multiarch/dl-symbol-redir-ifunc.h +++ b/sysdeps/loongarch/lp64/multiarch/dl-symbol-redir-ifunc.h @@ -23,6 +23,8 @@ asm ("memset = __memset_aligned"); asm ("memcmp = __memcmp_aligned"); asm ("strlen = __strlen_aligned"); +asm ("memcpy = __memcpy_unaligned"); +asm ("memmove = __memmove_unaligned"); #endif #endif diff --git a/sysdeps/unix/sysv/linux/aarch64/libc-start.h b/sysdeps/unix/sysv/linux/aarch64/libc-start.h index 4ccd13741bc..53683ee511b 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libc-start.h +++ b/sysdeps/unix/sysv/linux/aarch64/libc-start.h @@ -76,7 +76,6 @@ aarch64_libc_setup_tls (void) # define ARCH_SETUP_IREL() apply_irel () # define ARCH_SETUP_TLS() aarch64_libc_setup_tls () -# define ARCH_APPLY_IREL() #endif /* ! SHARED */ #endif /* _LIBC_START_H */ diff --git a/sysdeps/unix/sysv/linux/powerpc/libc-start.h b/sysdeps/unix/sysv/linux/powerpc/libc-start.h deleted file mode 100644 index 67c361a83c1..00000000000 --- a/sysdeps/unix/sysv/linux/powerpc/libc-start.h +++ /dev/null @@ -1,30 +0,0 @@ -/* PowerPC definitions for libc main startup. - Copyright (C) 2017-2026 Free Software Foundation, Inc. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -#ifndef _LIBC_START_H -#define _LIBC_START_H - -#ifndef SHARED -/* IREL{,A} must happen after TCB initialization in order to allow IFUNC - resolvers to read TCB fields, e.g. hwcap and at_platform. */ -#define ARCH_SETUP_IREL() -#define ARCH_SETUP_TLS() __libc_setup_tls () -#define ARCH_APPLY_IREL() apply_irel () -#endif /* ! SHARED */ - -#endif /* _LIBC_START_H */ diff --git a/sysdeps/x86_64/libc-start.h b/sysdeps/x86_64/libc-start.h index 7e38fdef873..036e0e472a8 100644 --- a/sysdeps/x86_64/libc-start.h +++ b/sysdeps/x86_64/libc-start.h @@ -18,7 +18,6 @@ #ifndef SHARED # define ARCH_SETUP_IREL() apply_irel () -# define ARCH_APPLY_IREL() # ifdef __CET__ /* Get CET features enabled in the static executable. */ diff --git a/sysdeps/x86_64/multiarch/dl-symbol-redir-ifunc.h b/sysdeps/x86_64/multiarch/dl-symbol-redir-ifunc.h index b607e525f2c..1f3ca20307c 100644 --- a/sysdeps/x86_64/multiarch/dl-symbol-redir-ifunc.h +++ b/sysdeps/x86_64/multiarch/dl-symbol-redir-ifunc.h @@ -54,6 +54,25 @@ asm ("memcmp = " HAVE_MEMCMP_IFUNC_GENERIC); asm ("strlen = " HAVE_STRCMP_IFUNC_GENERIC); +#if MINIMUM_X86_ISA_LEVEL >= 4 +# define HAVE_MEMCPY_IFUNC_GENERIC "__memcpy_evex_unaligned" +# define HAVE_MEMMOVE_IFUNC_GENERIC "__memmove_evex_unaligned" +# define HAVE_MEMPCPY_IFUNC_GENERIC "__mempcpy_evex_unaligned" +#elif MINIMUM_X86_ISA_LEVEL == 3 +# define HAVE_MEMCPY_IFUNC_GENERIC "__memcpy_avx_unaligned" +# define HAVE_MEMMOVE_IFUNC_GENERIC "__memmove_avx_unaligned" +# define HAVE_MEMPCPY_IFUNC_GENERIC "__mempcpy_avx_unaligned" +#else +# define HAVE_MEMCPY_IFUNC_GENERIC "__memcpy_sse2_unaligned" +# define HAVE_MEMMOVE_IFUNC_GENERIC "__memmove_sse2_unaligned" +# define HAVE_MEMPCPY_IFUNC_GENERIC "__mempcpy_sse2_unaligned" +#endif + +asm ("memcpy = " HAVE_MEMCPY_IFUNC_GENERIC); +asm ("memmove = " HAVE_MEMMOVE_IFUNC_GENERIC); +asm ("mempcpy = " HAVE_MEMPCPY_IFUNC_GENERIC); +asm ("__mempcpy = " HAVE_MEMPCPY_IFUNC_GENERIC); + #endif /* SHARED */ #endif