From patchwork Mon Oct 20 15:50:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yao Zihong X-Patchwork-Id: 122290 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 988823858C53 for ; Mon, 20 Oct 2025 15:59:41 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 988823858C53 X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from cstnet.cn (smtp21.cstnet.cn [159.226.251.21]) by sourceware.org (Postfix) with ESMTPS id D76FC3858C25 for ; Mon, 20 Oct 2025 15:52:26 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D76FC3858C25 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=isrc.iscas.ac.cn Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=isrc.iscas.ac.cn ARC-Filter: OpenARC Filter v1.0.0 sourceware.org D76FC3858C25 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=159.226.251.21 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1760975547; cv=none; b=pHlF51yMuTcg4fNCtvn41fjme2y5nNIVXhQVjng9ivNX797W99Wx++iSc7BZ4fibjCnuyksU+KLNKNHDMHxZeOFQFrHQC0pUMfhGryOXPJomDaKQ0OpG2qOlxfV9GNkJeWkMSVp9AVGo8DF2v/ayi9Q1kIMLWxPCmRB1Oap7/5Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1760975547; c=relaxed/simple; bh=5B6wr77D7wBxvu1IRn7wmlBwqa6/ObOiKxqseoZNeCk=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=wotRhDi9VyUCJBH2dvg4LbiFS2HNM7q2+9n9qQvba2/Bd+SnNEhNuRZQxEIIndZb4GG9MCUF3gp/GedyYDVwaDjkWeg7FIR1C+nXsClqSZ/pldY3ERGPPRNWR6j7K8tiTEI2yGxGInCBbx7Z4xuYDU0zm8BqA/lcwXHNBVIVrvM= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from Mobilestation.localdomain (unknown [183.6.59.216]) by APP-01 (Coremail) with SMTP id qwCowAC3EaRwWvZoOCsdEg--.25622S3; Mon, 20 Oct 2025 23:51:20 +0800 (CST) From: Yao Zihong To: libc-alpha@sourceware.org Cc: adhemerval.zanella@linaro.org, andrew@sifive.com, schwab@linux-m68k.org, bergner@linux.ibm.com, jlaw@ventanamicro.com, palmer@dabbelt.com, vineetg@rivosinc.com, jerry.shih@sifive.com, zhangyin2018@iscas.ac.cn, zihong.plct@isrc.iscas.ac.cn, zihongyao@outlook.com, Jeff Law Subject: [PATCH v1 1/2] riscv: Add RVV memset for both multiarch and non-multiarch builds Date: Mon, 20 Oct 2025 23:50:28 +0800 Message-ID: <20251020155101.83064-2-zihong.plct@isrc.iscas.ac.cn> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20251020155101.83064-1-zihong.plct@isrc.iscas.ac.cn> References: <20251020155101.83064-1-zihong.plct@isrc.iscas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowAC3EaRwWvZoOCsdEg--.25622S3 X-Coremail-Antispam: 1UD129KBjvAXoW3tr4fWr4xCr45uF13WrykZrb_yoW8Gw1UXo WxWF43JF47Kr1DCr4rC3yUJ3srWw13WrW0qa15XaykJr1kJF1rCF4Syas8XrWakr15WFWf AFWxtFW3JF4fWFn3n29KB7ZKAUJUUUU8529EdanIXcx71UUUUU7v73VFW2AGmfu7bjvjm3 AaLaJ3UjIYCTnIWjp_UUUO77AC8VAFwI0_Wr0E3s1l1xkIjI8I6I8E6xAIw20EY4v20xva j40_Wr0E3s1l1IIY67AEw4v_Jr0_Jr4l82xGYIkIc2x26280x7IE14v26r18M28IrcIa0x kI8VCY1x0267AKxVW8JVW5JwA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK021l84AC jcxK6xIIjxv20xvE14v26r1j6r1xM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26r4j6F4UM2 8EF7xvwVC2z280aVAFwI0_Jr0_Gr1l84ACjcxK6I8E87Iv6xkF7I0E14v26r4j6r4UJwAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUGVWUXwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWU JVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67 kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY 6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0x vEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVj vjDU0xZFpf9x0JUfKs8UUUUU= X-Originating-IP: [183.6.59.216] X-CM-SenderInfo: p2lk00vjoszunw6l223fol2u1dvotugofq/ X-Spam-Status: No, score=-10.8 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org This patch introduces an RVV-optimized memset for RISC-V and enables it for both multiarch (IFUNC) and non-multiarch builds. * sysdeps layout - Add the RVV implementation under sysdeps/riscv/rvv. - Use Implies in sysdeps/rv{32,64}/rvv to pull required subdirs. * toolchain/feature detection - Extend preconfigure{,.ac} to detect __riscv_vector and, when present, include the rvv sysdeps directory in the search path. When the assembler/toolchain lacks vector support, the sysdeps selection stays unchanged and no RVV objects are built. * multiarch/IFUNC - Provide a generic re-export (__memset_generic) and a vector wrapper (__memset_vector) under sysdeps/riscv/multiarch; the latter includes the RVV body from sysdeps/riscv/rvv. - Register memset in ifunc-impl-list.c and select the RVV variant when RISCV_HWPROBE_KEY_IMA_EXT_0 reports V. Co-authored-by: Jerry Shih Co-authored-by: Jeff Law Signed-off-by: Yao Zihong --- sysdeps/riscv/configure | 0 .../riscv/multiarch/dl-symbol-redir-ifunc.h | 27 +++++++++ sysdeps/riscv/multiarch/memset-generic.c | 26 ++++++++ sysdeps/riscv/multiarch/memset_vector.S | 24 ++++++++ sysdeps/riscv/preconfigure | 9 +++ sysdeps/riscv/preconfigure.ac | 9 +++ sysdeps/riscv/rv32/rvv/Implies | 2 + sysdeps/riscv/rv64/rvv/Implies | 2 + sysdeps/riscv/rvv/memset.S | 53 +++++++++++++++++ .../unix/sysv/linux/riscv/multiarch/Makefile | 3 + .../linux/riscv/multiarch/ifunc-impl-list.c | 11 ++++ .../unix/sysv/linux/riscv/multiarch/memset.c | 59 +++++++++++++++++++ 12 files changed, 225 insertions(+) mode change 100644 => 100755 sysdeps/riscv/configure create mode 100644 sysdeps/riscv/multiarch/dl-symbol-redir-ifunc.h create mode 100644 sysdeps/riscv/multiarch/memset-generic.c create mode 100644 sysdeps/riscv/multiarch/memset_vector.S mode change 100644 => 100755 sysdeps/riscv/preconfigure create mode 100644 sysdeps/riscv/rv32/rvv/Implies create mode 100644 sysdeps/riscv/rv64/rvv/Implies create mode 100644 sysdeps/riscv/rvv/memset.S create mode 100644 sysdeps/unix/sysv/linux/riscv/multiarch/memset.c diff --git a/sysdeps/riscv/configure b/sysdeps/riscv/configure old mode 100644 new mode 100755 diff --git a/sysdeps/riscv/multiarch/dl-symbol-redir-ifunc.h b/sysdeps/riscv/multiarch/dl-symbol-redir-ifunc.h new file mode 100644 index 0000000000..d718c2f9d4 --- /dev/null +++ b/sysdeps/riscv/multiarch/dl-symbol-redir-ifunc.h @@ -0,0 +1,27 @@ +/* Symbol rediretion for loader/static initialization code. + Copyright (C) 2025 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _DL_IFUNC_GENERIC_H +#define _DL_IFUNC_GENERIC_H + +#ifndef SHARED +asm ("memset = __memset_generic"); +asm ("memcpy = __memcpy_generic"); +#endif + +#endif diff --git a/sysdeps/riscv/multiarch/memset-generic.c b/sysdeps/riscv/multiarch/memset-generic.c new file mode 100644 index 0000000000..c93bb43c8f --- /dev/null +++ b/sysdeps/riscv/multiarch/memset-generic.c @@ -0,0 +1,26 @@ +/* Re-include the default memset implementation. + Copyright (C) 2025 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#if IS_IN(libc) +# define MEMSET __memset_generic +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(x) +#endif +#include diff --git a/sysdeps/riscv/multiarch/memset_vector.S b/sysdeps/riscv/multiarch/memset_vector.S new file mode 100644 index 0000000000..4c8b198187 --- /dev/null +++ b/sysdeps/riscv/multiarch/memset_vector.S @@ -0,0 +1,24 @@ +/* RVV versions memset. RISC-V version. + Copyright (C) 2025 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN(libc) +# define MEMSET __memset_vector +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) +#include +#endif diff --git a/sysdeps/riscv/preconfigure b/sysdeps/riscv/preconfigure old mode 100644 new mode 100755 index a5de5ccb7d..ca6fdf5856 --- a/sysdeps/riscv/preconfigure +++ b/sysdeps/riscv/preconfigure @@ -7,6 +7,7 @@ riscv*) flen=`$CC $CFLAGS $CPPFLAGS -E -dM -xc /dev/null | sed -n 's/^#define __riscv_flen \(.*\)/\1/p'` float_abi=`$CC $CFLAGS $CPPFLAGS -E -dM -xc /dev/null | sed -n 's/^#define __riscv_float_abi_\([^ ]*\) .*/\1/p'` atomic=`$CC $CFLAGS $CPPFLAGS -E -dM -xc /dev/null | grep '#define __riscv_atomic' | cut -d' ' -f2` + vector=`$CC $CFLAGS $CPPFLAGS -E -dM -xc /dev/null | grep '#define __riscv_vector' | cut -d' ' -f2` case "$xlen" in 64 | 32) @@ -55,6 +56,14 @@ riscv*) ;; esac + case "$vector" in + __riscv_vector) + float_machine=rvv + ;; + *) + ;; + esac + base_machine=riscv machine=riscv/rv$xlen/$float_machine diff --git a/sysdeps/riscv/preconfigure.ac b/sysdeps/riscv/preconfigure.ac index a5c30e0dbf..5e143ac760 100644 --- a/sysdeps/riscv/preconfigure.ac +++ b/sysdeps/riscv/preconfigure.ac @@ -7,6 +7,7 @@ riscv*) flen=`$CC $CFLAGS $CPPFLAGS -E -dM -xc /dev/null | sed -n 's/^#define __riscv_flen \(.*\)/\1/p'` float_abi=`$CC $CFLAGS $CPPFLAGS -E -dM -xc /dev/null | sed -n 's/^#define __riscv_float_abi_\([^ ]*\) .*/\1/p'` atomic=`$CC $CFLAGS $CPPFLAGS -E -dM -xc /dev/null | grep '#define __riscv_atomic' | cut -d' ' -f2` + vector=`$CC $CFLAGS $CPPFLAGS -E -dM -xc /dev/null | grep '#define __riscv_vector' | cut -d' ' -f2` case "$xlen" in 64 | 32) @@ -55,6 +56,14 @@ riscv*) ;; esac + case "$vector" in + __riscv_vector) + float_machine=rvv + ;; + *) + ;; + esac + base_machine=riscv machine=riscv/rv$xlen/$float_machine diff --git a/sysdeps/riscv/rv32/rvv/Implies b/sysdeps/riscv/rv32/rvv/Implies new file mode 100644 index 0000000000..25ce1df222 --- /dev/null +++ b/sysdeps/riscv/rv32/rvv/Implies @@ -0,0 +1,2 @@ +riscv/rv32/rvd +riscv/rvv diff --git a/sysdeps/riscv/rv64/rvv/Implies b/sysdeps/riscv/rv64/rvv/Implies new file mode 100644 index 0000000000..9993bb30e3 --- /dev/null +++ b/sysdeps/riscv/rv64/rvv/Implies @@ -0,0 +1,2 @@ +riscv/rv64/rvd +riscv/rvv diff --git a/sysdeps/riscv/rvv/memset.S b/sysdeps/riscv/rvv/memset.S new file mode 100644 index 0000000000..1f5f9a9f62 --- /dev/null +++ b/sysdeps/riscv/rvv/memset.S @@ -0,0 +1,53 @@ +/* RVV versions memset. RISC-V version. + Copyright (C) 2025 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +#ifndef MEMSET +# define MEMSET memset +#endif + +#define dst a0 +#define value a1 +#define num a2 + +#define ivl a3 +#define dst_ptr a5 + +#define ELEM_LMUL_SETTING m8 +#define vdata v0 + +ENTRY (MEMSET) +.option push +.option arch, +v + mv dst_ptr, dst + + vsetvli ivl, num, e8, ELEM_LMUL_SETTING, ta, ma + vmv.v.x vdata, value +L(loop): + vse8.v vdata, (dst_ptr) + sub num, num, ivl + add dst_ptr, dst_ptr, ivl + vsetvli ivl, num, e8, ELEM_LMUL_SETTING, ta, ma + bnez num, L(loop) + + ret +.option pop +END (MEMSET) +libc_hidden_builtin_def (memset) diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile index fcef5659d4..de8024b86d 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile @@ -3,6 +3,9 @@ sysdep_routines += \ memcpy \ memcpy-generic \ memcpy_noalignment \ + memset \ + memset-generic \ + memset_vector \ # sysdep_routines CFLAGS-memcpy_noalignment.c += -mno-strict-align diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c index 1c1deca8f6..b4defac9c4 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c @@ -27,6 +27,7 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, size_t i = max; bool fast_unaligned = false; + bool rvv_ext = false; struct riscv_hwprobe pair = { .key = RISCV_HWPROBE_KEY_CPUPERF_0 }; if (__riscv_hwprobe (&pair, 1, 0, NULL, 0) == 0 @@ -34,10 +35,20 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, == RISCV_HWPROBE_MISALIGNED_FAST) fast_unaligned = true; + struct riscv_hwprobe ext_pair = { .key = RISCV_HWPROBE_KEY_IMA_EXT_0 }; + if (__riscv_hwprobe (&ext_pair, 1, 0, NULL, 0) == 0 + && (ext_pair.value & RISCV_HWPROBE_IMA_V)) + rvv_ext = true; + IFUNC_IMPL (i, name, memcpy, IFUNC_IMPL_ADD (array, i, memcpy, fast_unaligned, __memcpy_noalignment) IFUNC_IMPL_ADD (array, i, memcpy, 1, __memcpy_generic)) + IFUNC_IMPL (i, name, memset, + IFUNC_IMPL_ADD (array, i, memset, rvv_ext, + __memset_vector) + IFUNC_IMPL_ADD (array, i, memset, 1, __memset_generic)) + return 0; } diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/memset.c b/sysdeps/unix/sysv/linux/riscv/multiarch/memset.c new file mode 100644 index 0000000000..8c1362e064 --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/memset.c @@ -0,0 +1,59 @@ +/* Multiple versions of memset. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2025 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +/* Redefine memset so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef memset +# define memset __redirect_memset +# include +# include +# include +# include +# include +# include + +extern __typeof (__redirect_memset) __libc_memset; + +extern __typeof (__redirect_memset) __memset_generic attribute_hidden; +extern __typeof (__redirect_memset) __memset_vector attribute_hidden; + +static inline __typeof (__redirect_memset) * +select_memset_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) +{ + unsigned long long v; + + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_IMA_EXT_0, &v) == 0 + && (v & RISCV_HWPROBE_IMA_V) == RISCV_HWPROBE_IMA_V) + return __memset_vector; + + return __memset_generic; +} + +riscv_libc_ifunc (__libc_memset, select_memset_ifunc); + +# undef memset +strong_alias (__libc_memset, memset); +# ifdef SHARED +__hidden_ver1 (memset, __GI_memset, __redirect_memset) + __attribute__ ((visibility ("hidden"))) __attribute_copy__ (memset); +# endif +#else +# include +#endif