From patchwork Tue Feb 3 15:05:30 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zihong Yao X-Patchwork-Id: 129488 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [127.0.0.1]) by sourceware.org (Postfix) with ESMTP id 7639B4BA23FE for ; Tue, 3 Feb 2026 15:15:36 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 7639B4BA23FE X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from cstnet.cn (smtp21.cstnet.cn [159.226.251.21]) by sourceware.org (Postfix) with ESMTPS id 0C9F34BA23C7 for ; Tue, 3 Feb 2026 15:14:57 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0C9F34BA23C7 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=isrc.iscas.ac.cn Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=isrc.iscas.ac.cn ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 0C9F34BA23C7 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=159.226.251.21 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1770131699; cv=none; b=b/VpZ/5QYVmSjBteZX0g0Rbzmir6bDHHRsLNbeV/j9FqOvHpkPVxxozRmqr2v80j+3vXZ0kkIARTGI9MnvG0rJ0y+NeS/Xbz1KS45QlOy5po12nxN9gbQ2auU7O8pUg69tKqJ5sTHE9BSyjGEFmess69nb/EPuAa6KT4cjFUwu8= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1770131699; c=relaxed/simple; bh=iMTWjJNJKyc4A2FW8++6o/ML6tZ4zcuJfQToKt9RP/g=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=ibaQV24PkcUjbUlNDoca7E0oSH9lOw8KNaWNEXfjj7BVAT1MZdL/CtranqBL+Sv3yz+MAUqnBTDSvZ9PZLL/mxCAzfsvkJnJL5X1P4P9xIMrDgUgsQms1emDYqG4dVS/Dlzc0VX1AEUat+IxFgU6J2hi9EzqDTNM0HjNT2R2CAw= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 0C9F34BA23C7 Received: from Mobilestation.localdomain (unknown [183.6.59.140]) by APP-01 (Coremail) with SMTP id qwCowADHbmriEIJpZ2EVBw--.24930S8; Tue, 03 Feb 2026 23:14:50 +0800 (CST) From: Yao Zihong To: libc-alpha@sourceware.org Cc: adhemerval.zanella@linaro.org, andrew@sifive.com, schwab@linux-m68k.org, bergner@tenstorrent.com, jlaw@ventanamicro.com, zhangyin2018@iscas.ac.cn, enh@google.com, zihongyao@outlook.com, Yao Zihong , Hau Hsu , Jerry Shih Subject: [PATCH v5 06/18] riscv: Add RVV memmove for multiarch and non-multiarch Date: Tue, 3 Feb 2026 23:05:30 +0800 Message-ID: <20260203151406.27450-7-zihong.plct@isrc.iscas.ac.cn> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260203151406.27450-1-zihong.plct@isrc.iscas.ac.cn> References: <20260203151406.27450-1-zihong.plct@isrc.iscas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADHbmriEIJpZ2EVBw--.24930S8 X-Coremail-Antispam: 1UD129KBjvJXoW3JF17tw1kGr4rAr15tF13urg_yoWftr1DpF Z5uF13Grn7Jr1xGrWSgF1Yv3W3tF4rGF1YkryYkw4UA3yUXryxXFZFvwsYgF95JrW8C3yr Z3Z8uF98CF18ZaDanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_JFI_Gr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr 1UM28EF7xvwVC2z280aVAFwI0_Jr0_Gr1l84ACjcxK6I8E87Iv6xkF7I0E14v26r4j6r4U JwAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCw CI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnI WIevJa73UjIFyTuYvjfUOyIUUUUUU X-Originating-IP: [183.6.59.140] X-CM-SenderInfo: p2lk00vjoszunw6l223fol2u1dvotugofq/ X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org Co-authored-by: Hau Hsu Co-authored-by: Jerry Shih Signed-off-by: Yao Zihong --- sysdeps/riscv/multiarch/memmove-generic.c | 26 +++++++ sysdeps/riscv/multiarch/memmove-vector.S | 24 ++++++ sysdeps/riscv/rvv/memmove.S | 74 +++++++++++++++++++ .../unix/sysv/linux/riscv/multiarch/Makefile | 3 + .../linux/riscv/multiarch/ifunc-impl-list.c | 5 ++ .../unix/sysv/linux/riscv/multiarch/memmove.c | 56 ++++++++++++++ 6 files changed, 188 insertions(+) create mode 100644 sysdeps/riscv/multiarch/memmove-generic.c create mode 100644 sysdeps/riscv/multiarch/memmove-vector.S create mode 100644 sysdeps/riscv/rvv/memmove.S create mode 100644 sysdeps/unix/sysv/linux/riscv/multiarch/memmove.c diff --git a/sysdeps/riscv/multiarch/memmove-generic.c b/sysdeps/riscv/multiarch/memmove-generic.c new file mode 100644 index 0000000000..44c01c887a --- /dev/null +++ b/sysdeps/riscv/multiarch/memmove-generic.c @@ -0,0 +1,26 @@ +/* Re-include the default memmove implementation. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#if IS_IN(libc) +# define MEMMOVE __memmove_generic +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(x) +# include +#endif diff --git a/sysdeps/riscv/multiarch/memmove-vector.S b/sysdeps/riscv/multiarch/memmove-vector.S new file mode 100644 index 0000000000..6bf2d9f614 --- /dev/null +++ b/sysdeps/riscv/multiarch/memmove-vector.S @@ -0,0 +1,24 @@ +/* Re-include the RISC-V RVV based memmove implementation. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN(libc) +# define MEMMOVE __memmove_vector +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) +# include +#endif diff --git a/sysdeps/riscv/rvv/memmove.S b/sysdeps/riscv/rvv/memmove.S new file mode 100644 index 0000000000..5e41b13ec5 --- /dev/null +++ b/sysdeps/riscv/rvv/memmove.S @@ -0,0 +1,74 @@ +/* RISC-V RVV based memmove. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +#ifndef MEMMOVE +# define MEMMOVE memmove +#endif + +#define dst a0 +#define src a1 +#define num a2 + +#define ivl a3 +#define dst_ptr a4 +#define src_backward_ptr a5 +#define dst_backward_ptr a6 + +#define ELEM_LMUL_SETTING m8 +#define vdata v0 + +ENTRY (MEMMOVE) +.option push +.option arch, +v + mv dst_ptr, dst + + /* If src is equal or after dst, all data in src will be loaded before + overwrited for the overlapping case. We could use faster `forward-copy`. */ + bgeu src, dst, L(forward_copy_loop) + add src_backward_ptr, src, num + add dst_backward_ptr, dst, num + /* If dst inside source data range, we need to use `backward_copy_loop` to + handle the overlapping issue. */ + bltu dst, src_backward_ptr, L(backward_copy_loop) +L(forward_copy_loop): + vsetvli ivl, num, e8, ELEM_LMUL_SETTING, ta, ma + + vle8.v vdata, (src) + sub num, num, ivl + add src, src, ivl + vse8.v vdata, (dst_ptr) + add dst_ptr, dst_ptr, ivl + + bnez num, L(forward_copy_loop) + ret +L(backward_copy_loop): + vsetvli ivl, num, e8, ELEM_LMUL_SETTING, ta, ma + + sub src_backward_ptr, src_backward_ptr, ivl + vle8.v vdata, (src_backward_ptr) + sub num, num, ivl + sub dst_backward_ptr, dst_backward_ptr, ivl + vse8.v vdata, (dst_backward_ptr) + bnez num, L(backward_copy_loop) + ret +.option pop +END (MEMMOVE) +libc_hidden_builtin_def (memmove) diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile index 0bf7c8e37d..a0990c9ff1 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile @@ -16,6 +16,9 @@ sysdep_routines += \ memcpy-generic \ memcpy-vector \ memcpy_noalignment \ + memmove \ + memmove-generic \ + memmove-vector \ memset \ memset-generic \ memset-vector \ diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c index 27a7817308..d955f56dba 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c @@ -75,5 +75,10 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, ____memcmpeq_vector) IFUNC_IMPL_ADD (array, i, __memcmpeq, 1, ____memcmpeq_generic)) + IFUNC_IMPL (i, name, memmove, + IFUNC_IMPL_ADD (array, i, memmove, rvv_enabled, + __memmove_vector) + IFUNC_IMPL_ADD (array, i, memmove, 1, __memmove_generic)) + return 0; } diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/memmove.c b/sysdeps/unix/sysv/linux/riscv/multiarch/memmove.c new file mode 100644 index 0000000000..670f108029 --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/memmove.c @@ -0,0 +1,56 @@ +/* Multiple versions of memmove. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +/* Redefine memmove so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef memmove +# define memmove __redirect_memmove +# include +# include +# include +# include +# include + +extern __typeof (__redirect_memmove) __libc_memmove; + +extern __typeof (__redirect_memmove) __memmove_generic attribute_hidden; +extern __typeof (__redirect_memmove) __memmove_vector attribute_hidden; + +static inline __typeof (__redirect_memmove) * +select_memmove_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) +{ + unsigned long long int v; + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_IMA_EXT_0, &v) == 0 + && (v & RISCV_HWPROBE_IMA_V) == RISCV_HWPROBE_IMA_V) + return __memmove_vector; + return __memmove_generic; +} + +riscv_libc_ifunc (__libc_memmove, select_memmove_ifunc); + +# undef memmove +strong_alias (__libc_memmove, memmove); +# ifdef SHARED +__hidden_ver1 (memmove, __GI_memmove, __redirect_memmove) + __attribute__ ((visibility ("hidden"))) __attribute_copy__ (memmove); +# endif +#else +# include +#endif