From patchwork Fri May 8 08:06:23 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: daichengrong X-Patchwork-Id: 134670 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 1B6624BA2E1F for ; Fri, 8 May 2026 08:07:07 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 1B6624BA2E1F X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from cstnet.cn (smtp25.cstnet.cn [159.226.251.25]) by sourceware.org (Postfix) with ESMTPS id CA8D14BA2E07 for ; Fri, 8 May 2026 08:06:32 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org CA8D14BA2E07 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=iscas.ac.cn Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=iscas.ac.cn ARC-Filter: OpenARC Filter v1.0.0 sourceware.org CA8D14BA2E07 Authentication-Results: sourceware.org; arc=none smtp.remote-ip=159.226.251.25 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1778227593; cv=none; b=Bew515boPEKIDxM5CN4ga2At9aV8t9h6+p0hZzQZm4onP8wtK/oQQOyEn8E8Qs8dBkF4LO7ZK5xKpcsL2IjfbpCiA9VOp6gfic4+Y4Uxiim2ICnOuDf8JTkb+28txHk/ujMNSKgnfANUOnln6DhI1XXnAmx/dvRdCFzvGnIgEOo= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1778227593; c=relaxed/simple; bh=eHFgKwMuD3UAwX8l2xI8tyHwGEPDZ3Akd9rKR/QcTco=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=CSwwAQHULBY3QhVF2QSREYLSV0Bik/WyBURa5m6FYdxrJSIsbQbFAPnJdrK5mAQbY84wLO7QSVYnC1CuMjAC0dnNVLi6Sq+opVRe7qubLY05AHZA18dmS6IbaqABEkRc7WSomx/bJIS0zyerZnTmw9sigZtcMfvsbdAihoRfPis= ARC-Authentication-Results: i=1; sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org CA8D14BA2E07 Received: from localhost.localdomain (unknown [121.237.245.103]) by APP-05 (Coremail) with SMTP id zQCowAD3iw+Dmf1pQPixDw--.3520S2; Fri, 08 May 2026 16:06:28 +0800 (CST) From: daichengrong To: libc-alpha@sourceware.org Cc: Jeffrey Law , Adhemerval Zanella Netto , bergner@tenstorrent.com, daichengrong Subject: [PATCH] riscv: add RVV-optimized memcmp Date: Fri, 8 May 2026 16:06:23 +0800 Message-Id: <20260508080623.21107-1-daichengrong@iscas.ac.cn> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-CM-TRANSID: zQCowAD3iw+Dmf1pQPixDw--.3520S2 X-Coremail-Antispam: 1UD129KBjvJXoW3ZF15Jw18ZFWUZFWDAF1DZFb_yoWDCw13pF Z5u3WUKF4kJrWfGrWI93WUW3W3AFWkJr1YkryY9w4UA3yUJw1fWanF9FnxGFykJrWSk3yr uFn8WFyj9FWUAaDanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUvv14x267AKxVWUJVW8JwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK02 1l84ACjcxK6xIIjxv20xvE14v26ryj6F1UM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26F4j 6r4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oV Cq3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0 I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Gr0_Cr1lOx8S6xCaFVCjc4AY6r1j6r 4UM4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwCY1x0262kKe7AKxVWU AVWUtwCY02Avz4vE14v_GFyl42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr 1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE 14v26r126r1DMIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIxAIcVC0I7 IYx2IY6xkF7I0E14v26r1j6r4UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E 87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73Uj IFyTuYvjfUjOzsDUUUU X-Originating-IP: [121.237.245.103] X-CM-SenderInfo: pgdluxxhqj201qj6x2xfdvhtffof0/ X-Spam-Status: No, score=-12.0 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_DNSWL_BLOCKED, SPF_HELO_PASS, SPF_PASS, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org This patch introduces an RVV based implementation of memcmp, leveraging vector instructions to improve data-level parallelism. Signed-off-by: daichengrong --- sysdeps/riscv/multiarch/memcmp-generic.c | 30 ++++ sysdeps/riscv/multiarch/memcmp-vector.S | 30 ++++ sysdeps/riscv/rvv/memcmp.S | 132 ++++++++++++++++++ .../unix/sysv/linux/riscv/multiarch/Makefile | 3 + .../linux/riscv/multiarch/ifunc-impl-list.c | 5 + .../unix/sysv/linux/riscv/multiarch/memcmp.c | 60 ++++++++ 6 files changed, 260 insertions(+) create mode 100644 sysdeps/riscv/multiarch/memcmp-generic.c create mode 100644 sysdeps/riscv/multiarch/memcmp-vector.S create mode 100644 sysdeps/riscv/rvv/memcmp.S create mode 100644 sysdeps/unix/sysv/linux/riscv/multiarch/memcmp.c diff --git a/sysdeps/riscv/multiarch/memcmp-generic.c b/sysdeps/riscv/multiarch/memcmp-generic.c new file mode 100644 index 0000000000..14f6752406 --- /dev/null +++ b/sysdeps/riscv/multiarch/memcmp-generic.c @@ -0,0 +1,30 @@ +/* Re-include the default memcmp implementation. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#if IS_IN(libc) +# define MEMCMP __memcmp_generic +# undef libc_hidden_def +# define libc_hidden_def(x) +# undef weak_alias +# define weak_alias(x, x2) +# undef strong_alias +# define strong_alias(x, x2) +# include +#endif diff --git a/sysdeps/riscv/multiarch/memcmp-vector.S b/sysdeps/riscv/multiarch/memcmp-vector.S new file mode 100644 index 0000000000..b447e945a8 --- /dev/null +++ b/sysdeps/riscv/multiarch/memcmp-vector.S @@ -0,0 +1,30 @@ +/* Re-include the RISC-V RVV based memcmp implementation. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN(libc) +# define MEMCMP __memcmp_vector +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) +# undef libc_hidden_def +# define libc_hidden_def(name) +# undef weak_alias +# define weak_alias(name, alias) +# undef strong_alias +# define strong_alias(name, alias) +# include +#endif diff --git a/sysdeps/riscv/rvv/memcmp.S b/sysdeps/riscv/rvv/memcmp.S new file mode 100644 index 0000000000..c7ee412d67 --- /dev/null +++ b/sysdeps/riscv/rvv/memcmp.S @@ -0,0 +1,132 @@ +/* RISC-V RVV based memcmp. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +#ifndef MEMCMP +# define MEMCMP memcmp +#endif + +#define result a0 + +#define src1 a0 +#define src2 a1 +#define cnt a2 + +#define cnt_rem a2 +#define cnt_rem_2 a5 +#define new_vl t6 + +#define temp t0 + +#define src1_2 a6 +#define src2_2 t3 + +#define vdata1 v0 +#define vdata2 v16 + +#define vdata1_2 v8 +#define vdata2_2 v24 + +#define vmask1 v0 +#define vmask2 v8 + +#define vfirst_index a4 +#define vfirst_index_2 t1 + +#define chr1 a0 +#define chr2 a1 + +ENTRY (MEMCMP) +.option push +.option arch, +v + beqz cnt, L(ret) + and temp, cnt, 0x7F + beqz temp, L(loop_pre) + vsetvli new_vl, temp, e8, m8, ta, ma + vle8.v vdata1, (src1) + vle8.v vdata2, (src2) + + vmsne.vv vmask1, vdata1, v16 + vfirst.m vfirst_index, vmask1 + bgez vfirst_index, L(found) + + beq new_vl, cnt, L(ret) + + add src1, src1, new_vl + add src2, src2, new_vl + sub cnt_rem, cnt, new_vl + +L(loop_pre): + srli cnt_rem_2, cnt_rem, 1 +L(loop): + vsetvli new_vl, cnt_rem_2, e8, m8, ta, ma + + vle8.v vdata1, (src1) + add src1_2, src1, new_vl + vle8.v vdata2, (src2) + add src2_2, src2, new_vl + + vle8.v vdata1_2, (src1_2) + vle8.v vdata2_2, (src2_2) + + vmsne.vv vmask1, vdata1, vdata2 + vmsne.vv vmask2, vdata1_2, vdata2_2 + + vfirst.m vfirst_index, vmask1 + vfirst.m vfirst_index_2, vmask2 + + bgez vfirst_index, L(found1) + bgez vfirst_index_2, L(found2) + + add src1, src1_2, new_vl + add src2, src2_2, new_vl + + sub cnt_rem_2, cnt_rem_2, new_vl + bnez cnt_rem_2, L(loop) +L(ret): + li result, 0 + ret +L(found): +L(found1): + add src1, src1, vfirst_index + add src2, src2, vfirst_index + lbu chr1, (src1) + lbu chr2, (src2) + sub result, chr1, chr2 + ret +L(found2): + add src1_2, src1_2, vfirst_index_2 + add src2_2, src2_2, vfirst_index_2 + lbu chr1, (src1_2) + lbu chr2, (src2_2) + sub result, chr1, chr2 + ret +.option pop +END (MEMCMP) +libc_hidden_builtin_def(memcmp) +#ifdef weak_alias +#undef bcmp +weak_alias (MEMCMP, bcmp) +#endif + +#undef __memcmpeq +strong_alias (MEMCMP, __memcmpeq) +libc_hidden_builtin_def (MEMCMP) +libc_hidden_def (__memcmpeq) diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile index 2a2b90960f..86b045ad45 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile @@ -1,5 +1,8 @@ ifeq ($(subdir),string) sysdep_routines += \ + memcmp \ + memcmp-generic \ + memcmp-vector \ memcpy \ memcpy-generic \ memcpy-vector \ diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c index 2501546665..eb5d376fe9 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c @@ -43,6 +43,11 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, rvv_enabled = true; } + IFUNC_IMPL (i, name, memcmp, + IFUNC_IMPL_ADD (array, i, memcmp, rvv_enabled, + __memcmp_vector) + IFUNC_IMPL_ADD (array, i, memcmp, 1, __memcmp_generic)) + IFUNC_IMPL (i, name, memcpy, IFUNC_IMPL_ADD (array, i, memcpy, rvv_enabled, __memcpy_vector) diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/memcmp.c b/sysdeps/unix/sysv/linux/riscv/multiarch/memcmp.c new file mode 100644 index 0000000000..20e7b046ab --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/memcmp.c @@ -0,0 +1,60 @@ +/* Multiple versions of memcmp. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +/* Redefine memcmp so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef memcmp +# define memcmp __redirect_memcmp +# include +# include +# include +# include +# include + +extern __typeof (__redirect_memcmp) __libc_memcmp; + +extern __typeof (__redirect_memcmp) __memcmp_generic attribute_hidden; +extern __typeof (__redirect_memcmp) __memcmp_vector attribute_hidden; + +static inline __typeof (__redirect_memcmp) * +select_memcmp_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) +{ + unsigned long long v; + + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_IMA_EXT_0, &v) == 0 + && (v & RISCV_HWPROBE_IMA_V) == RISCV_HWPROBE_IMA_V) + return __memcmp_vector; + + return __memcmp_generic; +} + +riscv_libc_ifunc (__libc_memcmp, select_memcmp_ifunc); + +# undef memcmp +# undef bcmp +strong_alias (__libc_memcmp, memcmp); +weak_alias (memcmp, bcmp); +# ifdef SHARED +__hidden_ver1 (memcmp, __GI_memcmp, __redirect_memcmp) + __attribute__ ((visibility ("hidden"))) __attribute_copy__ (memcmp); +# endif +#else +# include +#endif