From patchwork Tue Mar 3 02:58:44 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: daichengrong X-Patchwork-Id: 130981 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [127.0.0.1]) by sourceware.org (Postfix) with ESMTP id A68854BA2E16 for ; Tue, 3 Mar 2026 02:59:30 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A68854BA2E16 X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from cstnet.cn (smtp25.cstnet.cn [159.226.251.25]) by sourceware.org (Postfix) with ESMTPS id 220E44BA2E0B for ; Tue, 3 Mar 2026 02:58:54 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 220E44BA2E0B Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=iscas.ac.cn Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=iscas.ac.cn ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 220E44BA2E0B Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=159.226.251.25 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1772506736; cv=none; b=Ff+gcyF96wTA34E+TVpv5MgZf47EKqU7wQ/g4MVZAvF0nYtIeJLngzUzySE17xbGx1uagqP1Wxl2URFLM++mAHhEv0jgKwhiph+a9AMLHb+lEy+yFLfLXNCSLKDDedm0984tlrPVdtjdLGnCh0bDXpN8jDW0czcV0xN1QGD1qmo= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1772506736; c=relaxed/simple; bh=y05Bjfw+kkZlQaQH9PbdeOackadlkPZWl244fDviG5Q=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=XzN8TaqeQJVTkfrZuLK1dhfkgYNl2nMySJWUi8XY81DQ+XnZ3zbD9PecNGwlYwmAGL/o0eGgH5OtLljwnMdPYsHSsFNs/cwebOnKcPXkWlgBAXdScMKs3G7vNjnqul7UCAyezQCxVQfrgd+Gwga1V0lCkDNWd7stG73SnqkXPhs= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 220E44BA2E0B Received: from localhost.localdomain (unknown [121.237.245.164]) by APP-05 (Coremail) with SMTP id zQCowADnfBBoTqZpogeOCQ--.1032S2; Tue, 03 Mar 2026 10:58:48 +0800 (CST) From: daichengrong To: libc-alpha@sourceware.org Cc: daichengrong Subject: [PATCH] riscv: Add RVV strrchr Date: Tue, 3 Mar 2026 10:58:44 +0800 Message-Id: <20260303025844.113756-1-daichengrong@iscas.ac.cn> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-CM-TRANSID: zQCowADnfBBoTqZpogeOCQ--.1032S2 X-Coremail-Antispam: 1UD129KBjvJXoW3KF47Zr1kWr15tFyDZryrWFg_yoWDCw13pa 95C3W5Gw1rXw4xKryxKF1Yq3W5GrZ5Jr15KryY9w4Utw4vq3yxXFZIqw1rWFZ5J3yrC3y3 ZFn8uFyDuF18A3DanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUkq14x267AKxVWUJVW8JwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK02 1l84ACjcxK6xIIjxv20xvE14v26ryj6F1UM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26F4j 6r4UJwA2z4x0Y4vEx4A2jsIE14v26F4UJVW0owA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_Gc CE3s1le2I262IYc4CY6c8Ij28IcVAaY2xG8wAqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E 2Ix0cI8IcVAFwI0_Jrv_JF1lYx0Ex4A2jsIE14v26F4j6r4UJwAm72CE4IkC6x0Yz7v_Jr 0_Gr1lF7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7MxkIecxEwVAFwVW8 JwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r 1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jrv_JF1lIxkGc2Ij 64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Jr 0_Gr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF 0xvEx4A2jsIEc7CjxVAFwI0_Jr0_GrUvcSsGvfC2KfnxnUUI43ZEXa7VUbivtJUUUUU== X-Originating-IP: [121.237.245.164] X-CM-SenderInfo: pgdluxxhqj201qj6x2xfdvhtffof0/ X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_PASS, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org This patch adds an RVV-optimized implementation of strrchr for RISC-V. Benchmark results show no noticeable regression on small inputs, while delivering clear performance improvements on larger inputs. Signed-off-by: daichengrong --- sysdeps/riscv/multiarch/strrchr-generic.c | 28 ++++ sysdeps/riscv/multiarch/strrchr-vector.S | 26 ++++ sysdeps/riscv/rvv/strrchr.S | 126 ++++++++++++++++++ .../unix/sysv/linux/riscv/multiarch/Makefile | 3 + .../linux/riscv/multiarch/ifunc-impl-list.c | 5 + .../unix/sysv/linux/riscv/multiarch/strrchr.c | 59 ++++++++ 6 files changed, 247 insertions(+) create mode 100644 sysdeps/riscv/multiarch/strrchr-generic.c create mode 100644 sysdeps/riscv/multiarch/strrchr-vector.S create mode 100644 sysdeps/riscv/rvv/strrchr.S create mode 100644 sysdeps/unix/sysv/linux/riscv/multiarch/strrchr.c diff --git a/sysdeps/riscv/multiarch/strrchr-generic.c b/sysdeps/riscv/multiarch/strrchr-generic.c new file mode 100644 index 0000000000..c61c6e1a03 --- /dev/null +++ b/sysdeps/riscv/multiarch/strrchr-generic.c @@ -0,0 +1,28 @@ +/* Re-include the default strrchr implementation. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#if IS_IN(libc) +# define STRRCHR __strrchr_generic +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(x) +# undef weak_alias +# define weak_alias(x, x2) +# include +#endif diff --git a/sysdeps/riscv/multiarch/strrchr-vector.S b/sysdeps/riscv/multiarch/strrchr-vector.S new file mode 100644 index 0000000000..8fef68eae0 --- /dev/null +++ b/sysdeps/riscv/multiarch/strrchr-vector.S @@ -0,0 +1,26 @@ +/* Re-include the RISC-V RVV based strrchr implementation. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN(libc) +# define STRRCHR __strrchr_vector +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) +# undef weak_alias +# define weak_alias(name, alias) +# include +#endif diff --git a/sysdeps/riscv/rvv/strrchr.S b/sysdeps/riscv/rvv/strrchr.S new file mode 100644 index 0000000000..f1471b2c32 --- /dev/null +++ b/sysdeps/riscv/rvv/strrchr.S @@ -0,0 +1,126 @@ +/* RISC-V RVV zbb based strrchr: find the last instance of a character in a string. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +#ifndef STRRCHR +# define STRRCHR strrchr +#endif + +#define str a0 +#define ch a1 +#define len a2 +#define in a3 +#define match a4 +#define end a5 +#define vmask v0 +#define vmask_t v0.t +#define vdata v8 +#define vmatch_null v16 +#define vmatch_ch v24 + +#define vindex v16 +#define vmax v24 + +#define chr_hit t1 +#define null_hit t2 + +#define cur_vl t0 +#define tmp t0 + +ENTRY (STRRCHR) +.option push +.option arch, +v +.option arch, +zbb + lbu tmp, (str) + beq tmp, ch, L(hit_0) + beqz ch, L(search_null) + add tmp, str, -1 +L(start): + mv match, tmp + mv in, str + mv cur_vl, zero +L(loop): + add str, str, cur_vl + vsetvli cur_vl, zero, e8, m4, ta, ma + vle8ff.v vdata, (str) + vmseq.vx vmatch_null, vdata, zero + vmseq.vx vmatch_ch, vdata, ch + vfirst.m null_hit, vmatch_null + vfirst.m chr_hit, vmatch_ch + + bgez null_hit, L(finish) + csrr cur_vl, vl + bgez chr_hit, L(update) + j L(loop) +L(update): + add match, str, chr_hit + j L(loop) + +L(finish): + mv tmp, zero + bltz chr_hit, L(no_update) + bgt chr_hit, null_hit, L(no_update) + sub tmp, str, match + add tmp, tmp, chr_hit +L(no_update): + add match, match, tmp + blt match, in, L(not_found) + + add end, str, null_hit + sub len, end, match + vsetvli cur_vl, zero, e8, m4, ta, ma + min len, len, cur_vl + + vsetvli zero, len, e8, m4, ta, ma + vle8.v vdata, (match) + vmseq.vx vmask, vdata, ch + vsetvli zero, zero, e16, m8, ta, ma + vid.v vindex + vmv.s.x vmax, zero + vredmaxu.vs vmax, vindex, vmax, vmask_t + vmv.x.s tmp, vmax + add str, match, tmp + ret + +L(hit_0): + beqz ch, L(ret_0) + mv tmp, str + j L(start) + +L(search_null): + mv cur_vl, zero +L(search_null_loop): + add str, str, cur_vl + vsetvli cur_vl, zero, e8, m8, ta, ma + vle8ff.v vdata, (str) + vmseq.vx vmatch_null, vdata, zero + vfirst.m null_hit, vmatch_null + bltz null_hit, L(search_null_loop) + add str, str, null_hit + ret + +L(not_found): + li a0, 0 +L(ret_0): + ret +.option pop +END (STRRCHR) +weak_alias (STRRCHR, rindex) +libc_hidden_builtin_def (strrchr) diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile index a865090a53..cd59fb39f3 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile @@ -6,6 +6,9 @@ sysdep_routines += \ memset \ memset-generic \ memset-vector \ + strrchr \ + strrchr-generic \ + strrchr-vector \ # sysdep_routines CFLAGS-memcpy_noalignment.c += -mno-strict-align diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c index a3b5731411..dc39335b76 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c @@ -53,5 +53,10 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, __memset_vector) IFUNC_IMPL_ADD (array, i, memset, 1, __memset_generic)) + IFUNC_IMPL (i, name, strrchr, + IFUNC_IMPL_ADD (array, i, strrchr, rvv_enabled, + __strrchr_vector) + IFUNC_IMPL_ADD (array, i, strrchr, 1, __strrchr_generic)) + return 0; } diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/strrchr.c b/sysdeps/unix/sysv/linux/riscv/multiarch/strrchr.c new file mode 100644 index 0000000000..3532af0dc0 --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/strrchr.c @@ -0,0 +1,59 @@ +/* Multiple versions of strrchr. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +/* Redefine strrchr so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef strrchr +# define strrchr __redirect_strrchr +# include +# include +# include +# include +# include + +extern __typeof (__redirect_strrchr) __libc_strrchr; + +extern __typeof (__redirect_strrchr) __strrchr_generic attribute_hidden; +extern __typeof (__redirect_strrchr) __strrchr_vector attribute_hidden; + +static inline __typeof (__redirect_strrchr) * +select_strrchr_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) +{ + unsigned long long v; + + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_IMA_EXT_0, &v) == 0 + && (v & RISCV_HWPROBE_IMA_V) == RISCV_HWPROBE_IMA_V) + return __strrchr_vector; + return __strrchr_generic; +} + +riscv_libc_ifunc (__libc_strrchr, select_strrchr_ifunc); + +# undef strrchr +# undef rindex +strong_alias (__libc_strrchr, strrchr); +weak_alias (strrchr, rindex); +# ifdef SHARED +__hidden_ver1 (strrchr, __GI_strrchr, __redirect_strrchr) + __attribute__ ((visibility ("hidden"))) __attribute_copy__ (strrchr); +# endif +#else +# include +#endif