From patchwork Tue Feb 3 15:05:42 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zihong Yao X-Patchwork-Id: 129497 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [127.0.0.1]) by sourceware.org (Postfix) with ESMTP id C5D654BA23CA for ; Tue, 3 Feb 2026 15:18:35 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C5D654BA23CA X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from cstnet.cn (smtp21.cstnet.cn [159.226.251.21]) by sourceware.org (Postfix) with ESMTPS id 2475E4BA23F8 for ; Tue, 3 Feb 2026 15:15:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2475E4BA23F8 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=isrc.iscas.ac.cn Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=isrc.iscas.ac.cn ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2475E4BA23F8 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=159.226.251.21 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1770131707; cv=none; b=xBwpcfEkGFEGaFiw3//MWGPhLgqiNElYiuaF6/gh2bd3q8y0fRzEwVOwitSQV5cYf/eKp6se/95cwlJ4YclKTq0WIaF/7lSscRNp8i8vfz/z374npwDmbl90IRzrXQc9jSAG+K36KreX1Ez8TudTGtlg2GDTngZWv4NElvkstD4= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1770131707; c=relaxed/simple; bh=oKqnkkKH9aJilFo2sgLnowzG23tZh09zW58MS9hGjSo=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=hPHQAEUYOw/eTLzsYM5xQRoFnF8k3Ge7fg4E9pjmbpjc7Y5s4H1qDPsSlkprBDOSWqJCpSWL+AHsOttPWXGGtGcaIPLobWtn/YBgm3F+Jg45Hw931p98haHPm8DtT6X8EU9q6wk3kTjSPbKGHg16GlHYaDBddeTgu1xl6aE08EI= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2475E4BA23F8 Received: from Mobilestation.localdomain (unknown [183.6.59.140]) by APP-01 (Coremail) with SMTP id qwCowADHbmriEIJpZ2EVBw--.24930S20; Tue, 03 Feb 2026 23:14:58 +0800 (CST) From: Yao Zihong To: libc-alpha@sourceware.org Cc: adhemerval.zanella@linaro.org, andrew@sifive.com, schwab@linux-m68k.org, bergner@tenstorrent.com, jlaw@ventanamicro.com, zhangyin2018@iscas.ac.cn, enh@google.com, zihongyao@outlook.com, Yao Zihong Subject: [PATCH v5 18/18] riscv: Add RVV strrchr for multiarch and non-multiarch Date: Tue, 3 Feb 2026 23:05:42 +0800 Message-ID: <20260203151406.27450-19-zihong.plct@isrc.iscas.ac.cn> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260203151406.27450-1-zihong.plct@isrc.iscas.ac.cn> References: <20260203151406.27450-1-zihong.plct@isrc.iscas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: qwCowADHbmriEIJpZ2EVBw--.24930S20 X-Coremail-Antispam: 1UD129KBjvPXoW5ZFy5Gw4xZFyrGFWDuF45p5X_trWkAoZagF ZrXF42gr18CrW5Cw4DJasrXr4Ygr1jqw4YqaykXwn5K34rCryIv34DWrW3Gr4rWFs5AFy7 trW3JFsrGFn3n29KB7ZKAUJUUUU8529EdanIXcx71UUUUU7v73VFW2AGmfu7bjvjm3AaLa J3UjIYCTnIWjp_UUUOa7AC8VAFwI0_Wr0E3s1l1xkIjI8I6I8E6xAIw20EY4v20xvaj40_ Wr0E3s1l1IIY67AEw4v_Jr0_Jr4l82xGYIkIc2x26280x7IE14v26r126s0DM28IrcIa0x kI8VCY1x0267AKxVW5JVCq3wA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK021l84AC jcxK6xIIjxv20xvE14v26r4j6ryUM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26r4UJVWxJr 1l84ACjcxK6I8E87Iv67AKxVW8JVWxJwA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_Gr1j6F4U JwAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF 4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBI daVFxhVjvjDU0xZFpf9x0JUQFxUUUUUU= X-Originating-IP: [183.6.59.140] X-CM-SenderInfo: p2lk00vjoszunw6l223fol2u1dvotugofq/ X-Spam-Status: No, score=-11.6 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_DNSWL_BLOCKED, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_PASS, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org Signed-off-by: Yao Zihong --- sysdeps/riscv/multiarch/strrchr-generic.c | 28 ++++ sysdeps/riscv/multiarch/strrchr-vector.S | 26 ++++ sysdeps/riscv/rvv/strrchr.S | 123 ++++++++++++++++++ .../unix/sysv/linux/riscv/multiarch/Makefile | 3 + .../linux/riscv/multiarch/ifunc-impl-list.c | 5 + .../unix/sysv/linux/riscv/multiarch/strrchr.c | 59 +++++++++ 6 files changed, 244 insertions(+) create mode 100644 sysdeps/riscv/multiarch/strrchr-generic.c create mode 100644 sysdeps/riscv/multiarch/strrchr-vector.S create mode 100644 sysdeps/riscv/rvv/strrchr.S create mode 100644 sysdeps/unix/sysv/linux/riscv/multiarch/strrchr.c diff --git a/sysdeps/riscv/multiarch/strrchr-generic.c b/sysdeps/riscv/multiarch/strrchr-generic.c new file mode 100644 index 0000000000..c61c6e1a03 --- /dev/null +++ b/sysdeps/riscv/multiarch/strrchr-generic.c @@ -0,0 +1,28 @@ +/* Re-include the default strrchr implementation. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#if IS_IN(libc) +# define STRRCHR __strrchr_generic +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(x) +# undef weak_alias +# define weak_alias(x, x2) +# include +#endif diff --git a/sysdeps/riscv/multiarch/strrchr-vector.S b/sysdeps/riscv/multiarch/strrchr-vector.S new file mode 100644 index 0000000000..8fef68eae0 --- /dev/null +++ b/sysdeps/riscv/multiarch/strrchr-vector.S @@ -0,0 +1,26 @@ +/* Re-include the RISC-V RVV based strrchr implementation. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN(libc) +# define STRRCHR __strrchr_vector +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) +# undef weak_alias +# define weak_alias(name, alias) +# include +#endif diff --git a/sysdeps/riscv/rvv/strrchr.S b/sysdeps/riscv/rvv/strrchr.S new file mode 100644 index 0000000000..f2da8a4b23 --- /dev/null +++ b/sysdeps/riscv/rvv/strrchr.S @@ -0,0 +1,123 @@ +/* RISC-V RVV based strrchr. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +#ifndef STRRCHR +# define STRRCHR strrchr +#endif + +#define str a0 +#define ch a1 +#define blk_base a2 +#define loc_max a3 +#define latest_ptr a4 +#define first_idx a5 +#define len_valid a5 +#define cur_vl a6 + +#define vhit v0 +#define vstr v4 +#define vmask_end v8 +#define vmask_ch v10 +#define vvalid v12 +#define vidx v16 +#define vacc v20 + +#if __riscv_xlen == 64 +# define XLEN_SHIFT 63 +#else +# define XLEN_SHIFT 31 +#endif + +ENTRY (STRRCHR) +.option push +.option arch, +v + beqz ch, L(search_zero) + mv latest_ptr, zero +L(loop): + mv blk_base, str + + vsetvli cur_vl, zero, e8, m2, ta, ma + vle8ff.v vstr, (str) + + vmseq.vi vmask_end, vstr, 0 + vmsbf.m vvalid, vmask_end + vmseq.vx vmask_ch, vstr, ch + vmand.mm vhit, vmask_ch, vvalid + + vfirst.m first_idx, vhit + csrr cur_vl, vl + bltz first_idx, L(no_hit_in_block) + + vsetvli zero, cur_vl, e16, m4, ta, ma + vid.v vidx + vmv.s.x vacc, x0 + /* Compute per-block last hit index (or -1 if none). */ + vredmaxu.vs vacc, vidx, vacc, vhit.t + vmv.x.s loc_max, vacc + j L(loc_max_ready) + +L(no_hit_in_block): + li loc_max, -1 + +L(loc_max_ready): + vsetvli zero, cur_vl, e8, m2, ta, ma + vcpop.m len_valid, vvalid + + bne len_valid, cur_vl, L(tail_block) + bltz loc_max, L(advance) + + add latest_ptr, blk_base, loc_max + +L(advance): + add str, str, cur_vl + j L(loop) + +L(tail_block): + add str, blk_base, loc_max + + /* sign = -1 if loc_max < 0 else 0; then invert to + build selection mask */ + srai loc_max, loc_max, XLEN_SHIFT + and len_valid, loc_max, latest_ptr + xori loc_max, loc_max, -1 + and str, loc_max, str + or str, str, len_valid + ret + +L(search_zero): +L(sz_loop): + vsetvli cur_vl, zero, e8, m4, ta, ma + vle8ff.v vstr, (str) + + vmseq.vi vmask_end, vstr, 0 + vfirst.m first_idx, vmask_end + bltz first_idx, L(sz_advance) + add str, str, first_idx + ret +L(sz_advance): + csrr cur_vl, vl + add str, str, cur_vl + j L(sz_loop) + +.option pop +END (STRRCHR) +weak_alias (STRRCHR, rindex) +libc_hidden_builtin_def (strrchr) diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile index 4e10723636..8e2b1632da 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile @@ -55,6 +55,9 @@ sysdep_routines += \ strnlen \ strnlen-generic \ strnlen-vector \ + strrchr \ + strrchr-generic \ + strrchr-vector \ # sysdep_routines CFLAGS-memcpy_noalignment.c += -mno-strict-align diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c index dba3d852fd..d7db46cf81 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c @@ -135,5 +135,10 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, __strnlen_vector) IFUNC_IMPL_ADD (array, i, strnlen, 1, __strnlen_generic)) + IFUNC_IMPL (i, name, strrchr, + IFUNC_IMPL_ADD (array, i, strrchr, rvv_enabled, + __strrchr_vector) + IFUNC_IMPL_ADD (array, i, strrchr, 1, __strrchr_generic)) + return 0; } diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/strrchr.c b/sysdeps/unix/sysv/linux/riscv/multiarch/strrchr.c new file mode 100644 index 0000000000..3532af0dc0 --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/strrchr.c @@ -0,0 +1,59 @@ +/* Multiple versions of strrchr. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +/* Redefine strrchr so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef strrchr +# define strrchr __redirect_strrchr +# include +# include +# include +# include +# include + +extern __typeof (__redirect_strrchr) __libc_strrchr; + +extern __typeof (__redirect_strrchr) __strrchr_generic attribute_hidden; +extern __typeof (__redirect_strrchr) __strrchr_vector attribute_hidden; + +static inline __typeof (__redirect_strrchr) * +select_strrchr_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) +{ + unsigned long long v; + + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_IMA_EXT_0, &v) == 0 + && (v & RISCV_HWPROBE_IMA_V) == RISCV_HWPROBE_IMA_V) + return __strrchr_vector; + return __strrchr_generic; +} + +riscv_libc_ifunc (__libc_strrchr, select_strrchr_ifunc); + +# undef strrchr +# undef rindex +strong_alias (__libc_strrchr, strrchr); +weak_alias (strrchr, rindex); +# ifdef SHARED +__hidden_ver1 (strrchr, __GI_strrchr, __redirect_strrchr) + __attribute__ ((visibility ("hidden"))) __attribute_copy__ (strrchr); +# endif +#else +# include +#endif