From patchwork Tue Sep 30 00:56:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yao Zihong X-Patchwork-Id: 121020 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 7C2243858D33 for ; Tue, 30 Sep 2025 00:59:58 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 7C2243858D33 X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from cstnet.cn (smtp84.cstnet.cn [159.226.251.84]) by sourceware.org (Postfix) with ESMTPS id AD6B63858D33 for ; Tue, 30 Sep 2025 00:57:39 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org AD6B63858D33 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=isrc.iscas.ac.cn Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=isrc.iscas.ac.cn ARC-Filter: OpenARC Filter v1.0.0 sourceware.org AD6B63858D33 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=159.226.251.84 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1759193860; cv=none; b=wvS5+HjtN3SnFALTsFOApr5O436RRuOWEgsWTcWzs9oaXefVWyNogyaIwXfXgtPrdU7NZAOeLaGJP8SHOoQ3XU5eReasAoTPtpgv2ooON4/UZ63yTGtGwKIDymycNJKMyg7CNR7WW6wV4xuaGwh1g28NpmiwXz8JaL6YG1yL8oI= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1759193860; c=relaxed/simple; bh=4zqvxaN/OoYY9w2nWTrX7nbwYrm2PN2CvFsNKojwQeE=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=RXvQDRGPvYFXcX3BX0YJHlJvQlMhmrJYUhf/t8HMXH3+XNhk/pA0/kIttAdg//nx9rlkAxKTky3M3ROgbObWxGl6808dt4+6u/D5TQVZqsf9/qW77D8lrQx+kiy2J0fgHf5XP5XO4b285+2Sntb8OAIzgDULIzOVNYZcxZLBdXk= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org AD6B63858D33 Received: from Mobilestation.localdomain (unknown [183.6.60.69]) by APP-05 (Coremail) with SMTP id zQCowAAHqhPxKttorL0WCQ--.11769S5; Tue, 30 Sep 2025 08:57:32 +0800 (CST) From: Yao Zihong To: libc-alpha@sourceware.org Cc: bergner@linux.ibm.com, evan@rivosinc.com, jlaw@ventanamicro.com, palmer@dabbelt.com, vineetg@rivosinc.com, zhangyin2018@iscas.ac.cn, zihong.plct@isrc.iscas.ac.cn, zihongyao@outlook.com Subject: [RFC PATCH v2 3/4] riscv: Add hwcaps-subdir support for RVA22U64 and RVA23U64 Date: Tue, 30 Sep 2025 08:56:55 +0800 Message-ID: <20250930005715.95436-4-zihong.plct@isrc.iscas.ac.cn> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250930005715.95436-1-zihong.plct@isrc.iscas.ac.cn> References: <20250930005715.95436-1-zihong.plct@isrc.iscas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: zQCowAAHqhPxKttorL0WCQ--.11769S5 X-Coremail-Antispam: 1UD129KBjvAXoW3Zw1UJrWkZr1xtrWUCFW3ZFb_yoW8XFW5Ko WxW3WSvay0grs5CrWru3WUG3yUur98Wr4xX3Z5JFZ5JF1fGr18CrsYyas7Wr43Kw1Iga1r Ca48GFZYqrWFv3Zrn29KB7ZKAUJUUUU8529EdanIXcx71UUUUU7v73VFW2AGmfu7bjvjm3 AaLaJ3UjIYCTnIWjp_UUUYE7AC8VAFwI0_Wr0E3s1l1xkIjI8I6I8E6xAIw20EY4v20xva j40_Wr0E3s1l1IIY67AEw4v_Jr0_Jr4l82xGYIkIc2x26280x7IE14v26r1rM28IrcIa0x kI8VCY1x0267AKxVW5JVCq3wA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK021l84AC jcxK6xIIjxv20xvE14v26r1j6r1xM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26r4j6F4UM2 8EF7xvwVC2z280aVAFwI0_Jr0_Gr1l84ACjcxK6I8E87Iv6xkF7I0E14v26r4UJVWxJr1l e2I262IYc4CY6c8Ij28IcVAaY2xG8wAqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI 8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE14v26r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwAC jcxG0xvY0x0EwIxGrwACjI8F5VA0II8E6IAqYI8I648v4I1lc7CjxVAaw2AFwI0_Jw0_GF yl42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWU JVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1q6r43MIIYrxkI7V AKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4j 6F4UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJVW8JwCI42 IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7VUbpwZ7UUUUU= = X-Originating-IP: [183.6.60.69] X-CM-SenderInfo: p2lk00vjoszunw6l223fol2u1dvotugofq/ X-Spam-Status: No, score=-11.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, PROLO_LEO1, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org Add glibc-hwcaps subdirs "rva22u64" and "rva23u64" with minimal profile-level wiring, and extend memcpy ifunc gating: - prefer RVV __memcpy_vector at RVA23, - prefer __memcpy_noalignment at RVA20 to RVA22, - keep generic as fallback. Note: the RVV memcpy here is only introduced as an example. It is based on the earlier strcpy_vector proposal from https://inbox.sourceware.org/libc-alpha/20230504074851.38763-1-hau.hsu@sifive.com/, with comments, naming adjusted and '.option' directives added. Signed-off-by: Yao Zihong --- sysdeps/riscv/Makefile | 30 ++++++++-- sysdeps/riscv/get-profile-level.h | 28 +++++++++ sysdeps/riscv/multiarch/memcpy_noalignment.S | 2 +- sysdeps/riscv/multiarch/memcpy_vector.S | 54 +++++++++++++++++ sysdeps/riscv/profile-ifunc-macros.h | 16 +++++ sysdeps/riscv/profile-level.h | 59 ++++++++++++++++++- sysdeps/riscv/rv64/dl-hwcaps-subdirs.c | 12 +++- .../unix/sysv/linux/riscv/multiarch/Makefile | 1 + .../linux/riscv/multiarch/ifunc-impl-list.c | 10 +++- .../unix/sysv/linux/riscv/multiarch/memcpy.c | 8 +++ 10 files changed, 210 insertions(+), 10 deletions(-) create mode 100644 sysdeps/riscv/multiarch/memcpy_vector.S diff --git a/sysdeps/riscv/Makefile b/sysdeps/riscv/Makefile index 4c52f0d1ea..7d149d42e6 100644 --- a/sysdeps/riscv/Makefile +++ b/sysdeps/riscv/Makefile @@ -5,13 +5,35 @@ endif ifeq ($(subdir),elf) gen-as-const-headers += dl-link.sym -$(objpfx)tst-glibc-hwcaps: $(objpfx)libmarkermod2-1.so - +$(objpfx)tst-glibc-hwcaps: $(objpfx)libmarkermod2-1.so \ + $(objpfx)libmarkermod3-1.so $(objpfx)libmarkermod4-1.so $(objpfx)tst-glibc-hwcaps.out: \ $(objpfx)libmarkermod2.so \ - $(objpfx)glibc-hwcaps/rva20u64/libmarkermod2.so \ + $(objpfx)glibc-hwcaps/rva20u64/libmarkermod2.so \ + $(objpfx)libmarkermod3.so \ + $(objpfx)glibc-hwcaps/rva20u64/libmarkermod3.so \ + $(objpfx)glibc-hwcaps/rva22u64/libmarkermod3.so \ + $(objpfx)libmarkermod4.so \ + $(objpfx)glibc-hwcaps/rva20u64/libmarkermod4.so \ + $(objpfx)glibc-hwcaps/rva22u64/libmarkermod4.so \ + $(objpfx)glibc-hwcaps/rva23u64/libmarkermod4.so \ -$(objpfx)glibc-hwcaps/rva20u64/libmarkermod2.so: $(objpfx)libmarkermod2-2.so +$(objpfx)glibc-hwcaps/rva20u64/libmarkermod2.so: $(objpfx)libmarkermod2-2.so + $(make-target-directory) + cp $< $@ +$(objpfx)glibc-hwcaps/rva20u64/libmarkermod3.so: $(objpfx)libmarkermod3-2.so + $(make-target-directory) + cp $< $@ +$(objpfx)glibc-hwcaps/rva22u64/libmarkermod3.so: $(objpfx)libmarkermod3-3.so + $(make-target-directory) + cp $< $@ +$(objpfx)glibc-hwcaps/rva20u64/libmarkermod4.so: $(objpfx)libmarkermod4-2.so + $(make-target-directory) + cp $< $@ +$(objpfx)glibc-hwcaps/rva22u64/libmarkermod4.so: $(objpfx)libmarkermod4-3.so + $(make-target-directory) + cp $< $@ +$(objpfx)glibc-hwcaps/rva23u64/libmarkermod4.so: $(objpfx)libmarkermod4-4.so $(make-target-directory) cp $< $@ diff --git a/sysdeps/riscv/get-profile-level.h b/sysdeps/riscv/get-profile-level.h index ce98d8f807..a0327a8d7b 100644 --- a/sysdeps/riscv/get-profile-level.h +++ b/sysdeps/riscv/get-profile-level.h @@ -21,6 +21,21 @@ # define PROFILE_MASK_RISCV_RVA20U64 (RISCV_HWPROBE_IMA_FD | RISCV_HWPROBE_IMA_C | \ RISCV_HWPROBE_EXT_ZICNTR) +# define PROFILE_MASK_RISCV_RVA22U64 (PROFILE_MASK_RISCV_RVA20U64 | \ + RISCV_HWPROBE_EXT_ZIHINTPAUSE | RISCV_HWPROBE_EXT_ZBA | \ + RISCV_HWPROBE_EXT_ZBB | RISCV_HWPROBE_EXT_ZBS | \ + RISCV_HWPROBE_EXT_ZICBOM | RISCV_HWPROBE_EXT_ZICBOZ | \ + RISCV_HWPROBE_EXT_ZFHMIN | RISCV_HWPROBE_EXT_ZKT) + +# define PROFILE_MASK_RISCV_RVA23U64 (PROFILE_MASK_RISCV_RVA22U64 | \ + RISCV_HWPROBE_IMA_V | RISCV_HWPROBE_EXT_ZVFHMIN | \ + RISCV_HWPROBE_EXT_ZVBB | RISCV_HWPROBE_EXT_ZVKT | \ + RISCV_HWPROBE_EXT_ZIHINTNTL | RISCV_HWPROBE_EXT_ZICOND | \ + RISCV_HWPROBE_EXT_ZICOND | RISCV_HWPROBE_EXT_ZIMOP | \ + RISCV_HWPROBE_EXT_ZCMOP | RISCV_HWPROBE_EXT_ZCB | \ + RISCV_HWPROBE_EXT_ZFA | RISCV_HWPROBE_EXT_ZAWRS | \ + RISCV_HWPROBE_EXT_SUPM) + static unsigned int get_profile_level() { @@ -50,5 +65,18 @@ get_profile_level() } profile_level = 20; + // Extensions without explicit detection (yet): + // Zic64b, Zicbop + if ((pair.value & PROFILE_MASK_RISCV_RVA22U64) != \ + PROFILE_MASK_RISCV_RVA22U64) { + return profile_level; + } + profile_level = 22; + + if ((pair.value & PROFILE_MASK_RISCV_RVA23U64) != \ + PROFILE_MASK_RISCV_RVA23U64) { + return profile_level; + } + profile_level = 23; return profile_level; } diff --git a/sysdeps/riscv/multiarch/memcpy_noalignment.S b/sysdeps/riscv/multiarch/memcpy_noalignment.S index 8f892ba550..42f719f6c6 100644 --- a/sysdeps/riscv/multiarch/memcpy_noalignment.S +++ b/sysdeps/riscv/multiarch/memcpy_noalignment.S @@ -20,7 +20,7 @@ #include #include -#if PROFILE_SHOULD_BUILD (20) +#if PROFILE_SHOULD_BUILD (22) /* memcpy optimization for CPUs with fast unaligned support (RISCV_HWPROBE_MISALIGNED_FAST). diff --git a/sysdeps/riscv/multiarch/memcpy_vector.S b/sysdeps/riscv/multiarch/memcpy_vector.S new file mode 100644 index 0000000000..9bca323ab5 --- /dev/null +++ b/sysdeps/riscv/multiarch/memcpy_vector.S @@ -0,0 +1,54 @@ +/* RVV versions memcpy. RISC-V version. + Copyright (C) 2025 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include + +#if PROFILE_SHOULD_BUILD (23) + +#define dst a0 +#define src a1 +#define num a2 + +#define ivl a3 +#define dst_ptr a4 + +#define ELEM_LMUL_SETTING m8 +#define vdata v0 + +ENTRY (__memcpy_vector) +.option push +.option arch, +v + mv dst_ptr, dst +L(loop): + vsetvli ivl, num, e8, ELEM_LMUL_SETTING, ta, ma + + vle8.v vdata, (src) + sub num, num, ivl + add src, src, ivl + vse8.v vdata, (dst_ptr) + add dst_ptr, dst_ptr, ivl + + bnez num, L(loop) + + ret +.option pop +END (__memcpy_vector) + +#endif diff --git a/sysdeps/riscv/profile-ifunc-macros.h b/sysdeps/riscv/profile-ifunc-macros.h index 33a16af241..52b11fd4f5 100644 --- a/sysdeps/riscv/profile-ifunc-macros.h +++ b/sysdeps/riscv/profile-ifunc-macros.h @@ -30,6 +30,22 @@ implementations. If there is no implementation at or above the minimum build profile level, then include the highest profile level implementation. */ +#if MINIMUM_RISCV_PROFILE_LEVEL <= 23 +# define RISCV_IFUNC_RESOLVE_RVA23(cond, impl) {if(cond) { return impl; }} +# define RISCV_IFUNC_IMPL_ADD_RVA23(...) IFUNC_IMPL_ADD (__VA_ARGS__) +#else +# define RISCV_IFUNC_RESOLVE_RVA23(cond, impl) +# define RISCV_IFUNC_IMPL_ADD_RVA23(...) +#endif + +#if MINIMUM_RISCV_PROFILE_LEVEL <= 22 +# define RISCV_IFUNC_RESOLVE_RVA22(cond, impl) {if(cond) { return impl; }} +# define RISCV_IFUNC_IMPL_ADD_RVA22(...) IFUNC_IMPL_ADD (__VA_ARGS__) +#else +# define RISCV_IFUNC_RESOLVE_RVA22(cond, impl) +# define RISCV_IFUNC_IMPL_ADD_RVA22(...) +#endif + #if MINIMUM_RISCV_PROFILE_LEVEL <= 20 # define RISCV_IFUNC_RESOLVE_RVA20(cond, impl) {if(cond) { return impl; }} # define RISCV_IFUNC_IMPL_ADD_RVA20(...) IFUNC_IMPL_ADD (__VA_ARGS__) diff --git a/sysdeps/riscv/profile-level.h b/sysdeps/riscv/profile-level.h index c846e1beb1..280ca86766 100644 --- a/sysdeps/riscv/profile-level.h +++ b/sysdeps/riscv/profile-level.h @@ -39,8 +39,8 @@ #if __RISCV_GC && defined __riscv_zicsr && defined __riscv_zicntr && \ defined __riscv_ziccif && defined __riscv_ziccrse && \ - defined __riscv_ziccamoa && defined __riscv_za128rs && \ - defined __riscv_zicclsm + defined __riscv_ziccamoa && defined __riscv_zicclsm && \ + (defined __riscv_za128rs || defined __riscv_za64rs) # define __RISCV_PROFILE_RVA20 1 # undef MINIMUM_RISCV_PROFILE_LEVEL # define MINIMUM_RISCV_PROFILE_LEVEL 20 @@ -48,6 +48,61 @@ # define __RISCV_PROFILE_RVA20 0 #endif +#if __RISCV_PROFILE_RVA20 && defined __riscv_zic64b && \ + defined __riscv_za64rs && defined __riscv_zihintpause && \ + defined __riscv_zba && defined __riscv_zbb && \ + defined __riscv_zicbom && defined __riscv_zicbop && \ + defined __riscv_zicboz && defined __riscv_zfhmin && \ + defined __riscv_zkt +# define __RISCV_PROFILE_RVA22 1 +# undef MINIMUM_RISCV_PROFILE_LEVEL +# define MINIMUM_RISCV_PROFILE_LEVEL 22 +#else +# define __RISCV_PROFILE_RVA22 0 +#endif + +#if __RISCV_PROFILE_RVA22 && defined __riscv_v && \ + defined __riscv_zvfhmin && defined __riscv_zvbb && \ + defined __riscv_zvkt && defined __riscv_zihintntl && \ + defined __riscv_zicond && defined __riscv_zimop && \ + defined __riscv_zcmop && defined __riscv_zcb && \ + defined __riscv_zfa && defined __riscv_zawrs && \ + defined __riscv_supm +# define __RISCV_PROFILE_RVA23 1 +# undef MINIMUM_RISCV_PROFILE_LEVEL +# define MINIMUM_RISCV_PROFILE_LEVEL 23 +#else +# define __RISCV_PROFILE_RVA23 0 +#endif + +/* Profile level >= 23 guaranteed includes. */ +#define V_RISCV_PROFILE_LEVEL 23 +#define ZVFHMIN_RISCV_PROFILE_LEVEL 23 +#define ZVBB_RISCV_PROFILE_LEVEL 23 +#define ZVKT_RISCV_PROFILE_LEVEL 23 +#define ZIHINTNTL_RISCV_PROFILE_LEVEL 23 +#define ZICOND_RISCV_PROFILE_LEVEL 23 +#define ZIMOP_RISCV_PROFILE_LEVEL 23 +#define ZCMOP_RISCV_PROFILE_LEVEL 23 +#define ZCB_RISCV_PROFILE_LEVEL 23 +#define ZFA_RISCV_PROFILE_LEVEL 23 +#define ZAWRS_RISCV_PROFILE_LEVEL 23 +#define SUPM_RISCV_PROFILE_LEVEL 23 + +/* Profile level >= 22 guaranteed includes. */ +#define HPM_RISCV_PROFILE_LEVEL 22 +#define ZA64RS_RISCV_PROFILE_LEVEL 22 +#define ZIHINTPAUSE_ISCV_PROFILE_LEVEL 22 +#define ZBA_RISCV_PROFILE_LEVEL 22 +#define ZBB_RISCV_PROFILE_LEVEL 22 +#define ZBS_RISCV_PROFILE_LEVEL 22 +#define ZIC64B_RISCV_PROFILE_LEVEL 22 +#define ZICBOM_RISCV_PROFILE_LEVEL 22 +#define ZICBOP_RISCV_PROFILE_LEVEL 22 +#define ZICBOZ_RISCV_PROFILE_LEVEL 22 +#define ZFHMIN_RISCV_PROFILE_LEVEL 22 +#define ZKT_RISCV_PROFILE_LEVEL 22 + /* Profile level >= 20 guaranteed includes. */ #define FD_RISCV_PROFILE_LEVEL 20 #define A_RISCV_PROFILE_LEVEL 20 diff --git a/sysdeps/riscv/rv64/dl-hwcaps-subdirs.c b/sysdeps/riscv/rv64/dl-hwcaps-subdirs.c index 3a88271ebd..ec807b9360 100644 --- a/sysdeps/riscv/rv64/dl-hwcaps-subdirs.c +++ b/sysdeps/riscv/rv64/dl-hwcaps-subdirs.c @@ -21,8 +21,8 @@ #include #include -const char _dl_hwcaps_subdirs[] = "rva20u64"; -enum { subdirs_count = 1 }; /* Number of components in _dl_hwcaps_subdirs. */ +const char _dl_hwcaps_subdirs[] = "rva23u64:rva22u64:rva20u64"; +enum { subdirs_count = 3 }; /* Number of components in _dl_hwcaps_subdirs. */ uint32_t _dl_hwcaps_subdirs_active (void) @@ -36,5 +36,13 @@ _dl_hwcaps_subdirs_active (void) return _dl_hwcaps_subdirs_build_bitmask (subdirs_count, active); } ++active; + + /* v2: RVA22U64 */ + if(profile_level < 22) { + return _dl_hwcaps_subdirs_build_bitmask (subdirs_count, active); + } + + /* v3: RVA23U64 */ + ++active; return _dl_hwcaps_subdirs_build_bitmask (subdirs_count, active); } \ No newline at end of file diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile index fcef5659d4..8d3df8edcf 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile @@ -3,6 +3,7 @@ sysdep_routines += \ memcpy \ memcpy-generic \ memcpy_noalignment \ + memcpy_vector \ # sysdep_routines CFLAGS-memcpy_noalignment.c += -mno-strict-align diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c index f0271679fd..e9b96f31d4 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c @@ -28,6 +28,7 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, size_t i = max; bool unaligned = false; + bool rvv_ext = false; struct riscv_hwprobe pair = { .key = RISCV_HWPROBE_KEY_CPUPERF_0 }; if (__riscv_hwprobe (&pair, 1, 0, NULL, 0) == 0 @@ -35,8 +36,15 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, != RISCV_HWPROBE_MISALIGNED_UNSUPPORTED) unaligned = true; + struct riscv_hwprobe ext_pair = { .key = RISCV_HWPROBE_KEY_IMA_EXT_0 }; + if (__riscv_hwprobe (&ext_pair, 1, 0, NULL, 0) == 0 + && (ext_pair.value & RISCV_HWPROBE_IMA_V)) + rvv_ext = true; + IFUNC_IMPL (i, name, memcpy, - RISCV_IFUNC_IMPL_ADD_RVA20 (array, i, memcpy, unaligned, + RISCV_IFUNC_IMPL_ADD_RVA23 (array, i, memcpy, rvv_ext, + __memcpy_vector) + RISCV_IFUNC_IMPL_ADD_RVA22 (array, i, memcpy, unaligned, __memcpy_noalignment) RISCV_IFUNC_IMPL_ADD_INIT (array, i, memcpy, 1, __memcpy_generic)) diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/memcpy.c b/sysdeps/unix/sysv/linux/riscv/multiarch/memcpy.c index 353eace6bb..6eb550ec26 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/memcpy.c +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/memcpy.c @@ -28,20 +28,28 @@ # include # include # include +# include extern __typeof (__redirect_memcpy) __libc_memcpy; extern __typeof (__redirect_memcpy) __memcpy_generic attribute_hidden; extern __typeof (__redirect_memcpy) __memcpy_noalignment attribute_hidden; +extern __typeof (__redirect_memcpy) __memcpy_vector attribute_hidden; static inline __typeof (__redirect_memcpy) * select_memcpy_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) { unsigned long long int v; bool unaligned = false; + bool rvv_ext = false; + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_CPUPERF_0, &v) == 0 && (v & RISCV_HWPROBE_MISALIGNED_MASK) != RISCV_HWPROBE_MISALIGNED_UNSUPPORTED) unaligned = true; + + if(RISCV_PROFILE_COND(rvv_ext, V)) { + return __memcpy_vector; + } if(RISCV_PROFILE_COND(unaligned, ZICCLSM)) { return __memcpy_noalignment;