From patchwork Fri Jan 17 13:01:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: daichengrong X-Patchwork-Id: 104992 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 224583847EC7 for ; Fri, 17 Jan 2025 13:02:33 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 224583847EC7 X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from cstnet.cn (smtp84.cstnet.cn [159.226.251.84]) by sourceware.org (Postfix) with ESMTPS id 31F153848434 for ; Fri, 17 Jan 2025 13:01:43 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 31F153848434 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=iscas.ac.cn Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=iscas.ac.cn ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 31F153848434 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=159.226.251.84 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1737118905; cv=none; b=XTPkyrtKgES5DJ2cK2UzFAaZF6aO0j3fFQsr1uQT22GOVX/UQzWSC7Fjl4BR72q/qP3ty3vtBVTXKkuipggfXbsS6szXFNXfMW6hk8OU+yC3H9CyflTPIHw9AYfQiz70TnD+ov5msiqll2qcCxKpZlfoTqA3ys+vuK3FAUh4tho= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1737118905; c=relaxed/simple; bh=jaHSM4cLgPEzpK+pgzJKOXVWgbFoIS97m3JZySdh1N4=; h=Message-ID:Date:MIME-Version:Subject:To:From; b=EdE3D4aRbTWk39PHMbNi8FemFnI6t9O08jpF17rgaPDY9l/IN6taHcLKydLyhPq6V9fRfraACZU51q2SlhdkG02taF/dVz7BFyy4MvNQdDGFQ1xkiT+Yi6hKFzE8SGGa/wYyk78vMC6A/kQKOAJYJW1h5E4ut3k9soLh62iiQMw= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 31F153848434 Received: from [192.168.101.30] (unknown [112.1.53.248]) by APP-05 (Coremail) with SMTP id zQCowADHziqtVIpndYezBw--.38681S2; Fri, 17 Jan 2025 21:01:33 +0800 (CST) Message-ID: <9b4374d8-ba4a-4b45-a3e0-757dc65ea0b3@iscas.ac.cn> Date: Fri, 17 Jan 2025 21:01:33 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: [PATCH v2] RISC-V: add multiarch RVV support for memcpy using FMV IFUNC To: Andrew Waterman References: <7fa4ac68.3f903.1946e72cef3.Coremail.daichengrong@iscas.ac.cn> Content-Language: en-CA Cc: Palmer Dabbelt , libc-alpha From: daichengrong In-Reply-To: X-CM-TRANSID: zQCowADHziqtVIpndYezBw--.38681S2 X-Coremail-Antispam: 1UD129KBjvJXoWxJFyUWF1UKF1kKFykWFykZrb_yoWrAFW3pF Z3CF15Kan5Jr1xGFWS9w1jq3W5ZryrJF4Yk34Y93yUJrW0gr43Jas2yw1DWFnrJrWSkFWr uF1vqFyDu398A3JanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUgEb7Iv0xC_Kw4lb4IE77IF4wAFF20E14v26r1j6r4UM7CY07I2 0VC2zVCF04k26cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28lY4IEw2IIxxk0rw A2F7IY1VAKz4vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_Gr0_Xr1l84ACjcxK6xII jxv20xvEc7CjxVAFwI0_Gr0_Cr1l84ACjcxK6I8E87Iv67AKxVW8JVWxJwA2z4x0Y4vEx4 A2jsIEc7CjxVAFwI0_Gr0_Gr1UM2AIxVAIcxkEcVAq07x20xvEncxIr21lYx0E2Ix0cI8I cVAFwI0_JrI_JrylYx0Ex4A2jsIE14v26r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjc xG0xvEwIxGrwCjr7xvwVCIw2I0I7xG6c02F41l42xK82IYc2Ij64vIr41l4I8I3I0E4IkC 6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUGVWUWwC20s026x8GjcxK67AKxVWUGVWUWw C2zVAF1VAY17CE14v26r1Y6r17MIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_ JF4lIxAIcVC0I7IYx2IY6xkF7I0E14v26r1j6r4UMIIF0xvE42xK8VAvwI8IcIk0rVWUJV WUCwCI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIY CTnIWIevJa73UjIFyTuYvjxUqr-PUUUUU X-Originating-IP: [112.1.53.248] X-CM-SenderInfo: pgdluxxhqj201qj6x2xfdvhtffof0/ X-Spam-Status: No, score=-10.9 required=5.0 tests=BAYES_00, BODY_8BITS, GIT_PATCH_0, HTML_MESSAGE, KAM_DMARC_STATUS, KAM_SHORT, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces~patchwork=sourceware.org@sourceware.org On 2025/1/17 07:46, Andrew Waterman wrote: > On Thu, Jan 16, 2025 at 1:31 AM 戴成荣 wrote: >> With RISC-V GCC Function multi-versioning IFUNC support,glibc could utilize some new RISC-V CPUs with new extensions such as Vector to re-realize or speed up some functions like memcpy. >> >> +ENTRY (__memcpy_vector) >> + beq a2, zero, L(ret) > This branch should be deleted; the strip-mine loop below does the > correct thing for a2=0. Including the branch is an anti-optimization: > it speeds up the uncommon case of size-0 memcpy at the expense of the > common case. Thanks! This branch has been deleted. >> + mv a6, a0 >> +L(loop): >> + vsetvli a3,a2,e8,m8,ta,mu >> + vle8.v v8,(a1) >> + vse8.v v8,(a6) >> + add a1,a1,a3 >> + sub a2,a2,a3 >> + add a6,a6,a3 >> + bnez a2,L(loop) >> +L(ret): >> + ret >> +END (__memcpy_vector) Add riscv vector support for memcpy. Signed-off-by: daichengrong --- sysdeps/riscv/multiarch/memcpy_vector.S | 35 +++++++++++++++++++ .../unix/sysv/linux/riscv/multiarch/Makefile | 2 ++ .../unix/sysv/linux/riscv/multiarch/memcpy.c | 5 +++ 3 files changed, 42 insertions(+) create mode 100644 sysdeps/riscv/multiarch/memcpy_vector.S diff --git a/sysdeps/riscv/multiarch/memcpy_vector.S b/sysdeps/riscv/multiarch/memcpy_vector.S new file mode 100644 index 0000000000..8fddab8432 --- /dev/null +++ b/sysdeps/riscv/multiarch/memcpy_vector.S @@ -0,0 +1,35 @@ +/* memcpy for RISC-V Vector. + Copyright (C) 2024-2025 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + + +#include +#include + +ENTRY (__memcpy_vector) + mv a6, a0 +L(loop): + vsetvli a3,a2,e8,m8,ta,mu + vle8.v v8,(a1) + vse8.v v8,(a6) + add a1,a1,a3 + sub a2,a2,a3 + add a6,a6,a3 + bnez a2,L(loop) +L(ret): + ret +END (__memcpy_vector) diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile index fcef5659d4..a8b6c22af1 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile @@ -3,7 +3,9 @@ sysdep_routines += \ memcpy \ memcpy-generic \ memcpy_noalignment \ + memcpy_vector \ # sysdep_routines CFLAGS-memcpy_noalignment.c += -mno-strict-align +ASFLAGS-memcpy_vector.S += -march=rv64gcv endif diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/memcpy.c b/sysdeps/unix/sysv/linux/riscv/multiarch/memcpy.c index 8544f5402a..c9879762f6 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/memcpy.c +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/memcpy.c @@ -32,11 +32,16 @@ extern __typeof (__redirect_memcpy) __libc_memcpy; extern __typeof (__redirect_memcpy) __memcpy_generic attribute_hidden; extern __typeof (__redirect_memcpy) __memcpy_noalignment attribute_hidden; +extern __typeof (__redirect_memcpy) __memcpy_vector attribute_hidden; static inline __typeof (__redirect_memcpy) * select_memcpy_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) { unsigned long long int v; + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_IMA_EXT_0, &v) == 0 + && (v & RISCV_HWPROBE_IMA_V) == RISCV_HWPROBE_IMA_V) + return __memcpy_vector; + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_CPUPERF_0, &v) == 0 && (v & RISCV_HWPROBE_MISALIGNED_MASK) == RISCV_HWPROBE_MISALIGNED_FAST) return __memcpy_noalignment; --