From patchwork Wed May 17 15:26:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Desnoyers X-Patchwork-Id: 69523 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6AF5D3854179 for ; Wed, 17 May 2023 15:28:11 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 6AF5D3854179 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1684337291; bh=I+n62NOzeqoDJaIS0fVUVqcuLGX1IUfTknQBNq/CXhc=; h=To:Cc:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=f7SxcgcqA0fhOJj8OFLkvJ9yekzjbJkBFWqR+UqVzLIV8kV5iKXj9qNcON/96sFVL s3L4Z1vFL5BDjVuex6lZ81Y8SAQti45hNGd3tS0pdKdakNAB6T2yScqB5g1h9al6CX JfVRLtcBb2K7rSXI1AZPzhOFUO8XN0A72tOUK3K8= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from smtpout.efficios.com (unknown [IPv6:2607:5300:203:b2ee::31e5]) by sourceware.org (Postfix) with ESMTPS id 65B1A3858C41 for ; Wed, 17 May 2023 15:27:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 65B1A3858C41 Received: from localhost.localdomain (192-222-143-198.qc.cable.ebox.net [192.222.143.198]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4QLxmR53HXz131r; Wed, 17 May 2023 11:27:03 -0400 (EDT) To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , "Paul E . McKenney" , Boqun Feng , "H . Peter Anvin" , Paul Turner , linux-api@vger.kernel.org, Christian Brauner , Florian Weimer , David.Laight@ACULAB.COM, carlos@redhat.com, Peter Oskolkov , Alexander Mikhalitsyn , Chris Kennelly , Ingo Molnar , Darren Hart , Davidlohr Bueso , =?utf-8?q?Andr=C3=A9_Almeida?= , libc-alpha@sourceware.org, Steven Rostedt , Jonathan Corbet , Mathieu Desnoyers , Florian Weimer Subject: [RFC PATCH 1/4] rseq: Add sched_state field to struct rseq Date: Wed, 17 May 2023 11:26:51 -0400 Message-Id: <20230517152654.7193-2-mathieu.desnoyers@efficios.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230517152654.7193-1-mathieu.desnoyers@efficios.com> References: <20230517152654.7193-1-mathieu.desnoyers@efficios.com> MIME-Version: 1.0 X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mathieu Desnoyers via Libc-alpha From: Mathieu Desnoyers Reply-To: Mathieu Desnoyers Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Expose the "on-cpu" state for each thread through struct rseq to allow adaptative mutexes to decide more accurately between busy-waiting and calling sys_futex() to release the CPU, based on the on-cpu state of the mutex owner. It is only provided as an optimization hint, because there is no guarantee that the page containing this field is in the page cache, and therefore the scheduler may very well fail to clear the on-cpu state on preemption. This is expected to be rare though, and is resolved as soon as the task returns to user-space. The goal is to improve use-cases where the duration of the critical sections for a given lock follows a multi-modal distribution, preventing statistical guesses from doing a good job at choosing between busy-wait and futex wait behavior. Signed-off-by: Mathieu Desnoyers Cc: Peter Zijlstra (Intel) Cc: Jonathan Corbet Cc: Steven Rostedt (Google) Cc: Carlos O'Donell Cc: Florian Weimer Cc: libc-alpha@sourceware.org --- include/linux/sched.h | 12 ++++++++++++ include/uapi/linux/rseq.h | 17 +++++++++++++++++ kernel/rseq.c | 14 ++++++++++++++ 3 files changed, 43 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index eed5d65b8d1f..c7e9248134c1 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2351,11 +2351,20 @@ static inline void rseq_signal_deliver(struct ksignal *ksig, rseq_handle_notify_resume(ksig, regs); } +void __rseq_set_sched_state(struct task_struct *t, unsigned int state); + +static inline void rseq_set_sched_state(struct task_struct *t, unsigned int state) +{ + if (t->rseq) + __rseq_set_sched_state(t, state); +} + /* rseq_preempt() requires preemption to be disabled. */ static inline void rseq_preempt(struct task_struct *t) { __set_bit(RSEQ_EVENT_PREEMPT_BIT, &t->rseq_event_mask); rseq_set_notify_resume(t); + rseq_set_sched_state(t, 0); } /* rseq_migrate() requires preemption to be disabled. */ @@ -2405,6 +2414,9 @@ static inline void rseq_signal_deliver(struct ksignal *ksig, struct pt_regs *regs) { } +static inline void rseq_set_sched_state(struct task_struct *t, unsigned int state) +{ +} static inline void rseq_preempt(struct task_struct *t) { } diff --git a/include/uapi/linux/rseq.h b/include/uapi/linux/rseq.h index c233aae5eac9..c6d8537e23ca 100644 --- a/include/uapi/linux/rseq.h +++ b/include/uapi/linux/rseq.h @@ -37,6 +37,13 @@ enum rseq_cs_flags { (1U << RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE_BIT), }; +enum rseq_sched_state { + /* + * Task is currently running on a CPU if bit is set. + */ + RSEQ_SCHED_STATE_ON_CPU = (1U << 0), +}; + /* * struct rseq_cs is aligned on 4 * 8 bytes to ensure it is always * contained within a single cache-line. It is usually declared as @@ -148,6 +155,16 @@ struct rseq { */ __u32 mm_cid; + /* + * Restartable sequences sched_state field. Updated by the kernel. Read + * by user-space with single-copy atomicity semantics. This fields can + * be read by any userspace thread. Aligned on 32-bit. Contains a + * bitmask of enum rseq_sched_state. This field is provided as a hint + * by the scheduler, and requires that the page holding struct rseq is + * faulted-in for the state update to be performed by the scheduler. + */ + __u32 sched_state; + /* * Flexible array member at end of structure, after last feature field. */ diff --git a/kernel/rseq.c b/kernel/rseq.c index 9de6e35fe679..b2eb3bbaa9ef 100644 --- a/kernel/rseq.c +++ b/kernel/rseq.c @@ -91,6 +91,7 @@ static int rseq_update_cpu_node_id(struct task_struct *t) u32 cpu_id = raw_smp_processor_id(); u32 node_id = cpu_to_node(cpu_id); u32 mm_cid = task_mm_cid(t); + u32 sched_state = RSEQ_SCHED_STATE_ON_CPU; WARN_ON_ONCE((int) mm_cid < 0); if (!user_write_access_begin(rseq, t->rseq_len)) @@ -99,6 +100,7 @@ static int rseq_update_cpu_node_id(struct task_struct *t) unsafe_put_user(cpu_id, &rseq->cpu_id, efault_end); unsafe_put_user(node_id, &rseq->node_id, efault_end); unsafe_put_user(mm_cid, &rseq->mm_cid, efault_end); + unsafe_put_user(sched_state, &rseq->sched_state, efault_end); /* * Additional feature fields added after ORIG_RSEQ_SIZE * need to be conditionally updated only if @@ -339,6 +341,18 @@ void __rseq_handle_notify_resume(struct ksignal *ksig, struct pt_regs *regs) force_sigsegv(sig); } +/* + * Attempt to update rseq scheduler state. + */ +void __rseq_set_sched_state(struct task_struct *t, unsigned int state) +{ + if (unlikely(t->flags & PF_EXITING)) + return; + pagefault_disable(); + (void) put_user(state, &t->rseq->sched_state); + pagefault_enable(); +} + #ifdef CONFIG_DEBUG_RSEQ /* From patchwork Wed May 17 15:26:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Desnoyers X-Patchwork-Id: 69521 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id D3B3B385558A for ; Wed, 17 May 2023 15:27:28 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org D3B3B385558A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1684337248; bh=rFwpm2TeJ0c+EdYWU70YKVNWl31vEZpFl0AiI2tATUw=; h=To:Cc:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=aL/3MdgKPcFFCbT38ubPj/q/tlF6qeyuYtiU+FeT55qn5Ci7XgYjxoFptLchzyYON RZxKT6obg7DYVFduFcMJZaoKJ9lvkKGklS+EyYrz67l/H9LGP+eDqaRMbkqCURe6Ch Y2MHW/KZvp4pAd5hGxl4gRwtwYYRaCEwxez5ERvA= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from smtpout.efficios.com (smtpout.efficios.com [167.114.26.122]) by sourceware.org (Postfix) with ESMTPS id BD7223858C5E for ; Wed, 17 May 2023 15:27:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org BD7223858C5E Received: from localhost.localdomain (192-222-143-198.qc.cable.ebox.net [192.222.143.198]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4QLxmS1S00z12Mc; Wed, 17 May 2023 11:27:04 -0400 (EDT) To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , "Paul E . McKenney" , Boqun Feng , "H . Peter Anvin" , Paul Turner , linux-api@vger.kernel.org, Christian Brauner , Florian Weimer , David.Laight@ACULAB.COM, carlos@redhat.com, Peter Oskolkov , Alexander Mikhalitsyn , Chris Kennelly , Ingo Molnar , Darren Hart , Davidlohr Bueso , =?utf-8?q?Andr=C3=A9_Almeida?= , libc-alpha@sourceware.org, Steven Rostedt , Jonathan Corbet , Mathieu Desnoyers Subject: [RFC PATCH 2/4] selftests/rseq: Add sched_state rseq field and getter Date: Wed, 17 May 2023 11:26:52 -0400 Message-Id: <20230517152654.7193-3-mathieu.desnoyers@efficios.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230517152654.7193-1-mathieu.desnoyers@efficios.com> References: <20230517152654.7193-1-mathieu.desnoyers@efficios.com> MIME-Version: 1.0 X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mathieu Desnoyers via Libc-alpha From: Mathieu Desnoyers Reply-To: Mathieu Desnoyers Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Extend struct rseq in the rseq selftests to include the sched_state field. Implement a getter function for this field. Signed-off-by: Mathieu Desnoyers --- tools/testing/selftests/rseq/rseq-abi.h | 17 +++++++++++++++++ tools/testing/selftests/rseq/rseq.h | 5 +++++ 2 files changed, 22 insertions(+) diff --git a/tools/testing/selftests/rseq/rseq-abi.h b/tools/testing/selftests/rseq/rseq-abi.h index fb4ec8a75dd4..15ec333e9eec 100644 --- a/tools/testing/selftests/rseq/rseq-abi.h +++ b/tools/testing/selftests/rseq/rseq-abi.h @@ -37,6 +37,13 @@ enum rseq_abi_cs_flags { (1U << RSEQ_ABI_CS_FLAG_NO_RESTART_ON_MIGRATE_BIT), }; +enum rseq_sched_state { + /* + * Task is currently running on a CPU if bit is set. + */ + RSEQ_SCHED_STATE_ON_CPU = (1U << 0), +}; + /* * struct rseq_abi_cs is aligned on 4 * 8 bytes to ensure it is always * contained within a single cache-line. It is usually declared as @@ -164,6 +171,16 @@ struct rseq_abi { */ __u32 mm_cid; + /* + * Restartable sequences sched_state field. Updated by the kernel. Read + * by user-space with single-copy atomicity semantics. This fields can + * be read by any userspace thread. Aligned on 32-bit. Contains a + * bitmask of enum rseq_sched_state. This field is provided as a hint + * by the scheduler, and requires that the page holding struct rseq is + * faulted-in for the state update to be performed by the scheduler. + */ + __u32 sched_state; + /* * Flexible array member at end of structure, after last feature field. */ diff --git a/tools/testing/selftests/rseq/rseq.h b/tools/testing/selftests/rseq/rseq.h index d7364ea4d201..348e9385cb2b 100644 --- a/tools/testing/selftests/rseq/rseq.h +++ b/tools/testing/selftests/rseq/rseq.h @@ -236,6 +236,11 @@ static inline void rseq_prepare_unload(void) rseq_clear_rseq_cs(); } +static inline uint32_t rseq_current_sched_state(void) +{ + return RSEQ_ACCESS_ONCE(rseq_get_abi()->sched_state); +} + static inline __attribute__((always_inline)) int rseq_cmpeqv_storev(enum rseq_mo rseq_mo, enum rseq_percpu_mode percpu_mode, intptr_t *v, intptr_t expect, From patchwork Wed May 17 15:26:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Desnoyers X-Patchwork-Id: 69522 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 364B738438FD for ; Wed, 17 May 2023 15:27:39 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 364B738438FD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1684337259; bh=mqrSj2vd4XVIdxm9EDKqlQXsCArThMEl0iVqbFQ6GD0=; h=To:Cc:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=DIQemUJgRsY9k6nVl6QpvTJDtl/e3//OKC1nmGE0f/x9RUyk7APISfVQrZgSGJuv6 2uP4Tud3yVec/NyiDmVvOgNr9sOwiKTI/TjQ0HjeZ1N9SSQRzgeOS/EPK5oYId5ugA VPX4+1KMYnOd7ohs9v29VMIjf4pDShu59fy4nICs= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from smtpout.efficios.com (smtpout.efficios.com [167.114.26.122]) by sourceware.org (Postfix) with ESMTPS id C03CE3858C5F for ; Wed, 17 May 2023 15:27:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C03CE3858C5F Received: from localhost.localdomain (192-222-143-198.qc.cable.ebox.net [192.222.143.198]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4QLxmS504rz12wc; Wed, 17 May 2023 11:27:04 -0400 (EDT) To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , "Paul E . McKenney" , Boqun Feng , "H . Peter Anvin" , Paul Turner , linux-api@vger.kernel.org, Christian Brauner , Florian Weimer , David.Laight@ACULAB.COM, carlos@redhat.com, Peter Oskolkov , Alexander Mikhalitsyn , Chris Kennelly , Ingo Molnar , Darren Hart , Davidlohr Bueso , =?utf-8?q?Andr=C3=A9_Almeida?= , libc-alpha@sourceware.org, Steven Rostedt , Jonathan Corbet , Mathieu Desnoyers Subject: [RFC PATCH 3/4] selftests/rseq: Implement sched state test program Date: Wed, 17 May 2023 11:26:53 -0400 Message-Id: <20230517152654.7193-4-mathieu.desnoyers@efficios.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230517152654.7193-1-mathieu.desnoyers@efficios.com> References: <20230517152654.7193-1-mathieu.desnoyers@efficios.com> MIME-Version: 1.0 X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mathieu Desnoyers via Libc-alpha From: Mathieu Desnoyers Reply-To: Mathieu Desnoyers Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" This is a small test program with can be altered to show whether the target thread is on-cpu or not, dependending on whether it loops on poll() or does a busy-loop. Signed-off-by: Mathieu Desnoyers --- tools/testing/selftests/rseq/.gitignore | 1 + tools/testing/selftests/rseq/Makefile | 2 +- .../testing/selftests/rseq/sched_state_test.c | 71 +++++++++++++++++++ 3 files changed, 73 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/rseq/sched_state_test.c diff --git a/tools/testing/selftests/rseq/.gitignore b/tools/testing/selftests/rseq/.gitignore index 16496de5f6ce..a8db9f7a7cec 100644 --- a/tools/testing/selftests/rseq/.gitignore +++ b/tools/testing/selftests/rseq/.gitignore @@ -9,3 +9,4 @@ param_test_compare_twice param_test_mm_cid param_test_mm_cid_benchmark param_test_mm_cid_compare_twice +sched_state_test diff --git a/tools/testing/selftests/rseq/Makefile b/tools/testing/selftests/rseq/Makefile index b357ba24af06..7c8f4f2be74c 100644 --- a/tools/testing/selftests/rseq/Makefile +++ b/tools/testing/selftests/rseq/Makefile @@ -14,7 +14,7 @@ OVERRIDE_TARGETS = 1 TEST_GEN_PROGS = basic_test basic_percpu_ops_test basic_percpu_ops_mm_cid_test param_test \ param_test_benchmark param_test_compare_twice param_test_mm_cid \ - param_test_mm_cid_benchmark param_test_mm_cid_compare_twice + param_test_mm_cid_benchmark param_test_mm_cid_compare_twice sched_state_test TEST_GEN_PROGS_EXTENDED = librseq.so diff --git a/tools/testing/selftests/rseq/sched_state_test.c b/tools/testing/selftests/rseq/sched_state_test.c new file mode 100644 index 000000000000..47aeee6d980f --- /dev/null +++ b/tools/testing/selftests/rseq/sched_state_test.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: LGPL-2.1 + +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include + +#include "rseq.h" + +static struct rseq_abi *target_thread; + +//TODO: +//Use rseq c.s. and rseq fence to protect access to remote thread's rseq_abi. + +static +void show_sched_state(struct rseq_abi *rseq_thread) +{ + uint32_t state; + + state = rseq_thread->sched_state; + printf("Target thread: ON_CPU=%d\n", !!(state & RSEQ_SCHED_STATE_ON_CPU)); +} + + +static +void *test_thread(void *arg) +{ + int i; + + for (i = 0; i < 1000; i++) { + show_sched_state(target_thread); + (void) poll(NULL, 0, 100); + } + return NULL; +} + +int main(int argc, char **argv) +{ + pthread_t test_thread_id; + int i; + + if (rseq_register_current_thread()) { + fprintf(stderr, "Error: rseq_register_current_thread(...) failed(%d): %s\n", + errno, strerror(errno)); + goto init_thread_error; + } + target_thread = rseq_get_abi(); + + pthread_create(&test_thread_id, NULL, test_thread, NULL); + + for (i = 0; i < 1000000000; i++) + rseq_barrier(); + //for (i = 0; i < 10000; i++) + // (void) poll(NULL, 0, 75); + + pthread_join(test_thread_id, NULL); + + if (rseq_unregister_current_thread()) { + fprintf(stderr, "Error: rseq_unregister_current_thread(...) failed(%d): %s\n", + errno, strerror(errno)); + goto init_thread_error; + } + return 0; + +init_thread_error: + return -1; +} From patchwork Wed May 17 15:26:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathieu Desnoyers X-Patchwork-Id: 69524 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5BDA938323D0 for ; Wed, 17 May 2023 15:28:21 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5BDA938323D0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1684337301; bh=Qq1wVDIs+dVwNXnwpFr/UOrPE+dVUEsiQ9P+MK/TCNQ=; h=To:Cc:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=DEb2gNgo+2/E3vv28z+5HiYTIaLfUj6iAncJdi47HIT1fao1EYfnP6uxxV3ZUk4Ou W1TGw1ebribELgVe72pfqp1wESFmvP/VtsjEtGRc99xL1wxB5fbh9TBoEidXlJovvo lMFmj20VbxAry6E8bSg79fcfeZN4UhnbyrO+kQl8= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from smtpout.efficios.com (unknown [IPv6:2607:5300:203:b2ee::31e5]) by sourceware.org (Postfix) with ESMTPS id E6B8A3858C78 for ; Wed, 17 May 2023 15:27:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org E6B8A3858C78 Received: from localhost.localdomain (192-222-143-198.qc.cable.ebox.net [192.222.143.198]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4QLxmT1RBCz12rG; Wed, 17 May 2023 11:27:05 -0400 (EDT) To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , "Paul E . McKenney" , Boqun Feng , "H . Peter Anvin" , Paul Turner , linux-api@vger.kernel.org, Christian Brauner , Florian Weimer , David.Laight@ACULAB.COM, carlos@redhat.com, Peter Oskolkov , Alexander Mikhalitsyn , Chris Kennelly , Ingo Molnar , Darren Hart , Davidlohr Bueso , =?utf-8?q?Andr=C3=A9_Almeida?= , libc-alpha@sourceware.org, Steven Rostedt , Jonathan Corbet , Mathieu Desnoyers Subject: [RFC PATCH 4/4] selftests/rseq: Implement rseq_mutex test program Date: Wed, 17 May 2023 11:26:54 -0400 Message-Id: <20230517152654.7193-5-mathieu.desnoyers@efficios.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230517152654.7193-1-mathieu.desnoyers@efficios.com> References: <20230517152654.7193-1-mathieu.desnoyers@efficios.com> MIME-Version: 1.0 X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Mathieu Desnoyers via Libc-alpha From: Mathieu Desnoyers Reply-To: Mathieu Desnoyers Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Example use of the rseq sched state. Signed-off-by: Mathieu Desnoyers --- tools/testing/selftests/rseq/.gitignore | 1 + tools/testing/selftests/rseq/Makefile | 3 +- tools/testing/selftests/rseq/rseq_mutex.c | 120 ++++++++++++++++++++++ 3 files changed, 123 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/rseq/rseq_mutex.c diff --git a/tools/testing/selftests/rseq/.gitignore b/tools/testing/selftests/rseq/.gitignore index a8db9f7a7cec..38d5b2fe5905 100644 --- a/tools/testing/selftests/rseq/.gitignore +++ b/tools/testing/selftests/rseq/.gitignore @@ -10,3 +10,4 @@ param_test_mm_cid param_test_mm_cid_benchmark param_test_mm_cid_compare_twice sched_state_test +rseq_mutex diff --git a/tools/testing/selftests/rseq/Makefile b/tools/testing/selftests/rseq/Makefile index 7c8f4f2be74c..a9d7ceb5b79b 100644 --- a/tools/testing/selftests/rseq/Makefile +++ b/tools/testing/selftests/rseq/Makefile @@ -14,7 +14,8 @@ OVERRIDE_TARGETS = 1 TEST_GEN_PROGS = basic_test basic_percpu_ops_test basic_percpu_ops_mm_cid_test param_test \ param_test_benchmark param_test_compare_twice param_test_mm_cid \ - param_test_mm_cid_benchmark param_test_mm_cid_compare_twice sched_state_test + param_test_mm_cid_benchmark param_test_mm_cid_compare_twice sched_state_test \ + rseq_mutex TEST_GEN_PROGS_EXTENDED = librseq.so diff --git a/tools/testing/selftests/rseq/rseq_mutex.c b/tools/testing/selftests/rseq/rseq_mutex.c new file mode 100644 index 000000000000..7e580fc6eebf --- /dev/null +++ b/tools/testing/selftests/rseq/rseq_mutex.c @@ -0,0 +1,120 @@ +// SPDX-License-Identifier: LGPL-2.1 + +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include + +#include "rseq.h" + +#define RSEQ_MUTEX_MAX_BUSY_LOOP 100 + +struct rseq_mutex { + /* + * NULL: unlocked. When non-NULL, owner points to per-thread rseq_abi + * of owner thread. + */ + struct rseq_abi *owner; +}; + +static struct rseq_mutex lock = { .owner = NULL }; + +static int testvar; + +static void rseq_lock_slowpath(struct rseq_mutex *lock) +{ + int i = 0; + + for (;;) { + struct rseq_abi *expected = NULL, *self = rseq_get_abi(); + + if (__atomic_compare_exchange_n(&lock->owner, &expected, self, false, + __ATOMIC_ACQUIRE, __ATOMIC_RELAXED)) + break; + //TODO: use rseq critical section to protect dereference of owner thread's + //rseq_abi, combined with rseq fence at thread reclaim. + if ((RSEQ_ACCESS_ONCE(expected->sched_state) & RSEQ_SCHED_STATE_ON_CPU) && + i < RSEQ_MUTEX_MAX_BUSY_LOOP) { + rseq_barrier(); /* busy-wait, e.g. cpu_relax(). */ + i++; + } else { + //TODO: Enqueue waiter in a wait-queue, and integrate + //with sys_futex rather than waiting for 10ms. + (void) poll(NULL, 0, 10); /* wait 10ms */ + } + } +} + +static void rseq_lock(struct rseq_mutex *lock) +{ + struct rseq_abi *expected = NULL, *self = rseq_get_abi(); + + if (rseq_likely(__atomic_compare_exchange_n(&lock->owner, &expected, self, false, + __ATOMIC_ACQUIRE, __ATOMIC_RELAXED))) + return; + rseq_lock_slowpath(lock); +} + +static void rseq_unlock(struct rseq_mutex *lock) +{ + __atomic_store_n(&lock->owner, NULL, __ATOMIC_RELEASE); + //TODO: integrate with sys_futex and wakeup oldest waiter. +} + +static +void *test_thread(void *arg) +{ + int i; + + if (rseq_register_current_thread()) { + fprintf(stderr, "Error: rseq_register_current_thread(...) failed(%d): %s\n", + errno, strerror(errno)); + abort(); + } + + for (i = 0; i < 1000; i++) { + int var; + + rseq_lock(&lock); + var = RSEQ_READ_ONCE(testvar); + if (var) { + fprintf(stderr, "Unexpected value %d\n", var); + abort(); + } + RSEQ_WRITE_ONCE(testvar, 1); + if (!(i % 10)) + (void) poll(NULL, 0, 10); + else + rseq_barrier(); + RSEQ_WRITE_ONCE(testvar, 0); + rseq_unlock(&lock); + } + + if (rseq_unregister_current_thread()) { + fprintf(stderr, "Error: rseq_unregister_current_thread(...) failed(%d): %s\n", + errno, strerror(errno)); + abort(); + } + return NULL; +} + +int main(int argc, char **argv) +{ + int nr_threads = 5; + pthread_t test_thread_id[nr_threads]; + int i; + + for (i = 0; i < nr_threads; i++) { + pthread_create(&test_thread_id[i], NULL, test_thread, NULL); + } + + for (i = 0; i < nr_threads; i++) { + pthread_join(test_thread_id[i], NULL); + } + + return 0; +}