sched_ext: Fix scx_bpf_dsq_peek() with FIFO DSQs

When removing a task from a FIFO DSQ, we must delete it from the list
before updating dsq->first_task, otherwise the following lookup will
just re-read the same task, leaving first_task pointing to removed
entry.

This issue only affects DSQs operating in FIFO mode, as priority DSQs
correctly update the rbtree before re-evaluating the new first task.

Remove the item from the list before refreshing the first task to
guarantee the correct behavior in FIFO DSQs.

Fixes: 44f5c8ec5b9ad ("sched_ext: Add lockless peek operation for DSQs")
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>

diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
index bde49e47f1e4bd5160f2b6859a4954f5b11082b2..1cfe4b43d31b0dab78e68f96c13926ed4727c56f 100644 (file)
--- a/kernel/sched/ext.c
+++ b/kernel/sched/ext.c
@@ -1044,15 +1044,15 @@ static void task_unlink_from_dsq(struct task_struct *p,
                p->scx.dsq_flags &= ~SCX_TASK_DSQ_ON_PRIQ;
        }
 
+       list_del_init(&p->scx.dsq_list.node);
+       dsq_mod_nr(dsq, -1);
+
        if (!(dsq->id & SCX_DSQ_FLAG_BUILTIN) && dsq->first_task == p) {
                struct task_struct *first_task;
 
                first_task = nldsq_next_task(dsq, NULL, false);
                rcu_assign_pointer(dsq->first_task, first_task);
        }
-
-       list_del_init(&p->scx.dsq_list.node);
-       dsq_mod_nr(dsq, -1);
 }
 
 static void dispatch_dequeue(struct rq *rq, struct task_struct *p)