| From: Kinsey Ho <kinseyho@google.com> |
| Subject: mm: don't hold css->refcnt during traversal |
| Date: Thu, 5 Sep 2024 00:30:51 +0000 |
| |
| To obtain the pointer to the next memcg position, mem_cgroup_iter() |
| currently holds css->refcnt during memcg traversal only to put css->refcnt |
| at the end of the routine. This isn't necessary as an rcu_read_lock is |
| already held throughout the function. The use of the RCU read lock with |
| css_next_descendant_pre() guarantees that sibling linkage is safe without |
| holding a ref on the passed-in @css. |
| |
| Remove css->refcnt usage during traversal by leveraging RCU. |
| |
| Link: https://lkml.kernel.org/r/20240905003058.1859929-3-kinseyho@google.com |
| Signed-off-by: Kinsey Ho <kinseyho@google.com> |
| Reviewed-by: T.J. Mercier <tjmercier@google.com> |
| Cc: Johannes Weiner <hannes@cmpxchg.org> |
| Cc: Michal Hocko <mhocko@kernel.org> |
| Cc: Michal Koutný <mkoutny@suse.com> |
| Cc: Muchun Song <muchun.song@linux.dev> |
| Cc: Roman Gushchin <roman.gushchin@linux.dev> |
| Cc: Shakeel Butt <shakeel.butt@linux.dev> |
| Cc: Tejun Heo <tj@kernel.org> |
| Cc: Yosry Ahmed <yosryahmed@google.com> |
| Cc: Zefan Li <lizefan.x@bytedance.com> |
| Cc: Hugh Dickins <hughd@google.com> |
| Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
| --- |
| |
| mm/memcontrol.c | 18 +----------------- |
| 1 file changed, 1 insertion(+), 17 deletions(-) |
| |
| --- a/mm/memcontrol.c~mm-dont-hold-css-refcnt-during-traversal |
| +++ a/mm/memcontrol.c |
| @@ -1013,20 +1013,7 @@ struct mem_cgroup *mem_cgroup_iter(struc |
| else if (reclaim->generation != iter->generation) |
| goto out_unlock; |
| |
| - while (1) { |
| - pos = READ_ONCE(iter->position); |
| - if (!pos || css_tryget(&pos->css)) |
| - break; |
| - /* |
| - * css reference reached zero, so iter->position will |
| - * be cleared by ->css_released. However, we should not |
| - * rely on this happening soon, because ->css_released |
| - * is called from a work queue, and by busy-waiting we |
| - * might block it. So we clear iter->position right |
| - * away. |
| - */ |
| - (void)cmpxchg(&iter->position, pos, NULL); |
| - } |
| + pos = READ_ONCE(iter->position); |
| } else if (prev) { |
| pos = prev; |
| } |
| @@ -1067,9 +1054,6 @@ struct mem_cgroup *mem_cgroup_iter(struc |
| */ |
| (void)cmpxchg(&iter->position, pos, memcg); |
| |
| - if (pos) |
| - css_put(&pos->css); |
| - |
| if (!memcg) |
| iter->generation++; |
| } |
| _ |