Minki/linux
1
0
Fork 1
mirror of https://github.com/torvalds/linux.git synced 2024-11-25 13:41:51 +00:00
Code Issues Packages Projects Releases Wiki Activity

rhashtable: improve rhashtable_walk stability when stop/start used.

Browse Source 
When a walk of an rhashtable is interrupted with rhastable_walk_stop()
and then rhashtable_walk_start(), the location to restart from is based
on a 'skip' count in the current hash chain, and this can be incorrect
if insertions or deletions have happened.  This does not happen when
the walk is not stopped and started as iter->p is a placeholder which
is safe to use while holding the RCU read lock.

In rhashtable_walk_start() we can revalidate that 'p' is still in the
same hash chain.  If it isn't then the current method is still used.

With this patch, if a rhashtable walker ensures that the current
object remains in the table over a stop/start period (possibly by
elevating the reference count if that is sufficient), it can be sure
that a walk will not miss objects that were in the hashtable for the
whole time of the walk.

rhashtable_walk_start() may not find the object even though it is
still in the hashtable if a rehash has moved it to a new table.  In
this case it will (eventually) get -EAGAIN and will need to proceed
through the whole table again to be sure to see everything at least
once.

Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NeilBrown <neilb@suse.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This commit is contained in:
NeilBrown 2018-04-24 08:29:13 +10:00 committed by David S. Miller
parent b41cc04b66
commit 5d240a8936
1 changed files with 41 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

44
lib/rhashtable.c
View File

@ -727,6 +727,7 @@ int rhashtable_walk_start_check(struct rhashtable_iter *iter)
__acquires(RCU)
{
struct rhashtable *ht = iter->ht;
bool rhlist = ht->rhlist;
rcu_read_lock();
@ -735,13 +736,52 @@ int rhashtable_walk_start_check(struct rhashtable_iter *iter)
list_del(&iter->walker.list);
spin_unlock(&ht->lock);
if (!iter->walker.tbl && !iter->end_of_table) {
if (iter->end_of_table)
return 0;
if (!iter->walker.tbl) {
iter->walker.tbl = rht_dereference_rcu(ht->tbl, ht);
iter->slot = 0;
iter->skip = 0;
return -EAGAIN;
}
if (iter->p && !rhlist) {
/*
* We need to validate that 'p' is still in the table, and
* if so, update 'skip'
*/
struct rhash_head *p;
int skip = 0;
rht_for_each_rcu(p, iter->walker.tbl, iter->slot) {
skip++;
if (p == iter->p) {
iter->skip = skip;
goto found;
}
}
iter->p = NULL;
} else if (iter->p && rhlist) {
/* Need to validate that 'list' is still in the table, and
* if so, update 'skip' and 'p'.
*/
struct rhash_head *p;
struct rhlist_head *list;
int skip = 0;
rht_for_each_rcu(p, iter->walker.tbl, iter->slot) {
for (list = container_of(p, struct rhlist_head, rhead);
list;
list = rcu_dereference(list->next)) {
skip++;
if (list == iter->list) {
iter->p = p;
skip = skip;
goto found;
}
}
}
iter->p = NULL;
}
found:
return 0;
}
EXPORT_SYMBOL_GPL(rhashtable_walk_start_check);
@ -917,8 +957,6 @@ void rhashtable_walk_stop(struct rhashtable_iter *iter)
iter->walker.tbl = NULL;
spin_unlock(&ht->lock);
iter->p = NULL;
out:
rcu_read_unlock();
}