urcu.git
10 years agoMerge branch 'master' into urcu/ht-shrink
Mathieu Desnoyers [Thu, 29 Sep 2011 21:14:54 +0000 (17:14 -0400)] 
Merge branch 'master' into urcu/ht-shrink

10 years agourcu call_rcu: Use RCU read-side protection for per-cpu call_rcu data
Mathieu Desnoyers [Thu, 29 Sep 2011 21:13:48 +0000 (17:13 -0400)] 
urcu call_rcu: Use RCU read-side protection for per-cpu call_rcu data

A concurrent get_cpu_call_rcu_data(), called by get_call_rcu_data(),
could dereference this pointer without holding any mutex. So this
situation would happen if we have a concurrent call_rcu() executing
while we do the create_all_cpu_call_rcu_data().

I think we would need to put a rcu_dereference() around
per_cpu_call_rcu_data read within get_cpu_call_rcu_data() too.
per_cpu_call_rcu_data should be done with rcu_set_pointer.

Also, a rcu read-side critical section would be required around any
usage of per_cpu_call_rcu_data, and the action of tearing down the
per-cpu data would require to wait for a quiescent state. So we would
basically require that the call_rcu users need to be registered as
RCU reader threads.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agourcu,call_rcu: Cleanup call_rcu_data pointers before use in child
Lai Jiangshan [Thu, 29 Sep 2011 19:56:43 +0000 (15:56 -0400)] 
urcu,call_rcu: Cleanup call_rcu_data pointers before use in child

[ Edit by Mathieu Desnoyers: create maxcpus_reset to handle cases where
  maxcpus is 0 and -1, depending on the configuration. ]

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agourcu,call_rcu: avoid create call_rcu_data for child when unneed
Lai Jiangshan [Thu, 29 Sep 2011 17:55:11 +0000 (13:55 -0400)] 
urcu,call_rcu: avoid create call_rcu_data for child when unneed

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agourcu,defer_rcu: Make defer_rcu encoding more compact for marker
Lai Jiangshan [Thu, 29 Sep 2011 17:47:13 +0000 (13:47 -0400)] 
urcu,defer_rcu: Make defer_rcu encoding more compact for marker

When the function changes (and the function is aligned), and only the
data is the marker, we can get away with using only 2 pointers rather
than 3.

[ Edit by Mathieu Desnoyers: patch cleanup, changelog updates ]

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agourcu_defer: Use cancellation flag instead of pthread_cancel()
Mathieu Desnoyers [Thu, 29 Sep 2011 17:28:04 +0000 (13:28 -0400)] 
urcu_defer: Use cancellation flag instead of pthread_cancel()

- Provides better control over cancellation point location.
- Set the futex to 0 before exiting the defer thread.

This patch combines and enhances patches from Lai Jiangshan:
  urcu,defer_rcu: fix missing respond to a cancellation request
  urcu,defer_rcu: Avoid thread exit unexpected

Reported-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agourcu,call_rcu: protects call_rcu_data_list when remove node
Lai Jiangshan [Thu, 29 Sep 2011 17:04:12 +0000 (13:04 -0400)] 
urcu,call_rcu: protects call_rcu_data_list when remove node

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoMerge branch 'master' into urcu/ht-shrink
Mathieu Desnoyers [Wed, 28 Sep 2011 20:03:40 +0000 (16:03 -0400)] 
Merge branch 'master' into urcu/ht-shrink

10 years agoCreate default call rcu data upon per-cpu call-rcu teardown
Mathieu Desnoyers [Wed, 28 Sep 2011 20:03:13 +0000 (16:03 -0400)] 
Create default call rcu data upon per-cpu call-rcu teardown

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash test: fix 32-bit hash
Mathieu Desnoyers [Wed, 28 Sep 2011 17:46:23 +0000 (13:46 -0400)] 
rculfhash test: fix 32-bit hash

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash test: Use get first/get next to delete all entries
Mathieu Desnoyers [Wed, 28 Sep 2011 03:27:17 +0000 (23:27 -0400)] 
rculfhash test: Use get first/get next to delete all entries

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: fix get first / get next iterator
Mathieu Desnoyers [Wed, 28 Sep 2011 03:27:02 +0000 (23:27 -0400)] 
rculfhash: fix get first / get next iterator

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoFix handling of systems without sysconf nr possible cpu support
Mathieu Desnoyers [Wed, 28 Sep 2011 00:01:51 +0000 (20:01 -0400)] 
Fix handling of systems without sysconf nr possible cpu support

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash needs local config.h
Mathieu Desnoyers [Tue, 27 Sep 2011 23:54:08 +0000 (19:54 -0400)] 
rculfhash needs local config.h

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: update header documentation
Mathieu Desnoyers [Tue, 27 Sep 2011 21:47:35 +0000 (17:47 -0400)] 
rculfhash: update header documentation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash test: move init node outside of rcu read-side c.s. (unneeded protection)
Mathieu Desnoyers [Tue, 27 Sep 2011 21:39:30 +0000 (17:39 -0400)] 
rculfhash test: move init node outside of rcu read-side c.s. (unneeded protection)

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoAdd cds_lfht_first/cds_lfht_next for hash table iteration
Mathieu Desnoyers [Tue, 27 Sep 2011 21:24:55 +0000 (17:24 -0400)] 
Add cds_lfht_first/cds_lfht_next for hash table iteration

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: rename _next into _next_duplicate
Mathieu Desnoyers [Tue, 27 Sep 2011 18:45:41 +0000 (14:45 -0400)] 
rculfhash: rename _next into _next_duplicate

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: document use of caa_container_of()
Mathieu Desnoyers [Tue, 27 Sep 2011 15:19:21 +0000 (11:19 -0400)] 
rculfhash: document use of caa_container_of()

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: cleanup includes
Mathieu Desnoyers [Mon, 26 Sep 2011 18:48:30 +0000 (14:48 -0400)] 
rculfhash: cleanup includes

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoMerge branch 'master' into urcu/ht-shrink
Mathieu Desnoyers [Thu, 22 Sep 2011 15:01:54 +0000 (11:01 -0400)] 
Merge branch 'master' into urcu/ht-shrink

10 years agopowerpc: use __NO_LWSYNC__ check to use appropriate lwsync/sync opcode
Mathieu Desnoyers [Thu, 22 Sep 2011 15:00:14 +0000 (11:00 -0400)] 
powerpc: use __NO_LWSYNC__ check to use appropriate lwsync/sync opcode

We already used it in uatomic code, move it to arch ppc.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoMerge branch 'master' into urcu/ht-shrink
Mathieu Desnoyers [Thu, 22 Sep 2011 14:50:04 +0000 (10:50 -0400)] 
Merge branch 'master' into urcu/ht-shrink

10 years agocmm: provide lightweight smp_rmb/smp_wmb on PPC
Paolo Bonzini [Thu, 22 Sep 2011 09:12:44 +0000 (05:12 -0400)] 
cmm: provide lightweight smp_rmb/smp_wmb on PPC

lwsync orders loads in cacheable memory with respect to other loads,
and stores in cacheable memory with respect to other stores.  Use it
to implement smp_rmb/smp_wmb.

The heavy-weight sync is still used for the "full" rmb/wmb operations,
as well as for smp_mb.

[ Edit by Mathieu Desnoyers: rephrased the comments around the memory
  barriers. ]

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: factor out add_replace and replace
Mathieu Desnoyers [Thu, 22 Sep 2011 09:03:36 +0000 (05:03 -0400)] 
rculfhash: factor out add_replace and replace

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoatomic: provide seq_cst semantics on powerpc
Mathieu Desnoyers [Thu, 22 Sep 2011 00:25:20 +0000 (20:25 -0400)] 
atomic: provide seq_cst semantics on powerpc

We provide sequential consistency semantic over all architectures for
cmpxchg and add_return family of primitives, but the powerpc
implementation does not match that.

Change the isync after the atomic primitives to sync, and explain the
scheme.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash tests: make node count RCU aware
Mathieu Desnoyers [Wed, 21 Sep 2011 17:59:31 +0000 (13:59 -0400)] 
rculfhash tests: make node count RCU aware

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: set next to NULL when node is NULL
Mathieu Desnoyers [Wed, 21 Sep 2011 17:59:09 +0000 (13:59 -0400)] 
rculfhash: set next to NULL when node is NULL

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: approximation can be negative
Mathieu Desnoyers [Wed, 21 Sep 2011 14:50:58 +0000 (10:50 -0400)] 
rculfhash: approximation can be negative

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: min size only needed on shrink, take nr cpus into account
Mathieu Desnoyers [Wed, 21 Sep 2011 14:33:28 +0000 (10:33 -0400)] 
rculfhash: min size only needed on shrink, take nr cpus into account

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: type the ht count approx as long
Mathieu Desnoyers [Wed, 21 Sep 2011 14:22:39 +0000 (10:22 -0400)] 
rculfhash: type the ht count approx as long

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: handle small and negative table size approximation
Mathieu Desnoyers [Wed, 21 Sep 2011 14:21:21 +0000 (10:21 -0400)] 
rculfhash: handle small and negative table size approximation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: fix node approx counting
Mathieu Desnoyers [Wed, 21 Sep 2011 13:48:05 +0000 (09:48 -0400)] 
rculfhash: fix node approx counting

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: output approximation of number of nodes in counting
Mathieu Desnoyers [Wed, 21 Sep 2011 13:24:17 +0000 (09:24 -0400)] 
rculfhash: output approximation of number of nodes in counting

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash cleanup: count percpu deletes in the positive range
Mathieu Desnoyers [Wed, 21 Sep 2011 12:44:49 +0000 (08:44 -0400)] 
rculfhash cleanup: count percpu deletes in the positive range

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: node alignment is back to 4 bytes
Mathieu Desnoyers [Wed, 21 Sep 2011 05:11:05 +0000 (01:11 -0400)] 
rculfhash: node alignment is back to 4 bytes

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: remove now unneeded gc flag (combine with removed)
Mathieu Desnoyers [Wed, 21 Sep 2011 04:45:11 +0000 (00:45 -0400)] 
rculfhash: remove now unneeded gc flag (combine with removed)

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: implement lock-free replacement
Mathieu Desnoyers [Wed, 21 Sep 2011 04:40:38 +0000 (00:40 -0400)] 
rculfhash: implement lock-free replacement

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: support replacement operation
Mathieu Desnoyers [Wed, 21 Sep 2011 03:05:08 +0000 (23:05 -0400)] 
rculfhash: support replacement operation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: API rename: remove -> del
Mathieu Desnoyers [Tue, 20 Sep 2011 23:34:49 +0000 (19:34 -0400)] 
rculfhash: API rename: remove -> del

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: spawn only number of threads required for resize
Mathieu Desnoyers [Tue, 20 Sep 2011 21:50:58 +0000 (17:50 -0400)] 
rculfhash: spawn only number of threads required for resize

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: parallelize resize
Mathieu Desnoyers [Tue, 20 Sep 2011 20:46:22 +0000 (16:46 -0400)] 
rculfhash: parallelize resize

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: document flags
Mathieu Desnoyers [Mon, 19 Sep 2011 20:22:49 +0000 (16:22 -0400)] 
rculfhash: document flags

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: comment file inclusion order
Mathieu Desnoyers [Mon, 19 Sep 2011 20:12:43 +0000 (16:12 -0400)] 
rculfhash: comment file inclusion order

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: simplify lfht_new API with inline wrapper
Mathieu Desnoyers [Mon, 19 Sep 2011 20:07:18 +0000 (16:07 -0400)] 
rculfhash: simplify lfht_new API with inline wrapper

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: add TODO for resize worker threads urcu/ht-shrink-help
Mathieu Desnoyers [Mon, 19 Sep 2011 17:27:08 +0000 (13:27 -0400)] 
rculfhash: add TODO for resize worker threads

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: Remove leftover assertions
Mathieu Desnoyers [Mon, 19 Sep 2011 16:57:12 +0000 (12:57 -0400)] 
rculfhash: Remove leftover assertions

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: merge table hash and link stages into populate
Mathieu Desnoyers [Mon, 19 Sep 2011 16:56:04 +0000 (12:56 -0400)] 
rculfhash: merge table hash and link stages into populate

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoMerge branch 'urcu/ht-shrink-help' into urcu/ht-shrink
Mathieu Desnoyers [Mon, 19 Sep 2011 16:49:23 +0000 (12:49 -0400)] 
Merge branch 'urcu/ht-shrink-help' into urcu/ht-shrink

10 years agorculfhash: remove helper scheme
Mathieu Desnoyers [Mon, 19 Sep 2011 16:45:26 +0000 (12:45 -0400)] 
rculfhash: remove helper scheme

There is a trade-off to consider here:
- The helper scheme would require the helpers to allocate the memory for
  dummy nodes themself for correctness (the current implementation is
  buggy because lookups consider an half-linked dummy node to be ready for
  use), which does not work with a cache-efficient per-level array of
  dummy nodes.
- We want cache-efficiency, so we want to keep the per-level array
  allocation.

Therefore, letting insert/removal/lookups help expand is not the way to
go here. This patch removes the helping scheme.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoavoid leaking crdp for failed path
Lai Jiangshan [Thu, 15 Sep 2011 15:24:03 +0000 (11:24 -0400)] 
avoid leaking crdp for failed path

[ Comment: now that set_cpu_call_rcu_data() is not racy and detects
  overwrites, we can effectively trust its return value and free the
  crdp if already set. ]

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoReturn -EEXIST when the old cpu call_rcu_data has not been removed
Lai Jiangshan [Thu, 15 Sep 2011 15:22:03 +0000 (11:22 -0400)] 
Return -EEXIST when the old cpu call_rcu_data has not been removed

To make it matches the comments.

It is the caller's responsibility to use
set_cpu_call_rcu_data(cpu, NULL) to remove the CPU's
call_rcu_data structure and dispose it.

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoprotect writing to per_cpu_call_rcu_data[*]
Lai Jiangshan [Thu, 15 Sep 2011 15:20:29 +0000 (11:20 -0400)] 
protect writing to per_cpu_call_rcu_data[*]

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agowake up default call_rcu thread after we move the leftover callbacks
Lai Jiangshan [Thu, 15 Sep 2011 15:19:12 +0000 (11:19 -0400)] 
wake up default call_rcu thread after we move the leftover callbacks

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoavoid memory leak in call_rcu_data_free()
Lai Jiangshan [Thu, 15 Sep 2011 15:17:41 +0000 (11:17 -0400)] 
avoid memory leak in call_rcu_data_free()

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agourcu call_rcu: fix use after free()
Lai Jiangshan [Thu, 15 Sep 2011 15:14:43 +0000 (11:14 -0400)] 
urcu call_rcu: fix use after free()

call_rcu_after_fork_child() needs to use cds_list_for_each_entry_safe to
safely iterate on the list as its item is being freed.

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agouse get_cpu_call_rcu_data() for get_call_rcu_data()
Lai Jiangshan [Thu, 15 Sep 2011 15:10:31 +0000 (11:10 -0400)] 
use get_cpu_call_rcu_data() for get_call_rcu_data()

[ Impact: refactor duplicated code ]

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoinit maxcpus before use
Lai Jiangshan [Thu, 15 Sep 2011 15:01:38 +0000 (11:01 -0400)] 
init maxcpus before use

[ Edit:
  Covers the per-cpu call_rcu data setup (not all_cpus helper, which is
  why we did not trigger it in our tests. ]

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agocall_rcu implementation: add missing static
Lai Jiangshan [Thu, 15 Sep 2011 14:56:56 +0000 (10:56 -0400)] 
call_rcu implementation: add missing static

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agocall_rcu implementation: add missing static
Lai Jiangshan [Thu, 15 Sep 2011 14:56:56 +0000 (10:56 -0400)] 
call_rcu implementation: add missing static

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoDocument QSBR interaction with mutexes
Mathieu Desnoyers [Thu, 15 Sep 2011 14:04:30 +0000 (10:04 -0400)] 
Document QSBR interaction with mutexes

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoDocument QSBR interaction with mutexes
Mathieu Desnoyers [Thu, 15 Sep 2011 14:04:30 +0000 (10:04 -0400)] 
Document QSBR interaction with mutexes

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash test: exit upon failure
Mathieu Desnoyers [Thu, 15 Sep 2011 10:41:46 +0000 (06:41 -0400)] 
rculfhash test: exit upon failure

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: fix add_unique node counting
Mathieu Desnoyers [Thu, 15 Sep 2011 10:29:31 +0000 (06:29 -0400)] 
rculfhash: fix add_unique node counting

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash test: cast rand_r return value directly
Mathieu Desnoyers [Wed, 14 Sep 2011 21:24:00 +0000 (17:24 -0400)] 
rculfhash test: cast rand_r return value directly

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash test: add verbosity
Mathieu Desnoyers [Wed, 14 Sep 2011 21:19:07 +0000 (17:19 -0400)] 
rculfhash test: add verbosity

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash test: allow different size for lookup, write, init
Mathieu Desnoyers [Wed, 14 Sep 2011 18:27:48 +0000 (14:27 -0400)] 
rculfhash test: allow different size for lookup, write, init

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoqsbr vs call_rcu : remove exit assertion
Mathieu Desnoyers [Wed, 14 Sep 2011 17:36:12 +0000 (13:36 -0400)] 
qsbr vs call_rcu : remove exit assertion

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: validate lookups
Mathieu Desnoyers [Wed, 14 Sep 2011 17:17:16 +0000 (13:17 -0400)] 
rculfhash: validate lookups

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash test: add pool offsets
Mathieu Desnoyers [Wed, 14 Sep 2011 17:04:35 +0000 (13:04 -0400)] 
rculfhash test: add pool offsets

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: with resize stop and helpers, min size can now be 1
Mathieu Desnoyers [Wed, 14 Sep 2011 02:40:51 +0000 (22:40 -0400)] 
rculfhash: with resize stop and helpers, min size can now be 1

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: help scheme: fix end node, insertion, and lookups
Mathieu Desnoyers [Wed, 14 Sep 2011 02:27:15 +0000 (22:27 -0400)] 
rculfhash: help scheme: fix end node, insertion, and lookups

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: break in-progress resize when target size change (between levels)
Mathieu Desnoyers [Wed, 14 Sep 2011 00:33:15 +0000 (20:33 -0400)] 
rculfhash: break in-progress resize when target size change (between levels)

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: break in-progress resize when target size change (between levels)
Mathieu Desnoyers [Wed, 14 Sep 2011 00:33:15 +0000 (20:33 -0400)] 
rculfhash: break in-progress resize when target size change (between levels)

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: make add/removal help expand
Mathieu Desnoyers [Tue, 13 Sep 2011 23:16:28 +0000 (19:16 -0400)] 
rculfhash: make add/removal help expand

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: comment shrink operation
Mathieu Desnoyers [Tue, 13 Sep 2011 22:32:35 +0000 (18:32 -0400)] 
rculfhash: comment shrink operation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: remove verified TODOs
Mathieu Desnoyers [Tue, 13 Sep 2011 22:18:52 +0000 (18:18 -0400)] 
rculfhash: remove verified TODOs

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: put thread offline before taking mutex (fix G.P. deadlock)
Mathieu Desnoyers [Tue, 13 Sep 2011 22:12:38 +0000 (18:12 -0400)] 
rculfhash: put thread offline before taking mutex (fix G.P. deadlock)

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoMerge branch 'master' into urcu/ht-shrink-help
Mathieu Desnoyers [Tue, 13 Sep 2011 21:34:19 +0000 (17:34 -0400)] 
Merge branch 'master' into urcu/ht-shrink-help

Conflicts:
urcu-qsbr.c

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agourcu-pointer: implement URCU_FORCE_CAST for C++ compatibility of urcu-pointer.h
Mathieu Desnoyers [Tue, 13 Sep 2011 21:00:08 +0000 (17:00 -0400)] 
urcu-pointer: implement URCU_FORCE_CAST for C++ compatibility of urcu-pointer.h

We need to be careful with those, so we do not break aliasing. Our
use-case is to cast back and forth between the same type and a void *
(or void **) type when we pass pointers to C functions. As we cast back
to the same type when the pointer is returned from the function,
aliasing should still work.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: use single init-time allocation for order table
Mathieu Desnoyers [Tue, 13 Sep 2011 18:44:25 +0000 (14:44 -0400)] 
rculfhash: use single init-time allocation for order table

This simplifies management of hash table resizes. Thanks to Josh
Triplett for suggesting this.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agourcu-qsbr: use rcu_thread_offline/rcu_thread_online instead of inlining them
Paolo Bonzini [Tue, 13 Sep 2011 17:49:28 +0000 (13:49 -0400)] 
urcu-qsbr: use rcu_thread_offline/rcu_thread_online instead of inlining them

* Mathieu Desnoyers wrote:

> Just to let you know that I pushed two updates into urcu: one fixes a
> grace period hang caused by a missing wakeup in the synchronize_rcu
> QSBR code. This appears to hit us due to the more fine-grained wakeup
> code brought by Paolo. The wakeup was really missing from the
> synchronize_rcu code (so Paolo's code just triggered an existing
> problem). I thought it would be good to let you know the effect: grace
> periods are delayed forever. This problem never appeared in a release
> (I caught it before).

Good catch.  Why not use rcu_thread_offline/online in synchronize_rcu,
instead of touching rcu_reader.ctr directly?  I had this in my QEMU
branch but hadn't posted yet because it was meant as a cleanup only.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoPair all_cpu call_rcu create with free
Mathieu Desnoyers [Sun, 11 Sep 2011 06:45:59 +0000 (23:45 -0700)] 
Pair all_cpu call_rcu create with free

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash test: add missing call_rcu per-cpu worker threads teardown
Mathieu Desnoyers [Sun, 11 Sep 2011 06:39:03 +0000 (23:39 -0700)] 
rculfhash test: add missing call_rcu per-cpu worker threads teardown

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: disable poison and remove debugging code
Mathieu Desnoyers [Sun, 11 Sep 2011 06:29:54 +0000 (23:29 -0700)] 
rculfhash: disable poison and remove debugging code

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agocall_rcu: register work threads as rcu readers
Mathieu Desnoyers [Sun, 11 Sep 2011 05:02:58 +0000 (22:02 -0700)] 
call_rcu: register work threads as rcu readers

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoQSBR: add missing wakeups in synchronize_rcu code
Mathieu Desnoyers [Sun, 11 Sep 2011 06:03:09 +0000 (23:03 -0700)] 
QSBR: add missing wakeups in synchronize_rcu code

synchronize_rcu go into offline mode during grace period. It duplicates
the rcu_thread_online/offline code, and therefore adding the required
wake_up_gp() is required there too.

Failure to do so leads to grace period hangs.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agoQSBR: add missing wakeups in synchronize_rcu code
Mathieu Desnoyers [Sun, 11 Sep 2011 06:03:09 +0000 (23:03 -0700)] 
QSBR: add missing wakeups in synchronize_rcu code

synchronize_rcu go into offline mode during grace period. It duplicates
the rcu_thread_online/offline code, and therefore adding the required
wake_up_gp() is required there too.

Failure to do so leads to grace period hangs.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: hold rcu read-side lock in resize
Mathieu Desnoyers [Sun, 11 Sep 2011 05:09:55 +0000 (22:09 -0700)] 
rculfhash: hold rcu read-side lock in resize

We need to hold the RCU read-side lock in resize to protect against ABA
caused by re-use of nodes after going through the memory allocator
through a call_rcu performed on another CPU.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agocall_rcu: register work threads as rcu readers
Mathieu Desnoyers [Sun, 11 Sep 2011 05:02:58 +0000 (22:02 -0700)] 
call_rcu: register work threads as rcu readers

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: add list iteration tracer in gc
Mathieu Desnoyers [Sun, 11 Sep 2011 03:53:30 +0000 (20:53 -0700)] 
rculfhash: add list iteration tracer in gc

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: poison memory before free (for testing)
Mathieu Desnoyers [Sun, 11 Sep 2011 02:58:11 +0000 (19:58 -0700)] 
rculfhash: poison memory before free (for testing)

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: reinstate i - 1 for shrink
Mathieu Desnoyers [Sun, 11 Sep 2011 02:33:56 +0000 (19:33 -0700)] 
rculfhash: reinstate i - 1 for shrink

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: add assertion in path compression
Mathieu Desnoyers [Sun, 11 Sep 2011 01:32:45 +0000 (18:32 -0700)] 
rculfhash: add assertion in path compression

Detects incorrect usage of path compression.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: fix fini_table for size 1
Mathieu Desnoyers [Sun, 11 Sep 2011 01:14:59 +0000 (18:14 -0700)] 
rculfhash: fix fini_table for size 1

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: Fix off-by-one in fini_table
Mathieu Desnoyers [Sun, 11 Sep 2011 01:11:56 +0000 (18:11 -0700)] 
rculfhash: Fix off-by-one in fini_table

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash: add asserts and comments
Mathieu Desnoyers [Sat, 10 Sep 2011 23:52:53 +0000 (16:52 -0700)] 
rculfhash: add asserts and comments

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agocmm: do not generate code for smp_rmb/smp_wmb on x86_64
Paolo Bonzini [Sat, 10 Sep 2011 19:29:56 +0000 (12:29 -0700)] 
cmm: do not generate code for smp_rmb/smp_wmb on x86_64

We can assume, on x86, that no accesses to write-combining memory occur,
and also that there are no non-temporal load/stores (people would
presumably write those with assembly or intrinsics and put appropriate
lfence/sfence manually). In this case rmb and wmb are no-ops on x86.
according to the updated x86 memory models:

INTEL CORPORATION. Intel 64 Architecture Memory Ordering White Paper,
2007.
http://developer.intel.com/products/processor/manuals/318147.pdf

ADVANCED MICRO DEVICES. AMD x86-64 Architecture Programmer’s Manual
Volume 2: System Programming, 2007.

Paul E. McKenney. Memory Ordering in Modern Microprocessors
www.rdrop.com/users/paulmck/scalability/paper/ordering.2007.09.19a.pdf

x86 does not reorder loads vs loads, and stores vs stores when using
normal memory accesses, with the notable exceptions of Pentium Pro
(reorders reads) and WinChip (reorders writes). Therefore, it is safe
not to emit fence instructions for x86_64 cmm_smp_rmb()/cmm_smp_wmb(),
but we leave the memory fences in place for x86_32 for those two
special-cases.

Define cmm_smp_rmb and cmm_smp_wmb to be the "common" operations that
do not require fence instruction, while leaving cmm_rmb and cmm_wmb in
place for more sophisticated uses.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agocmm: let per-arch files provide cmm_smp_* barriers
Paolo Bonzini [Sat, 10 Sep 2011 19:26:19 +0000 (12:26 -0700)] 
cmm: let per-arch files provide cmm_smp_* barriers

x86 instructions lfence and sfence are rarely needed, thus we want
the cmm_smp_rmb/cmm_smp_wmb macros to be simple compiler barriers.
So, let the per-arch files override the default definitions in
arch/generic.h.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
10 years agorculfhash test: -A option for auto resize
Mathieu Desnoyers [Fri, 9 Sep 2011 21:43:58 +0000 (17:43 -0400)] 
rculfhash test: -A option for auto resize

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
This page took 0.064517 seconds and 4 git commands to generate.