4 by Mathieu Desnoyers and Paul E. McKenney
14 This must be called before any of the following functions
19 void rcu_read_lock(void);
22 Begin an RCU read-side critical section. These critical
23 sections may be nested.
27 void rcu_read_unlock(void);
30 End an RCU read-side critical section.
34 void rcu_register_thread(void);
37 Each thread must invoke this function before its first call to
38 `rcu_read_lock()`. Threads that never call `rcu_read_lock()` need
39 not invoke this function. In addition, `rcu-bp` ("bullet proof"
40 RCU) does not require any thread to invoke `rcu_register_thread()`.
44 void rcu_unregister_thread(void);
47 Each thread that invokes `rcu_register_thread()` must invoke
48 `rcu_unregister_thread()` before `invoking pthread_exit()`
49 or before returning from its top-level function.
53 void synchronize_rcu(void);
56 Wait until every pre-existing RCU read-side critical section
57 has completed. Note that this primitive will not necessarily
58 wait for RCU read-side critical sections that have not yet
59 started: this is not a reader-writer lock. The duration
60 actually waited is called an RCU grace period.
64 void call_rcu(struct rcu_head *head,
65 void (*func)(struct rcu_head *head));
68 Registers the callback indicated by "head". This means
69 that `func` will be invoked after the end of a future
70 RCU grace period. The `rcu_head` structure referenced
71 by `head` will normally be a field in a larger RCU-protected
72 structure. A typical implementation of `func` is as
76 void func(struct rcu_head *head)
78 struct foo *p = container_of(head, struct foo, rcu);
84 This RCU callback function can be registered as follows
85 given a pointer `p` to the enclosing structure:
88 call_rcu(&p->rcu, func);
91 `call_rcu` should be called from registered RCU read-side threads.
92 For the QSBR flavor, the caller should be online.
96 void rcu_barrier(void);
99 Wait for all `call_rcu()` work initiated prior to `rcu_barrier()` by
100 _any_ thread on the system to have completed before `rcu_barrier()`
101 returns. `rcu_barrier()` should never be called from a `call_rcu()`
102 thread. This function can be used, for instance, to ensure that
103 all memory reclaim involving a shared object has completed
104 before allowing `dlclose()` of this shared object to complete.
108 struct call_rcu_data *create_call_rcu_data(unsigned long flags,
112 Returns a handle that can be passed to the following
113 primitives. The `flags` argument can be zero, or can be
114 `URCU_CALL_RCU_RT` if the worker threads associated with the
115 new helper thread are to get real-time response. The argument
116 `cpu_affinity` specifies a CPU on which the `call_rcu` thread should
117 be affined to. It is ignored if negative.
121 void call_rcu_data_free(struct call_rcu_data *crdp);
124 Terminates a `call_rcu()` helper thread and frees its associated
125 data. The caller must have ensured that this thread is no longer
126 in use, for example, by passing `NULL` to `set_thread_call_rcu_data()`
127 and `set_cpu_call_rcu_data()` as required.
131 struct call_rcu_data *get_default_call_rcu_data(void);
134 Returns the handle for the default `call_rcu()` helper thread.
135 Creates it if necessary.
139 struct call_rcu_data *get_cpu_call_rcu_data(int cpu);
142 Returns the handle for the current CPU's `call_rcu()` helper
143 thread, or `NULL` if the current CPU has no helper thread
144 currently assigned. The call to this function and use of the
145 returned `call_rcu_data` should be protected by RCU read-side
150 struct call_rcu_data *get_thread_call_rcu_data(void);
153 Returns the handle for the current thread's hard-assigned
154 `call_rcu()` helper thread, or `NULL` if the current thread is
155 instead using a per-CPU or the default helper thread.
159 struct call_rcu_data *get_call_rcu_data(void);
162 Returns the handle for the current thread's `call_rcu()` helper
163 thread, which is either, in increasing order of preference:
164 per-thread hard-assigned helper thread, per-CPU helper thread,
165 or default helper thread. `get_call_rcu_data` should be called
166 from registered RCU read-side threads. For the QSBR flavor, the
167 caller should be online.
171 pthread_t get_call_rcu_thread(struct call_rcu_data *crdp);
174 Returns the helper thread's pthread identifier linked to a call
175 rcu helper thread data.
179 void set_thread_call_rcu_data(struct call_rcu_data *crdp);
182 Sets the current thread's hard-assigned `call_rcu()` helper to the
183 handle specified by `crdp`. Note that `crdp` can be `NULL` to
184 disassociate this thread from its helper. Once a thread is
185 disassociated from its helper, further `call_rcu()` invocations
186 use the current CPU's helper if there is one and the default
191 int set_cpu_call_rcu_data(int cpu, struct call_rcu_data *crdp);
194 Sets the specified CPU's `call_rcu()` helper to the handle
195 specified by `crdp`. Again, `crdp` can be `NULL` to disassociate
196 this CPU from its helper thread. Once a CPU has been
197 disassociated from its helper, further `call_rcu()` invocations
198 that would otherwise have used this CPU's helper will instead
199 use the default helper.
201 The caller must wait for a grace-period to pass between return from
202 `set_cpu_call_rcu_data()` and call to `call_rcu_data_free()` passing the
203 previous call rcu data as argument.
207 int create_all_cpu_call_rcu_data(unsigned long flags);
210 Creates a separate `call_rcu()` helper thread for each CPU.
211 After this primitive is invoked, the global default `call_rcu()`
212 helper thread will not be called.
214 The `set_thread_call_rcu_data()`, `set_cpu_call_rcu_data()`, and
215 `create_all_cpu_call_rcu_data()` functions may be combined to set up
216 pretty much any desired association between worker and `call_rcu()`
217 helper threads. If a given executable calls only `call_rcu()`,
218 then that executable will have only the single global default
219 `call_rcu()` helper thread. This will suffice in most cases.
223 void free_all_cpu_call_rcu_data(void);
226 Clean up all the per-CPU `call_rcu` threads. Should be paired with
227 `create_all_cpu_call_rcu_data()` to perform teardown. Note that
228 this function invokes `synchronize_rcu()` internally, so the
229 caller should be careful not to hold mutexes (or mutexes within a
230 dependency chain) that are also taken within a RCU read-side
231 critical section, or in a section where QSBR threads are online.
235 void call_rcu_before_fork_parent(void);
236 void call_rcu_after_fork_parent(void);
237 void call_rcu_after_fork_child(void);
240 Should be used as `pthread_atfork()` handler for programs using
241 `call_rcu` and performing `fork()` or `clone()` without a following