Programming with POSIX® Threads

19-22 Now we decrease counter, which is the number of threads that are required but haven't yet waited on the barrier. When counter reaches 0, no more threads are needed — they're all here and waiting anxiously to continue to the next attraction. Now all we need to do is tell them to wake up. We advance to the next cycle, reset the counter, and broadcast the barrier's condition variable.

28-29 Earlier, I mentioned that a program often needs one thread to perform some cleanup or setup between parallel regions. Each thread could lock a mutex and check a flag so that only one thread would perform the setup. However, the setup may not need additional synchronization, for example, because the other threads will wait at a barrier for the next parallel region, and, in that case, it would be nice to avoid locking an extra mutex.

The barrier_wait function has this capability built into it. One and only one thread will return with the special value of -1 while the others return 0. In this particular implementation, the one that waits last and wakes the others will take the honor, but in principle it is 'unspecified' which thread returns -1. The thread that receives -1 can perform the setup, while others race ahead. If you do not need the special return status, treat -1 as another form of success. The proposed POSIX.1j standard has a similar capability — one (unspecified) thread completing a barrier will return the status BARRIER_SERIAL_THREAD.

35 Any threaded code that uses condition variables should always either support deferred cancellation or disable cancellation. Remember that there are two distinct types of cancellation: deferred and asynchronous. Code that deals with asynchronous cancellation is rare. In general it is difficult or impossible to support asynchronous cancellation in any code that acquires resources (including locking a mutex). Programmers can't assume any function supports asynchronous cancellation unless its documentation specifically says so. Therefore we do not need to worry about asynchronous cancellation.

We could code barrier_wait to deal with deferred cancellation, but that raises difficult questions. How, for example, will the barrier wait ever be satisfied if one of the threads has been canceled? And if it won't be satisfied, what happens to all the other threads that have already waited (or are about to wait) on that barrier? There are various ways to answer these questions. One would be for barrier_wait to record the thread identifiers of all threads waiting on the barrier, and for any thread that's canceled within the wait to cancel all other waiters.

Or we might handle cancellation by setting some special error flag and broadcasting the condition variable, and modifying barrier_wait to return a special error when awakened in that way. However, it makes little sense to cancel one thread that's using a barrier. We're going to disallow it, by disabling cancellation prior to the wait, and restoring the previous state of cancellation afterward. This is the same approach taken by the proposed POSIX.1j standard, by the way—barrier waits are not cancellation points. 42-46 If there are more threads that haven't reached the barrier, we need to wait for them. We do that by waiting on the condition variable until the barrier has advanced to the next cycle — that is, the barrier's cycle no longer matches the local copy.

¦ barrier.c part 3 barrier_wait

1 /*
2 * Wait for all members of a barrier to reach the barrier. When
3 * the count (of remaining members) reaches 0, broadcast to wake
4 * all threads waiting.
5 */
6 int barrier_wait (barrier_t *barrier)
7 {
8 int status, cancel, tmp, cycle;
9
10 if (barrier->valid != BARRIER_VALID)
11 return EINVAL;
12
13 status = pthread_mutex_lock (&barrier->mutex);
14 if (status != 0)
15 return status;
16
17 cycle = barrier->cycle; /* Remember which cycle we're on */
18
19 if (--barrier->counter == 0) {
20 	barrier->cycle = !barrier->cycle;
21 	barrier->counter = barrier->threshold;
22 	status = pthread_cond_broadcast (&barrier->cv);
23 /*
24 * The last thread into the barrier will return status
25 * -1 rather than 0, so that it can be used to perform
26 * some special serial code following the barrier.
27 */
28 	if (status == 0)
29 		status = -1;
30 } else {
31 /*
32 * Wait with cancellation disabled, because barrier_wait
33 * should not be a cancellation point.
34 */
35 pthread_setcancelstate (PTHREAD_CANCEL_DISABLE, &cancel); 36
37 /*
38 * Wait until the barrier's cycle changes, which means
39 * that it has been broadcast, and we don't want to wait
40 * anymore.
41 */
42 while (cycle == barrier->cycle) {
43 	status = pthread_cond_wait (
44 		&barrier->cv, &barrier->mutex);
45 	if (status != 0) break;
46 }
47
48 pthread_setcancelstate (cancel, &tmp);
49 }
50 /*
51 * Ignore an error in unlocking. It shouldn't happen, and
52 * reporting it here would be misleading — the barrier wait
53 * completed, after all, whereas returning, for example,
54 * EINVAL would imply the wait had failed. The next attempt
55 * to use the barrier *will* return an error, or hang, due
56 * to whatever happened to the mutex.
57 */
58 pthread_mutex_unlock (&barrier->mutex);
59 return status; /* error, -1 for waker, or 0 */
60 }

Finally, barrier_main.c is a simple program that uses barriers. Each thread

loops on calculations within a private array. 35,47 At the beginning and end of each iteration, the threads, running function

thread_routine, all wait on a barrier to synchronize the operation. 56-61 At the end of each iteration, the 'lead thread' (the one receiving a -1 result