CMPSC 311, Intro to Threads

CMPSC 311, Introduction to Systems Programming

Intro to Posix Threads

Reading

CS:APP

Sec. 1.7.2, Threads
Sec. 1.9.1, Concurrency and Parallelism
Ch. 12, Concurrent Programming, Intro
Sec. 12.1, Concurrent Programming with Processes (depends on Ch. 11, which we will cover next)
Sec. 12.2, Concurrent Programming with I/O Multiplexing (same)
Sec. 12.3, Concurrent Programming with Threads
Sec. 12.4, Shared Variables in Threaded Programs
Skim the rest of Ch. 12 - some of it is used in the Print Server example here, and it's a good preview for CMPSC 473.

APUE, Ch. 11, 12

Processes vs. Threads

Processes vs.
Threads

Processes	Threads
fork	create
exec	create
wait	join
exit	exit

Processes	Threads
`unistd.h, stdlib.h`	`pthread.h`
`pid_t`	`pthread_t`
`pid_t getpid()`	`pthread_t pthread_self(void)`
`pid_t getppid()`	`---`
`pid_t fork(void)int exec..(...)`	`int pthread_create(...)`
`pid_t wait(...) pid_t waitpid(...)`	`int pthread_join(...)`
`void exit(int status)`	`void pthread_exit(...)`

Include files

#include <pthread.h>

Create and start a new thread

void * start_func(void * ptr) { ... }

int pthread_create(
pthread_t * restrict thread_id,
const pthread_attr_t * restrict attr,
void *(*start_routine)(void *),
void * restrict arg
);

return value from pthread_create(), success or failure indicator
*thread_id, output variable, identifier of newly-created thread
*attr, input variable, a struct, thread attributes to be used with newly-created thread

default, NULL

when the thread begins to execute, it is running (*start_routine)(arg)
the newly-created thread executes concurrently with all the other threads in the process

Terminate a thread

void pthread_exit(void *value_ptr);
or, return from the start function
or, some other thread calls pthread_cancel()

Wait for a thread to complete

int pthread_join(
pthread_t thread_id,
void **status
);

Wait for all threads to complete, then terminate the process, in main()

call pthread_join() repeatedly, then call exit()
or, call pthread_exit()

Compile, Solaris

cc -mt [ flag... ] file... -lpthread [ library... ]

Miscellaneous

Threads can run detached, which means they run independently and don't need a join.
Each thread maintains its own stack, and has access to all the process global variables, so you need to be careful with pointers and shared data.

In C11,

#include <threads.h>
thrd_start_t is the function pointer type int (*)(void *)
int thrd_create(thrd_t *thr, thrd_start_t func, void *arg);
thrd_t thrd_current(void);
int thrd_join(thrd_t thr, int *res);
_Noreturn void thrd_exit(int res);
etc.

APUE Fig. 11.2 (lightly modified)

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

#include <pthread.h>

void printids(const char *s)
{
pid_t pid = getpid();
pthread_t tid = pthread_self();

printf("%s: pid %u tid %u (0x%x)\n", s,
(unsigned int) pid,
(unsigned int) tid, (unsigned int) tid);
}

void *thr_fn(void *arg)
{
printids("new thread");
return NULL;
}

int main(void)
{
pthread_t ntid;
int err;

err = pthread_create(&ntid, NULL, thr_fn, NULL);
if (err != 0)

    { printf("can't create thread: %s\n",
        strerror(err)); exit(1); }

printids("main thread");

sleep(1);
exit(0);
}

Solaris

% cc -v fig.11.2.c

% a.out
main thread: pid 10511 tid 1 (0x1)
new thread: pid 10511 tid 2 (0x2)

Linux

% gcc -Wall -Wextra fig.11.2.c -lpthread
fig.11.2.c:18: warning: unused parameter 'arg'

% a.out
main thread: pid 13533 tid 3085883072 (0xb7eed6c0)
new thread: pid 13533 tid 3085880224 (0xb7eecba0)

Mac OS X

% gcc -Wall -Wextra fig.11.2.c
fig.11.2.c:18: warning: unused parameter ‘arg’

% ./a.out
main thread: pid 53471 tid 2688387104 (0xa03d8820)
new thread: pid 53471 tid 4027060224 (0xf0081000)

Model of a print server, based on a set of producers (users), a set of consumers (printers), and a control console.

This example originated as a programming project for CMPSC 473.

The main thread is responsible for reading the command line, determining the parameters of the model, starting the producer and consumer threads, and acting like an interactive command console.
The producer threads put items into a shared buffer, and the consumer threads remove them.

The print queue is a buffer shared between the producer and consumer threads. The main thread is allowed to initialize and inspect the print queue, but not to modify it after initialization. Producers insert a request into the buffer, and consumers remove requests from the buffer, in the order in which they were inserted. The buffer is finite - it can hold a limited number of requests.
The consumer thread is not told how many items to consume. You need to find a way for the main thread to communicate to each consumer thread that it should terminate. Do not simply use pthread_cancel() in the main thread. You will need to be careful about shutting down the threads, so that print requests in the queue are not discarded.
The producer thread determines the size of the print request, from the model parameters, and puts that information into the print queue. The producer may put additional information into the print queue, but may not remove anything from the queue.

Semaphores are used to coordinate the threads. See CS:APP Sec. 12.5 for much more info.

What happens to the threads in a process when fork() is called?

This is one of the design questions about processes and threads that could be discussed in CMPSC 473.
Here, we only ask, what does happen, and not, what should happen.

It would be nice to tell you that "the same thing happens on all versions of Unix", but that would not be true.

The C Standard doesn't say anything about this, because fork() is not part of C, it's part of Unix.

The Posix Standard for fork() says ...

The fork() function shall create a new process. The new process (child process) shall be an exact copy of the calling process (parent process) except as detailed below:
...
A process shall be created with a single thread. If a multi-threaded process calls fork(), the new process shall contain a replica of the calling thread and its entire address space, possibly including the states of mutexes and other resources. Consequently, to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the exec functions is called. Fork handlers may be established by means of the pthread_atfork() function in order to maintain application invariants across fork() calls.
...

The Posix Standard follows this with a Rationale section, which discusses proposals to change the behavior of fork(), and why those were rejected.

The Mac OS X man page for fork() says nothing about threads.

Solaris has two additional non-standard functions, fork1() and forkall(). The Solaris man page for fork() says ...

DESCRIPTION
Threads
     A call to forkall() replicates in the child process all of
     the threads (see thr_create(3C) and pthread_create(3C)) in
     the parent process. A call to fork1() replicates only the
     calling thread in the child process.

     In Solaris 10, a call to fork() is identical to a call to
     fork1(); only the calling thread is replicated in the child
     process. This is the POSIX-specified behavior for fork().

     In previous releases of Solaris, the behavior of fork()
     depended on whether or not the application was linked with
     the POSIX threads library. When linked with   -lthread
     (Solaris Threads) but not linked with -lpthread (POSIX
     Threads), fork() was the same as forkall().   When linked
     with -lpthread, whether or not also linked with -lthread,
     fork() was the same as fork1().

     In Solaris 10, neither -lthread nor -lpthread is required
     for multithreaded applications. The standard C library pro-
     vides all threading support for both sets of application
     programming    interfaces.     Applications   that   require
     replicate-all fork semantics must call forkall().

fork() Safety
     If a multithreaded application calls fork() or fork1(), and
     the child does more than simply call one of the exec(2)
     functions, there is a possibility of deadlock occurring in
     the child. The application should use pthread_atfork(3C) to
     ensure safety with respect to this deadlock. Should there be
     any outstanding mutexes throughout the process, the applica-
     tion should call pthread_atfork() to wait for and acquire
     those mutexes prior to calling fork() or fork1(). See "MT-
     Level of Libraries" on the attributes(5) manual page.

NOTES
     The thread in the child that calls fork() or fork1() must
     not depend on any resources held by threads that no longer
     exist in the child. In particular, locks held by these
     threads will not be released.

     In a multithreaded process, forkall() in one thread can
     cause blocking system calls to be interrupted and return
     with an EINTR error.

The Linux man page for fork() says ...

DESCRIPTION
   Note the following further points:

   * The child process is created with a single thread — the one that called
      fork(). The entire virtual address space of the parent is replicated in the
      child, including the states of mutexes, condition variables, and other
      pthreads objects; the use of pthread_atfork(3) may be helpful for dealing
      with problems that this can cause.

NOTES
   Since version 2.3.3, rather than invoking the kernel’s fork() system call, the
   glibc fork() wrapper that is provided as part of the NPTL threading implementa-
   tion invokes clone(2) with flags that provide the same effect as the traditional
   system call. The glibc wrapper invokes any fork handlers that have been estab-
   lished using pthread_atfork(3).

Here is what actually happens.

We wrote a program fork-example.c in which main() creates two threads, then forks, then tries to join with the two threads. The joins should succeed in the parent, since nothing about the parent's threads is affected by fork(). The joins should fail in the child, since the two threads don't exist in the child. Indeed, on Mac OS X and Solaris, that happens. On Linux, the child's joins succeed, but return a null exit status, which is probably the wrong thing to do. In each case, as claimed, the only active thread in the child is the original thread, running main().

On Solaris, we also tried forkall(), which works as advertised.

There is an older function vfork(), which should no longer be used; we tried it on Mac OS X to see what would happen. Don't use vfork().

Solaris, using fork()

[pid 23725] parent: &bogus = 217c8
[pid 23725] parent: &in[0] = 21814
[pid 23725] parent: &in[1] = 21824
[pid 23725] parent main pre-fork: pthread_create(0): trying
[pid 23725] parent main pre-fork: pthread_create(0): ok
[pid 23725] parent main pre-fork: pthread_create(1): trying
[pid 23725] parent: hello, world, this is thread 0, ID 2 2
[pid 23725] parent: thread 0, sleeping 1
[pid 23725] parent main pre-fork: pthread_create(1): ok
[pid 23725] parent: hello, world, this is thread 1, ID 3 3
[pid 23725] parent: main, child is 23726
[pid 23725] parent: in[0].tid = 2
[pid 23725] parent: in[1].tid = 3
[pid 23725] parent main: pthread_join(0): trying
[pid 23725] parent: thread 1, sleeping 2
[pid 23726] child: main, parent is 23725
[pid 23726] child: in[0].tid = 2
[pid 23726] child: in[1].tid = 3
[pid 23726] child main: pthread_join(0): trying
[pid 23726] child main: pthread_join(0): failed: No such process, exit status = 217c8, out = -1
[pid 23726] child main: pthread_join(1): trying
[pid 23726] child main: pthread_join(1): failed: No such process, exit status = 217c8, out = -1
[pid 23726] child: main, sleeping 1
[pid 23725] parent: thread 0, exiting
[pid 23725] parent main: pthread_join(0): ok, exit status = 21814, out = 1
[pid 23725] parent main: pthread_join(1): trying
[pid 23726] child: main, exiting
[pid 23725] parent: thread 1, exiting
[pid 23725] parent main: pthread_join(1): ok, exit status = 21824, out = 1
[pid 23725] parent: main, waiting
[pid 23725] parent: main, exiting

Solaris, using forkall()

[pid 23870] parent: &bogus = 217d0
[pid 23870] parent: &in[0] = 2181c
[pid 23870] parent: &in[1] = 2182c
[pid 23870] parent main pre-fork: pthread_create(0): trying
[pid 23870] parent main pre-fork: pthread_create(0): ok
[pid 23870] parent main pre-fork: pthread_create(1): trying
[pid 23870] parent: hello, world, this is thread 0, ID 2 2
[pid 23870] parent: thread 0, sleeping 1
[pid 23870] parent main pre-fork: pthread_create(1): ok
[pid 23870] parent: hello, world, this is thread 1, ID 3 3
[pid 23870] parent: main, child is 23871
[pid 23870] parent: in[0].tid = 2
[pid 23870] parent: in[1].tid = 3
[pid 23870] parent main: pthread_join(0): trying
[pid 23870] parent: thread 0, exiting
[pid 23870] parent: thread 1, sleeping 2
[pid 23870] parent main: pthread_join(0): ok, exit status = 2181c, out = 1
[pid 23870] parent main: pthread_join(1): trying
[pid 23871] child: thread 1, sleeping 2
[pid 23871] child: main, parent is 23870
[pid 23871] child: in[0].tid = 2
[pid 23871] child: in[1].tid = 3
[pid 23871] child main: pthread_join(0): trying
[pid 23871] child: thread 0, exiting
[pid 23871] child main: pthread_join(0): ok, exit status = 2181c, out = 1
[pid 23871] child main: pthread_join(1): trying
[pid 23870] parent: thread 1, exiting
[pid 23870] parent main: pthread_join(1): ok, exit status = 2182c, out = 1
[pid 23870] parent: main, waiting
[pid 23871] child: thread 1, exiting
[pid 23871] child main: pthread_join(1): ok, exit status = 2182c, out = 1
[pid 23871] child: main, sleeping 1
[pid 23871] child: main, exiting
[pid 23870] parent: main, exiting

Linux, using fork()

[pid 46128] parent: &bogus = 0x601850
[pid 46128] parent: &in[0] = 0x6018a0
[pid 46128] parent: &in[1] = 0x6018b8
[pid 46128] parent main pre-fork: pthread_create(0): trying
[pid 46128] parent main pre-fork: pthread_create(0): ok
[pid 46128] parent main pre-fork: pthread_create(1): trying
[pid 46128] parent main pre-fork: pthread_create(1): ok
[pid 46128] parent: hello, world, this is thread 0, ID 7f504ae6b700 7f504ae6b700
[pid 46128] parent: thread 0, sleeping 1
[pid 46128] parent: hello, world, this is thread 1, ID 7f504a46a700 7f504a46a700
[pid 46128] parent: thread 1, sleeping 2
[pid 46128] parent: main, child is 46131
[pid 46128] parent: in[0].tid = 7f504ae6b700
[pid 46128] parent: in[1].tid = 7f504a46a700
[pid 46128] parent main: pthread_join(0): trying
[pid 46131] child: main, parent is 46128
[pid 46131] child: in[0].tid = 7f504ae6b700
[pid 46131] child: in[1].tid = 7f504a46a700
[pid 46131] child main: pthread_join(0): trying
[pid 46131] child main: pthread_join(0): ok, exit status = (nil), out = -1
[pid 46131] child main: pthread_join(1): trying
[pid 46131] child main: pthread_join(1): ok, exit status = (nil), out = -1
[pid 46131] child: main, sleeping 1
[pid 46128] parent: thread 0, exiting
[pid 46131] child: main, exiting
[pid 46128] parent main: pthread_join(0): ok, exit status = 0x6018a0, out = 1
[pid 46128] parent main: pthread_join(1): trying
[pid 46128] parent: thread 1, exiting
[pid 46128] parent main: pthread_join(1): ok, exit status = 0x6018b8, out = 1
[pid 46128] parent: main, waiting
[pid 46128] parent: main, exiting

Mac OS X, using fork()

[pid 6202] parent: &bogus = 0x3018
[pid 6202] parent: &in[0] = 0x30a8
[pid 6202] parent: &in[1] = 0x30b8
[pid 6202] parent main pre-fork: pthread_create(0): trying
[pid 6202] parent main pre-fork: pthread_create(0): ok
[pid 6202] parent main pre-fork: pthread_create(1): trying
[pid 6202] parent main pre-fork: pthread_create(1): ok
[pid 6202] parent: hello, world, this is thread 0, ID f0081000 f0081000
[pid 6202] parent: thread 0, sleeping 1
[pid 6202] parent: hello, world, this is thread 1, ID f0103000 f0103000
[pid 6202] parent: thread 1, sleeping 2
[pid 6202] parent: main, child is 6203
[pid 6202] parent: in[0].tid = f0081000
[pid 6202] parent: in[1].tid = f0103000
[pid 6202] parent main: pthread_join(0): trying
[pid 6203] child: main, parent is 6202
[pid 6203] child: in[0].tid = f0081000
[pid 6203] child: in[1].tid = f0103000
[pid 6203] child main: pthread_join(0): trying
[pid 6203] child main: pthread_join(0): failed: No such process, exit status = 0x3018, out = -1
[pid 6203] child main: pthread_join(1): trying
[pid 6203] child main: pthread_join(1): failed: No such process, exit status = 0x3018, out = -1
[pid 6203] child: main, sleeping 1
[pid 6202] parent: thread 0, exiting
[pid 6202] parent main: pthread_join(0): ok, exit status = 0x30a8, out = 1
[pid 6202] parent main: pthread_join(1): trying
[pid 6203] child: main, exiting
[pid 6202] parent: thread 1, exiting
[pid 6202] parent main: pthread_join(1): ok, exit status = 0x30b8, out = 1
[pid 6202] parent: main, waiting
[pid 6202] parent: main, exiting

Mac OS X, using vfork() -- the lesson is, don't use vfork()

[pid 6952] parent: &bogus = 0x3018
[pid 6952] parent: &in[0] = 0x30a8
[pid 6952] parent: &in[1] = 0x30b8
[pid 6952] parent main pre-fork: pthread_create(0): trying
[pid 6952] parent main pre-fork: pthread_create(0): ok
[pid 6952] parent main pre-fork: pthread_create(1): trying
[pid 6952] parent main pre-fork: pthread_create(1): ok
[pid 6953] child: main, parent is 6952
[pid 6953] child: in[0].tid = f0081000
[pid 6953] child: in[1].tid = f0103000
[pid 6953] child main: pthread_join(0): trying
[pid 6952] child: hello, world, this is thread 0, ID f0081000 f0081000
[pid 6952] child: thread 0, sleeping 1
[pid 6952] child: hello, world, this is thread 1, ID f0103000 f0103000
[pid 6952] child: thread 1, sleeping 2
[pid 6952] child: thread 0, exiting
[pid 6953] child main: pthread_join(0): ok, exit status = 0x30a8, out = 1
[pid 6953] child main: pthread_join(1): trying
[pid 6952] child: thread 1, exiting
[pid 6953] child main: pthread_join(1): ok, exit status = 0x30b8, out = 1
[pid 6953] child: main, sleeping 1
[pid 6953] child: main, exiting
[pid 6952] child: main, child is 6953
[pid 6952] child: in[0].tid = f0081000
[pid 6952] child: in[1].tid = f0103000
[pid 6952] child main: pthread_join(0): trying
[pid 6952] child main: pthread_join(0): failed: No such process, exit status = 0x3018, out = 1
[pid 6952] child main: pthread_join(1): trying
[pid 6952] child main: pthread_join(1): failed: No such process, exit status = 0x3018, out = 1
[pid 6952] child: main, waiting
[pid 6952] child: main, exiting

Exercises.

Read about the function pthread_atfork().
Take CMPSC 473, and learn about mutexes, etc.

Last modified, 8 Apr. 2013