CMPSC 311, Introduction to Systems Programming

Intro to Posix Threads



Reading



Processes vs. Threads

Processes vs.
      Threads

Processes
Threads
fork
create
exec
wait
join
exit
exit


Processes Threads
unistd.h, stdlib.h pthread.h
pid_t pthread_t
pid_t getpid()
pthread_t pthread_self(void)
pid_t getppid() ---
pid_t fork(void)
int exec..(...)
int pthread_create(...)
pid_t wait(...)
pid_t waitpid(...)
int pthread_join(...)
void exit(int status) void pthread_exit(...)
        
Include files
Create and start a new thread
Terminate a thread
Wait for a thread to complete
Wait for all threads to complete, then terminate the process, in main()
Compile, Solaris
Miscellaneous
In C11,



APUE Fig. 11.2 (lightly modified)

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

#include <pthread.h>

void printids(const char *s)
{
  pid_t pid = getpid();
  pthread_t tid = pthread_self();

  printf("%s: pid %u tid %u (0x%x)\n", s,
    (unsigned int) pid,
    (unsigned int) tid, (unsigned int) tid);
}

void *thr_fn(void *arg)
{
  printids("new thread");
  return NULL;
}

int main(void)
{
  pthread_t ntid;
  int err;

  err = pthread_create(&ntid, NULL, thr_fn, NULL);
  if (err != 0)
    { printf("can't create thread: %s\n", strerror(err)); exit(1); }

  printids("main thread");

  sleep(1);
  exit(0);
}

Solaris

% cc -v fig.11.2.c

% a.out
main thread: pid 10511 tid 1 (0x1)
new thread: pid 10511 tid 2 (0x2)

Linux

% gcc -Wall -Wextra fig.11.2.c -lpthread
fig.11.2.c:18: warning: unused parameter 'arg'

% a.out
main thread: pid 13533 tid 3085883072 (0xb7eed6c0)
new thread: pid 13533 tid 3085880224 (0xb7eecba0)

Mac OS X

% gcc -Wall -Wextra fig.11.2.c
fig.11.2.c:18: warning: unused parameter ‘arg’

% ./a.out
main thread: pid 53471 tid 2688387104 (0xa03d8820)
new thread: pid 53471 tid 4027060224 (0xf0081000)



Model of a print server, based on a set of producers (users), a set of consumers (printers), and a control console.
Semaphores are used to coordinate the threads.  See CS:APP Sec. 12.5 for much more info.



What happens to the threads in a process when fork() is called?
It would be nice to tell you that "the same thing happens on all versions of Unix", but that would not be true.

The C Standard doesn't say anything about this, because fork() is not part of C, it's part of Unix.

The Posix Standard for fork() says ...

The fork() function shall create a new process.  The new process (child process) shall be an exact copy of the calling process (parent process) except as detailed below:
...
A process shall be created with a single thread.  If a multi-threaded process calls fork(), the new process shall contain a replica of the calling thread and its entire address space, possibly including the states of mutexes and other resources.  Consequently, to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the exec functions is called.  Fork handlers may be established by means of the pthread_atfork() function in order to maintain application invariants across fork() calls.
...

The Posix Standard follows this with a Rationale section, which discusses proposals to change the behavior of fork(), and why those were rejected.

The Mac OS X man page for fork() says nothing about threads.

Solaris has two additional non-standard functions, fork1() and forkall().  The Solaris man page for fork() says ...

DESCRIPTION
  Threads
     A call to forkall() replicates in the child process  all  of
     the  threads  (see thr_create(3C) and pthread_create(3C)) in
     the parent process. A call to fork1()  replicates  only  the
     calling thread in the child process.

     In Solaris 10, a call to fork() is identical to  a  call  to
     fork1();  only the calling thread is replicated in the child
     process. This is the POSIX-specified behavior for fork().

     In previous releases of  Solaris,  the  behavior  of  fork()
     depended  on  whether or not the application was linked with
     the  POSIX  threads  library.  When  linked  with   -lthread
     (Solaris  Threads)  but  not  linked  with  -lpthread (POSIX
     Threads), fork() was the same  as  forkall().   When  linked
     with  -lpthread,  whether  or not also linked with -lthread,
     fork() was the same as fork1().

     In Solaris 10, neither -lthread nor  -lpthread  is  required
     for  multithreaded applications. The standard C library pro-
     vides all threading support for  both  sets  of  application
     programming    interfaces.     Applications   that   require
     replicate-all fork semantics must call forkall().

  fork() Safety
     If a multithreaded application calls fork() or fork1(),  and
     the  child  does  more  than  simply call one of the exec(2)
     functions, there is a possibility of deadlock  occurring  in
     the  child. The application should use pthread_atfork(3C) to
     ensure safety with respect to this deadlock. Should there be
     any outstanding mutexes throughout the process, the applica-
     tion should call pthread_atfork() to wait  for  and  acquire
     those  mutexes prior to calling fork() or fork1(). See  "MT-
     Level of Libraries" on the attributes(5) manual page.

NOTES
     The thread in the child that calls fork()  or  fork1()  must
     not  depend  on any resources held by threads that no longer
     exist in the child.  In  particular,  locks  held  by  these
     threads will not be released.

     In a multithreaded process,  forkall()  in  one  thread  can
     cause  blocking  system  calls  to be interrupted and return
     with an EINTR error.

The Linux man page for fork() says ...

DESCRIPTION
   Note the following further points:

   *  The  child  process  is  created  with  a single thread — the one that called
      fork().  The entire virtual address space of the parent is replicated in  the
      child,  including  the  states  of  mutexes,  condition  variables, and other
      pthreads objects; the use of pthread_atfork(3) may  be  helpful  for  dealing
      with problems that this can cause.

NOTES
   Since  version  2.3.3, rather than invoking the kernel’s fork() system call, the
   glibc fork() wrapper that is provided as part of the NPTL threading  implementa-
   tion invokes clone(2) with flags that provide the same effect as the traditional
   system call.  The glibc wrapper invokes any fork handlers that have been  estab-
   lished using pthread_atfork(3).

Here is what actually happens.

We wrote a program fork-example.c in which main() creates two threads, then forks, then tries to join with the two threads.  The joins should succeed in the parent, since nothing about the parent's threads is affected by fork().  The joins should fail in the child, since the two threads don't exist in the child.  Indeed, on Mac OS X and Solaris, that happens.  On Linux, the child's joins succeed, but return a null exit status, which is probably the wrong thing to do.  In each case, as claimed, the only active thread in the child is the original thread, running main().

On Solaris, we also tried forkall(), which works as advertised.

There is an older function vfork(), which should no longer be used; we tried it on Mac OS X to see what would happen.  Don't use vfork().

Solaris, using fork()

[pid 23725] parent: &bogus = 217c8
[pid 23725] parent: &in[0] = 21814
[pid 23725] parent: &in[1] = 21824
[pid 23725] parent main pre-fork: pthread_create(0): trying
[pid 23725] parent main pre-fork: pthread_create(0): ok
[pid 23725] parent main pre-fork: pthread_create(1): trying
[pid 23725] parent: hello, world, this is thread 0, ID 2 2
[pid 23725] parent: thread 0, sleeping 1
[pid 23725] parent main pre-fork: pthread_create(1): ok
[pid 23725] parent: hello, world, this is thread 1, ID 3 3
[pid 23725] parent: main, child is 23726
[pid 23725] parent: in[0].tid = 2
[pid 23725] parent: in[1].tid = 3
[pid 23725] parent main: pthread_join(0): trying
[pid 23725] parent: thread 1, sleeping 2
[pid 23726] child: main, parent is 23725
[pid 23726] child: in[0].tid = 2
[pid 23726] child: in[1].tid = 3
[pid 23726] child main: pthread_join(0): trying
[pid 23726] child main: pthread_join(0): failed: No such process, exit status = 217c8, out = -1
[pid 23726] child main: pthread_join(1): trying
[pid 23726] child main: pthread_join(1): failed: No such process, exit status = 217c8, out = -1
[pid 23726] child: main, sleeping 1
[pid 23725] parent: thread 0, exiting
[pid 23725] parent main: pthread_join(0): ok, exit status = 21814, out = 1
[pid 23725] parent main: pthread_join(1): trying
[pid 23726] child: main, exiting
[pid 23725] parent: thread 1, exiting
[pid 23725] parent main: pthread_join(1): ok, exit status = 21824, out = 1
[pid 23725] parent: main, waiting
[pid 23725] parent: main, exiting

Solaris, using forkall()

[pid 23870] parent: &bogus = 217d0
[pid 23870] parent: &in[0] = 2181c
[pid 23870] parent: &in[1] = 2182c
[pid 23870] parent main pre-fork: pthread_create(0): trying
[pid 23870] parent main pre-fork: pthread_create(0): ok
[pid 23870] parent main pre-fork: pthread_create(1): trying
[pid 23870] parent: hello, world, this is thread 0, ID 2 2
[pid 23870] parent: thread 0, sleeping 1
[pid 23870] parent main pre-fork: pthread_create(1): ok
[pid 23870] parent: hello, world, this is thread 1, ID 3 3
[pid 23870] parent: main, child is 23871
[pid 23870] parent: in[0].tid = 2
[pid 23870] parent: in[1].tid = 3
[pid 23870] parent main: pthread_join(0): trying
[pid 23870] parent: thread 0, exiting
[pid 23870] parent: thread 1, sleeping 2
[pid 23870] parent main: pthread_join(0): ok, exit status = 2181c, out = 1
[pid 23870] parent main: pthread_join(1): trying
[pid 23871] child: thread 1, sleeping 2
[pid 23871] child: main, parent is 23870
[pid 23871] child: in[0].tid = 2
[pid 23871] child: in[1].tid = 3
[pid 23871] child main: pthread_join(0): trying
[pid 23871] child: thread 0, exiting
[pid 23871] child main: pthread_join(0): ok, exit status = 2181c, out = 1
[pid 23871] child main: pthread_join(1): trying
[pid 23870] parent: thread 1, exiting
[pid 23870] parent main: pthread_join(1): ok, exit status = 2182c, out = 1
[pid 23870] parent: main, waiting
[pid 23871] child: thread 1, exiting
[pid 23871] child main: pthread_join(1): ok, exit status = 2182c, out = 1
[pid 23871] child: main, sleeping 1
[pid 23871] child: main, exiting
[pid 23870] parent: main, exiting

Linux, using fork()

[pid 46128] parent: &bogus = 0x601850
[pid 46128] parent: &in[0] = 0x6018a0
[pid 46128] parent: &in[1] = 0x6018b8
[pid 46128] parent main pre-fork: pthread_create(0): trying
[pid 46128] parent main pre-fork: pthread_create(0): ok
[pid 46128] parent main pre-fork: pthread_create(1): trying
[pid 46128] parent main pre-fork: pthread_create(1): ok
[pid 46128] parent: hello, world, this is thread 0, ID 7f504ae6b700 7f504ae6b700
[pid 46128] parent: thread 0, sleeping 1
[pid 46128] parent: hello, world, this is thread 1, ID 7f504a46a700 7f504a46a700
[pid 46128] parent: thread 1, sleeping 2
[pid 46128] parent: main, child is 46131
[pid 46128] parent: in[0].tid = 7f504ae6b700
[pid 46128] parent: in[1].tid = 7f504a46a700
[pid 46128] parent main: pthread_join(0): trying
[pid 46131] child: main, parent is 46128
[pid 46131] child: in[0].tid = 7f504ae6b700
[pid 46131] child: in[1].tid = 7f504a46a700
[pid 46131] child main: pthread_join(0): trying
[pid 46131] child main: pthread_join(0): ok, exit status = (nil), out = -1
[pid 46131] child main: pthread_join(1): trying
[pid 46131] child main: pthread_join(1): ok, exit status = (nil), out = -1
[pid 46131] child: main, sleeping 1
[pid 46128] parent: thread 0, exiting
[pid 46131] child: main, exiting
[pid 46128] parent main: pthread_join(0): ok, exit status = 0x6018a0, out = 1
[pid 46128] parent main: pthread_join(1): trying
[pid 46128] parent: thread 1, exiting
[pid 46128] parent main: pthread_join(1): ok, exit status = 0x6018b8, out = 1
[pid 46128] parent: main, waiting
[pid 46128] parent: main, exiting

Mac OS X, using fork()

[pid 6202] parent: &bogus = 0x3018
[pid 6202] parent: &in[0] = 0x30a8
[pid 6202] parent: &in[1] = 0x30b8
[pid 6202] parent main pre-fork: pthread_create(0): trying
[pid 6202] parent main pre-fork: pthread_create(0): ok
[pid 6202] parent main pre-fork: pthread_create(1): trying
[pid 6202] parent main pre-fork: pthread_create(1): ok
[pid 6202] parent: hello, world, this is thread 0, ID f0081000 f0081000
[pid 6202] parent: thread 0, sleeping 1
[pid 6202] parent: hello, world, this is thread 1, ID f0103000 f0103000
[pid 6202] parent: thread 1, sleeping 2
[pid 6202] parent: main, child is 6203
[pid 6202] parent: in[0].tid = f0081000
[pid 6202] parent: in[1].tid = f0103000
[pid 6202] parent main: pthread_join(0): trying
[pid 6203] child: main, parent is 6202
[pid 6203] child: in[0].tid = f0081000
[pid 6203] child: in[1].tid = f0103000
[pid 6203] child main: pthread_join(0): trying
[pid 6203] child main: pthread_join(0): failed: No such process, exit status = 0x3018, out = -1
[pid 6203] child main: pthread_join(1): trying
[pid 6203] child main: pthread_join(1): failed: No such process, exit status = 0x3018, out = -1
[pid 6203] child: main, sleeping 1
[pid 6202] parent: thread 0, exiting
[pid 6202] parent main: pthread_join(0): ok, exit status = 0x30a8, out = 1
[pid 6202] parent main: pthread_join(1): trying
[pid 6203] child: main, exiting
[pid 6202] parent: thread 1, exiting
[pid 6202] parent main: pthread_join(1): ok, exit status = 0x30b8, out = 1
[pid 6202] parent: main, waiting
[pid 6202] parent: main, exiting

Mac OS X, using vfork() -- the lesson is, don't use vfork()

[pid 6952] parent: &bogus = 0x3018
[pid 6952] parent: &in[0] = 0x30a8
[pid 6952] parent: &in[1] = 0x30b8
[pid 6952] parent main pre-fork: pthread_create(0): trying
[pid 6952] parent main pre-fork: pthread_create(0): ok
[pid 6952] parent main pre-fork: pthread_create(1): trying
[pid 6952] parent main pre-fork: pthread_create(1): ok
[pid 6953] child: main, parent is 6952
[pid 6953] child: in[0].tid = f0081000
[pid 6953] child: in[1].tid = f0103000
[pid 6953] child main: pthread_join(0): trying
[pid 6952] child: hello, world, this is thread 0, ID f0081000 f0081000
[pid 6952] child: thread 0, sleeping 1
[pid 6952] child: hello, world, this is thread 1, ID f0103000 f0103000
[pid 6952] child: thread 1, sleeping 2
[pid 6952] child: thread 0, exiting
[pid 6953] child main: pthread_join(0): ok, exit status = 0x30a8, out = 1
[pid 6953] child main: pthread_join(1): trying
[pid 6952] child: thread 1, exiting
[pid 6953] child main: pthread_join(1): ok, exit status = 0x30b8, out = 1
[pid 6953] child: main, sleeping 1
[pid 6953] child: main, exiting
[pid 6952] child: main, child is 6953
[pid 6952] child: in[0].tid = f0081000
[pid 6952] child: in[1].tid = f0103000
[pid 6952] child main: pthread_join(0): trying
[pid 6952] child main: pthread_join(0): failed: No such process, exit status = 0x3018, out = 1
[pid 6952] child main: pthread_join(1): trying
[pid 6952] child main: pthread_join(1): failed: No such process, exit status = 0x3018, out = 1
[pid 6952] child: main, waiting
[pid 6952] child: main, exiting

Exercises.



Last modified, 8 Apr. 2013