Debugging Pthreads in C with GDB: A Hands-On Guide

Debugging Pthreads in C with GDB: A Hands-On Guide

Multithreaded programming brings immense power to software, but it also introduces complex bugs that are often tricky to diagnose. How do you debug issues when threads are running in parallel? How do you pause all threads, examine variables in individual threads, or view the backtrace for a specific thread? GDB (GNU Debugger) is here to make this manageable.

Let’s dive into debugging pthreads using a simple yet illustrative example.

A Multithreaded Program

Here’s a C program to demonstrate GDB’s capabilities with pthreads:

#include <stdio.h>
#include <pthread.h>
#include <unistd.h>

void *worker_function(void *arg) {
    int id = *(int *)arg;
    for (int i = 0; i < 5; i++) {
        printf("Thread %d: iteration %d\n", id, i);
        sleep(1);
    }
    return NULL;
}

int main() {
    pthread_t threads[3];
    int thread_ids[3] = {1, 2, 3};

    for (int i = 0; i < 3; i++) {
        pthread_create(&threads[i], NULL, worker_function, &thread_ids[i]);
    }

    for (int i = 0; i < 3; i++) {
        pthread_join(threads[i], NULL);
    }

    return 0;
}

Save the file as threads_demo.c and compile it with debugging symbols:

gcc -g -pthread threads_demo.c -o threads_demo

Launching GDB

Run GDB with the compiled program:

gdb ./threads_demo

Key GDB Features for Multithreaded Programs

1. Viewing Threads

When debugging multithreaded programs, the first step is to view all threads. Start the program inside GDB:

run

Once the program starts, use:

info threads

This lists all threads, along with their IDs and current states. For example:

  Id   Target Id         Frame
* 1    Thread 0x7ffff7fce740 (LWP 12345) "threads_demo" main () at threads_demo.c:17
  2    Thread 0x7ffff7fce8c0 (LWP 12346) "threads_demo" worker_function (arg=0x7fffffffdf4c) at threads_demo.c:7
  3    Thread 0x7ffff7fcebc0 (LWP 12347) "threads_demo" worker_function (arg=0x7fffffffdf50) at threads_demo.c:7

The * indicates the currently active thread.

2. Switching Between Threads

To focus on a specific thread, use:

thread <thread-id>

For example, to switch to thread 2:

thread 2

Once switched, you can inspect variables, set breakpoints, or view the backtrace for that thread.

3. Pausing All Threads

When the program hits a breakpoint, all threads are paused. This allows you to inspect the state of the entire program at a specific point in time.

4. Backtracing in Threads

To view the call stack of the current thread:

backtrace

For instance, if thread 2 is running worker_function, the backtrace might show:

#0  worker_function (arg=0x7fffffffdf4c) at threads_demo.c:7
#1  0x00007ffff7a5b609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#2  0x00007ffff7b23e43 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

This is invaluable for understanding how a thread reached its current state.

To backtrace all threads simultaneously, use:

thread apply all bt

This command shows the call stack for each thread, making it easier to spot problematic areas.

5. Setting Breakpoints in Threads

Breakpoints can be set globally or for specific threads. For a global breakpoint:

break worker_function

This triggers the breakpoint for any thread entering worker_function.

To set a breakpoint for a specific thread:

break worker_function thread 2

Now, only thread 2 will stop when it reaches worker_function.

6. Inspecting Variables in Threads

To examine variables in the context of a specific thread, switch to that thread and use the print command. For example:

thread 2
print id

This shows the value of id for thread 2.

7. Conditional Breakpoints

Suppose you want to break in worker_function only for thread 3 when i == 2. Use:

break worker_function if id == 3 && i == 2

8. Scheduler Locking Step

When debugging multithreaded applications, understanding scheduler behavior is crucial. Use:

set scheduler-locking on

This ensures that only the thread you’re debugging runs, while others remain paused. It’s invaluable for isolating issues in a specific thread.

9. Controlling Individual Threads

To continue execution of a specific thread while pausing others, use:

thread <thread-id>
continue

10. Debugging Deadlocks

If your program hangs, it might be due to a deadlock. Use:

info threads

Switch to each thread and view their backtraces. Look for threads stuck in synchronization functions like pthread_mutex_lock.

Debugging the Example Program

1. Track Thread Execution:
   • Set a global breakpoint in worker_function.
   • Use info threads to identify which thread hit the breakpoint.
   • Switch to that thread and inspect variables.
2. Diagnose Synchronization Issues:
   • Add a watchpoint on a shared variable if threads are modifying it.
   • Use info threads and thread apply all bt to identify where threads are waiting.
3. Understand Thread Lifecycle:
   • Use info threads at various points to see when threads are created and destroyed.

Wrapping Up

Debugging multithreaded programs may seem daunting, but GDB makes it manageable. With tools to inspect, control, and analyze threads, you can debug even the most complex pthread applications. Features like thread apply all bt and scheduler-locking further enhance your ability to pinpoint and resolve issues efficiently. Take the time to explore these features and build confidence in handling multithreaded bugs. Happy debugging!