int membarrier(int cmd, int flags);
Use of memory barriers needs to be done taking into account that a memory barrier always needs to be either matched with its memory barrier counterparts, or that the architecture's memory model doesn't require the matching barriers.
There are cases where one side of the matching barriers (which we will refer to as "fast side") is executed much more often than the other (which we will refer to as "slow side"). This is a prime target for the use of membarrier(). The key idea is to replace, for these matching barriers, the fast-side memory barriers by simple compiler barriers, for example:
asm volatile ("" : : : "memory")
and replace the slow-side memory barriers by calls to membarrier().
This will add overhead to the slow side, and remove overhead from the fast side, thus resulting in an overall performance increase as long as the slow side is infrequent enough that the overhead of the membarrier() calls does not outweigh the performance gain on the fast side.
The cmd argument is one of the following:
The flags argument is currently unused and must be specified as 0.
All memory accesses performed in program order from each targeted thread are guaranteed to be ordered with respect to membarrier().
If we use the semantic barrier() to represent a compiler barrier forcing memory accesses to be performed in program order across the barrier, and smp_mb() to represent explicit memory barriers forcing full memory ordering across the barrier, we have the following ordering table for each pairing of barrier(), membarrier() and smp_mb(). The pair ordering is detailed as (O: ordered, X: not ordered):
barrier() smp_mb() membarrier()
barrier() X X O
smp_mb() X O O
membarrier() O O O
For a given command, with flags set to 0, this system call is guaranteed to always return the same value until reboot. Further calls with the same arguments will lead to the same result. Therefore, with flags set to 0, error handling is required only for the first call to membarrier().
Program order is the order in which instructions are ordered in the program assembly code.
Examples where membarrier() can be useful include implementations of Read-Copy-Update libraries and garbage collectors.
#include <stdlib.h> static volatile int a, b; static void fast_path(void) { int read_a, read_b; read_b = b; asm volatile ("mfence" : : : "memory"); read_a = a; /* read_b == 1 implies read_a == 1. */ if (read_b == 1 && read_a == 0) abort(); } static void slow_path(void) { a = 1; asm volatile ("mfence" : : : "memory"); b = 1; } int main(int argc, char **argv) { /* * Real applications would call fast_path() and slow_path() * from different threads. Call those from main() to keep * this example short. */ slow_path(); fast_path(); exit(EXIT_SUCCESS); }
The code above transformed to use membarrier() becomes:
#define _GNU_SOURCE #include <stdlib.h> #include <stdio.h> #include <unistd.h> #include <sys/syscall.h> #include <linux/membarrier.h> static volatile int a, b; static int membarrier(int cmd, int flags) { return syscall(__NR_membarrier, cmd, flags); } static int init_membarrier(void) { int ret; /* Check that membarrier() is supported. */ ret = membarrier(MEMBARRIER_CMD_QUERY, 0); if (ret < 0) { perror("membarrier"); return -1; } if (!(ret & MEMBARRIER_CMD_SHARED)) { fprintf(stderr, "membarrier does not support MEMBARRIER_CMD_SHARED\n"); return -1; } return 0; } static void fast_path(void) { int read_a, read_b; read_b = b; asm volatile ("" : : : "memory"); read_a = a; /* read_b == 1 implies read_a == 1. */ if (read_b == 1 && read_a == 0) abort(); } static void slow_path(void) { a = 1; membarrier(MEMBARRIER_CMD_SHARED, 0); b = 1; } int main(int argc, char **argv) { if (init_membarrier()) exit(EXIT_FAILURE); /* * Real applications would call fast_path() and slow_path() * from different threads. Call those from main() to keep * this example short. */ slow_path(); fast_path(); exit(EXIT_SUCCESS); }