Keep track of your own memory with valgrind memcheck

Valgrind memcheck can detect various memory leaks and it keeps track of whether memory is accessible and defined. But what if you have build your own memory manager? Memcheck keeps track of memory by observing malloc/free, new/delete and mmap/munmap. But it doesn't know how a program subdivides that memory internally without a little help.

Replacing malloc

If you simply replace the whole system glibc malloc implementation by defining your own functions with the same names (for example by using tcmalloc or jemalloc) then valgrind will, since version 3.12.0, intercept all replacement malloc, free, etc. functions, unless you tell it not to (see the --soname-synonyms option in the valgrind manual). That way valgrind memcheck will be able to provide all normal memory issues as if you used the system malloc implementation.

RUNNING_ON_VALGRIND

If you wrote your own allocator for more specialized use then valgrind has a way for you to annotate your code so tools like memcheck can keep track of the memory blocks you hand out to the rest of your program. Make sure you have the valgrind-devel package installed. This will provide /usr/include/valgrind/valgrind.h which defines some basic macros which annotate your code using instructions that look like they don't do anything. So they provide near-zero overhead in normal use. But when run under valgrind it will be recognized as a "magic sequence" that instructs valgrind to do something special at that place in the code.

The simplest macro is RUNNING_ON_VALGRIND which is 0 if running natively and 1 when running under valgrind. If you have your own allocation and deallocation functions you could use the macro as follows:

#include <valgrind.h>
#include <stdlib.h>

/* Some global structures for the real allocator.  */

void *
my_alloc (size_t size)
{
  if (RUNNING_ON_VALGRIND)
    return malloc (size);

  void *p = NULL;
  /* The real allocator ... */
  return p;
}

void
my_free (void *ptr)
{
  if (RUNNING_ON_VALGRIND)
    free (ptr);

  /* The real deallocator ... */
}
Which you then compile with gcc -I/usr/include/valgrind -O2 -g -c my_alloc.c

Now, when running under valgrind (and only when running under valgrind) your allocation functions will simply use malloc and free and valgrind memcheck can track all memory usage as normal.

This might look like cheating. You put a lot of work in your own allocator which you believe to be way more efficient for your application than the system glibc allocator. And now when running under valgrind all that work is thrown away by simply calling the malloc and free you were trying to avoid. But if you are running under valgrind you don't do it for efficiency, but to catch memory issues. And now valgrind memcheck will be able to report memory leaks, use-after-free, undefined memory use, buffer overruns, etc. for all blocks allocated through my_alloc and my_free, without you having to add any extra instrumentation to your own allocator.

malloc and free like blocks of memory

But if you want to track memory as your own allocator hands out blocks then there is a different way to make valgrind aware of that. Use the VALGRIND_MALLOCLIKE_BLOCK and VALGRIND_FREELIKE_BLOCK macros. Like all other valgrind.h macros these do nothing when not running under valgrind. But when run under valgrind it lets memcheck track memory usage from your custom allocator.

VALGRIND_MALLOCLIKE_BLOCK takes four arguments, the start address of the block, the size of the block in bytes, the size of the redzone in bytes and whether the block contents has been initialized (with zero or a known pattern) or not. The redzone (padding around the block) may be zero. But if your allocator can reserve bytes before/after the block (memcheck by default uses 16 bytes when overriding malloc), then it will help with detecting overrun and underrun reads and writes.

VALGRIND_FREELIKE_BLOCK just takes two arguments, the start address of the block and any red zone bytes added as padding.

#include <valgrind.h>

/* Some global structures for the real allocator.  */

void *
my_alloc (size_t size)
{
  void *p = NULL;
  /* The allocator ... */

  VALGRIND_MALLOCLIKE_BLOCK (p, size, 0, 0);
  return p;
}

void
my_free (void *ptr)
{
  VALGRIND_FREELIKE_BLOCK (p, 0);
  /* The deallocator ... */
}
Note that we use VALGRIND_MALLOCLIKE_BLOCK as late as possible, and VALGRIND_FREELIKE_BLOCK as early as possible. This gives the best warnings from memcheck when an issue is detected. If those macros are embedded deeper in the allocator code then stack traces will include parts of your allocator internals, which will be confusing to users.

There is also VALGRIND_RESIZEINPLACE_BLOCK that can be used when you want to provide functionality similar to realloc.

Beyond blocks, accessible and defined memory

The valgrind BLOCK macros are all you need for a simple allocators where the memory blocks themselves are not used (read or written) by the allocator itself. But in most cases you will need to use some more annotations to tell memcheck when the allocator itself is manipulating addresses inside these blocks. These more complex macros are defined in /usr/include/valgrind/memcheck.h.

The BLOCK macros treat the memory areas as either accessible or inaccissble, and mark them as needing tracking for memory leaks. And that goes for the whole application, including the allocator parts, which might need special access. For example if your allocator wants to clear the contents of a memory block before returning it, or if you embded some allocator meta-information in the memory block redzone. Then you have to tell valgrind you are allowed to write to that memory block even if it isn't fully defined yet (or has been freed before and is now being reused). You would use VALGRIND_MAKE_MEM_UNDEFINED, VALGRIND_MAKE_MEM_UNDEFINED and VALGRIND_MAKE_MEM_NOACCESS for that.

All three MAKE_MEM macros take two arguments, a starting address and a size in bytes. The VALGRIND_MAKE_MEM_UNDEFINED and VALGRIND_MAKE_MEM_UNDEFINED macros mark the address range as accessible, and the contents as defined or undefined. VALGRIND_MAKE_MEM_NOACCESS marks the address range as inaccessible. But unlike the BLOCK macros they don't imply tracking the areas for memory leaks. This makes it possible to use the memory blocks inside your allocator, while the rest of your application won't (do remember to flip them back to inaccessible before leaving the allocator code).

These macros are also useful when reusing datastructures, or if a datastructure is allocated using calloc or cleared with memset but you do want to mark parts or the whole structure as having undefined values that need to be tracked by memcheck.

#include <string.h>
#include <memcheck.h>

/* The raw message as mapped in by frob ()  */
struct rawmsg
{
  int frob_count;
  long frob_flags;
  unsigned char frobs[128];
  /* lots of other fields...  */
};

struct message
{
  int seq_nr;
  struct rawmsg msg;
};

/* Cleans up the given message and returns a pointer to the next one.
   Don't forget to frob() the message before accessing the fields. */
struct message*
next_message (struct message *m)
{
  int next_seq_nr = m->seq_nr + 1;

  /* clean up the old message... */
  memset (m, 0, sizeof (struct message));
  m->seq_nr = next_seq_nr;

  /* Except for the sequence number, everything else is undefined till
     frob () is used on the message.  */
  VALGRIND_MAKE_MEM_UNDEFINED (&m->msg, sizeof (struct rawmsg));

  return m;
}
In this example a message data structure is passed around that gets "refreshed" from time to time. The next_message function clears various fields and the code assumes those will get "real" values later. Normally memcheck assumes all fields have defined values, since it sees the memset putting values into the message structure. But the VALGRIND_MAKE_MEM_UNDEFINED macros makes sure that valgrind knows that the values in those fields aren't really defined (the marco does nothing when not running under valgrind). Now memcheck will warn when any of the field values is used when not properly set first.

More information

To learn more about the valgrind client requests and the memcheck specific macros, see the valgrind manual chapters on The Client Request mechanism and Memcheck Client Requests.