Boost C++ Libraries Home Libraries People FAQ More

PrevUpHomeNext

Managed Memory Segments

Making Interprocess Data Communication Easy
Managed Shared Memory
Managed Mapped File

As we have seen, Boost.Interprocess offers some basic classes to create shared memory objects and file mappings and map those mappable classes to the process' address space.

However, managing those memory segments is not not easy for non-trivial tasks. A mapped region is a fixed-length memory buffer and creating and destroying objects of any type dynamically, requires a lot of work, since it would require programming a memory management algorithm to allocate portions of that segment. Many times, we also want to associate names to objects created in shared memory, so all the processes can find the object using the name.

Boost.Interprocess offers 4 managed memory segment classes:

  • To manage a shared memory mapped region (basic_managed_shared_memory class).
  • To manage a memory mapped file (basic_managed_mapped_file).
  • To manage a heap allocated (operator new) memory buffer (basic_managed_heap_memory class).
  • To manage a user provided fixed size buffer (basic_managed_external_buffer class).

The first two classes manage memory segments that can be shared between processes. The third is useful to create complex data-bases to be sent though other mechanisms like message queues to other processes. The fourth class can manage any fixed size memory buffer. The first two classes will be explained in the next two sections. basic_managed_heap_memory and basic_managed_external_buffer will be explained later.

The most important services of a managed memory segment are:

  • Dynamic allocation of portions of a memory the segment.
  • Construction of C++ objects in the memory segment. These objects can be anonymous or we can associate a name to them.
  • Searching capabilities for named objects.
  • Customization of many features: memory allocation algorithm, index types or character types.
  • Atomic constructions and destructions so that if the segment is shared between two processes it's impossible to create two objects associated with the same name, simplifying synchronization.

All Boost.Interprocess managed memory segment classes are templatized classes that can be customized by the user:

template
      <
         class CharType,
         class MemoryAlgorithm,
         template<class IndexConfig> class IndexType
      >
class basic_managed_shared_memory / basic_managed_mapped_file /
      basic_managed_heap_memory   / basic_external_buffer;

These classes can be customized with the following template parameters:

  • CharType is the type of the character that will be used to identify the created named objects (for example, char or wchar_t)
  • MemoryAlgorithm is the memory algorithm used to allocate portions of the segment (for example, rbtree_best_fit ). The internal typedefs of the memory algorithm also define:
    • The synchronization type (MemoryAlgorithm::mutex_family) to be used in all allocation operations. This allows the use of user-defined mutexes or avoiding internal locking (maybe code will be externally synchronized by the user).
    • The Pointer type (MemoryAlgorithm::void_pointer) to be used by the memory allocation algorithm or additional helper structures (like a map to maintain object/name associations). All STL compatible allocators and containers to be used with this managed memory segment will use this pointer type. The pointer type will define if the managed memory segment can be mapped between several processes. For example, if void_pointer is offset_ptr<void> we will be able to map the managed segment in different base addresses in each process. If void_pointer is void* only fixed address mapping could be used.
    • See Writing a new memory allocation algorithm for more details about memory algorithms.
  • IndexType is the type of index that will be used to store the name-object association (for example, a map, a hash-map, or an ordered vector).

This way, we can use char or wchar_t strings to identify created C++ objects in the memory segment, we can plug new shared memory allocation algorithms, and use the index type that is best suited to our needs.

As seen, basic_managed_shared_memory offers a great variety of customization. But for the average user, a common, default shared memory named object creation is needed. Because of this, Boost.Interprocess defines the most common managed shared memory specializations:

//!Defines a managed shared memory with c-strings as keys for named objects,
//!the default memory algorithm (with process-shared mutexes,
//!and offset_ptr as internal pointers) as memory allocation algorithm
//!and the default index type as the index.
//!This class allows the shared memory to be mapped in different base
//!in different processes
typedef
   basic_managed_shared_memory<char
                              ,/*Default memory algorithm defining offset_ptr<void> as void_pointer*/
                              ,/*Default index type*/>
   managed_shared_memory;

//!Defines a managed shared memory with wide strings as keys for named objects,
//!the default memory algorithm (with process-shared mutexes,
//!and offset_ptr as internal pointers) as memory allocation algorithm
//!and the default index type as the index.
//!This class allows the shared memory to be mapped in different base
//!in different processes
typedef
   basic_managed_shared_memory<wchar_t
                              ,/*Default memory algorithm defining offset_ptr<void> as void_pointer*/
                              ,/*Default index type*/>
   wmanaged_shared_memory;

managed_shared_memory allocates objects in shared memory associated with a c-string and wmanaged_shared_memory allocates objects in shared memory associated with a wchar_t null terminated string. Both define the pointer type as offset_ptr<void> so they can be used to map the shared memory at different base addresses in different processes.

If the user wants to map the shared memory in the same address in all processes and want to use raw pointers internally instead of offset pointers, Boost.Interprocess defines the following types:

//!Defines a managed shared memory with c-strings as keys for named objects,
//!the default memory algorithm (with process-shared mutexes,
//!and offset_ptr as internal pointers) as memory allocation algorithm
//!and the default index type as the index.
//!This class allows the shared memory to be mapped in different base
//!in different processes*/
typedef basic_managed_shared_memory
   <char
   ,/*Default memory algorithm defining void * as void_pointer*/
   ,/*Default index type*/>
fixed_managed_shared_memory;

//!Defines a managed shared memory with wide strings as keys for named objects,
//!the default memory algorithm (with process-shared mutexes,
//!and offset_ptr as internal pointers) as memory allocation algorithm
//!and the default index type as the index.
//!This class allows the shared memory to be mapped in different base
//!in different processes
typedef basic_managed_shared_memory
   <wchar_t
   ,/*Default memory algorithm defining void * as void_pointer*/
   ,/*Default index type*/>
wfixed_managed_shared_memory;

Managed shared memory is an advanced class that combines a shared memory object and a mapped region that covers all the shared memory object. That means that when we create a new managed shared memory:

  • A new shared memory object is created.
  • The whole shared memory object is mapped in the process' address space.
  • Some helper objects are constructed (name-object index, internal synchronization objects, internal variables...) in the mapped region to implement managed memory segment features.

When we open a managed shared memory

  • A shared memory object is opened.
  • The whole shared memory object is mapped in the process' address space.

To use a managed shared memory, you must include the following header:

#include <boost/interprocess/managed_shared_memory.hpp>
//1.  Creates a new shared memory object
//    called "MyName".
//2.  Maps the whole object to this
//    process' address space.
//3.  Constructs some objects in shared memory
//    to implement managed features.
//!!  If anything fails, throws interprocess_exception
//
managed_shared_memory segment      ( create_only
                                   , "MyName"    //Shared memory object name
                                   , 65536);     //Shared memory object size in bytes
//1.  Opens a shared memory object
//    called "MyName".
//2.  Maps the whole object to this
//    process' address space.
//3.  Obtains pointers to constructed internal objects
//    to implement managed features.
//!!  If anything fails, throws interprocess_exception
//
managed_shared_memory segment (open_only, "MyName"); //Shared memory object name
//1.  If the segment was previously created
//    equivalent to "open_only" (size is ignored).
//2.  Otherwise, equivalent to "create_only"
//!!  If anything fails, throws interprocess_exception
//
managed_shared_memory segment      ( open_or_create
                                   , "MyName"    //Shared memory object name
                                   , 65536);     //Shared memory object size in bytes

When the managed_shared_memory object is destroyed, the shared memory object is automatically unmapped, and all the resources are freed. To remove the shared memory object from the system you must use the shared_memory_object::remove function. Shared memory object removing might fail if any process still has the shared memory object mapped.

The user can also map the managed shared memory in a fixed address. This option is essential when using using fixed_managed_shared_memory. To do this, just add the mapping address as an extra parameter:

fixed_managed_shared_memory segment      (open_only      ,"MyFixedAddressSharedMemory" //Shared memory object name
   ,(void*)0x30000000            //Mapping address

Windows users might also want to use native windows shared memory instead of the portable shared_memory_object managed memory. This is achieved through the basic_managed_windows_shared_memory class. To use it just include:

#include <boost/interprocess/managed_windows_shared_memory.hpp>

This class has the same interface as basic_managed_shared_memory but uses native windows shared memory. Note that this managed class has the same lifetime issues as the windows shared memory: when the last process attached to the windows shared memory is detached from the memory (or ends/crashes) the memory is destroyed. So there is no persistence support for windows shared memory.

To communicate between system services and user applications using managed_windows_shared_memory, please read the explanations given in chapter Native windows shared memory.

Unix users might also want to use XSI (system V) instead of the portable shared_memory_object managed memory. This is achieved through the basic_managed_xsi_shared_memory class. To use it just include:

#include <boost/interprocess/managed_xsi_shared_memory.hpp>

This class has nearly the same interface as basic_managed_shared_memory but uses XSI shared memory as backend.

For more information about managed XSI shared memory capabilities, see basic_managed_xsi_shared_memory class reference.

As seen, basic_managed_mapped_file offers a great variety of customization. But for the average user, a common, default shared memory named object creation is needed. Because of this, Boost.Interprocess defines the most common managed mapped file specializations:

//Named object creation managed memory segment
//All objects are constructed in the memory-mapped file
//   Names are c-strings,
//   Default memory management algorithm(rbtree_best_fit with no mutexes)
//   Name-object mappings are stored in the default index type (flat_map)
typedef basic_managed_mapped_file <
   char,
   rbtree_best_fit<mutex_family, offset_ptr<void> >,
   flat_map_index
   >  managed_mapped_file;

//Named object creation managed memory segment
//All objects are constructed in the memory-mapped file
//   Names are wide-strings,
//   Default memory management algorithm(rbtree_best_fit with no mutexes)
//   Name-object mappings are stored in the default index type (flat_map)
typedef basic_managed_mapped_file<
   wchar_t,
   rbtree_best_fit<mutex_family, offset_ptr<void> >,
   flat_map_index
   >  wmanaged_mapped_file;

managed_mapped_file allocates objects in a memory mapped files associated with a c-string and wmanaged_mapped_file allocates objects in a memory mapped file associated with a wchar_t null terminated string. Both define the pointer type as offset_ptr<void> so they can be used to map the file at different base addresses in different processes.

Managed mapped file is an advanced class that combines a file and a mapped region that covers all the file. That means that when we create a new managed mapped file:

  • A new file is created.
  • The whole file is mapped in the process' address space.
  • Some helper objects are constructed (name-object index, internal synchronization objects, internal variables...) in the mapped region to implement managed memory segment features.

When we open a managed mapped file

  • A file is opened.
  • The whole file is mapped in the process' address space.

To use a managed mapped file, you must include the following header:

#include <boost/interprocess/managed_mapped_file.hpp>
//1.  Creates a new file
//    called "MyMappedFile".
//2.  Maps the whole file to this
//    process' address space.
//3.  Constructs some objects in the memory mapped
//    file to implement managed features.
//!!  If anything fails, throws interprocess_exception
//
managed_mapped_file mfile      ( create_only
                               , "MyMappedFile" //Mapped file name
                               , 65536);        //Mapped file size
//1.  Opens a file
//    called "MyMappedFile".
//2.  Maps the whole file to this
//    process' address space.
//3.  Obtains pointers to constructed internal objects
//    to implement managed features.
//!!  If anything fails, throws interprocess_exception
//
managed_mapped_file mfile      ( open_only
                               , "MyMappedFile"); //Mapped file name
//1.  If the file was previously created
//    equivalent to "open_only".
//2.  Otherwise, equivalent to "open_only" (size is ignored)
//
//!!  If anything fails, throws interprocess_exception
//
managed_mapped_file mfile      (open_or_create
                               , "MyMappedFile" //Mapped file name
                               , 65536);        //Mapped file size

When the managed_mapped_file object is destroyed, the file is automatically unmapped, and all the resources are freed. To remove the file from the filesystem you could use standard C std::remove or Boost.Filesystem's remove() functions, but file removing might fail if any process still has the file mapped in memory or the file is open by any process.

To obtain a more portable behaviour, use file_mapping::remove(const char *) operation, which will remove the file even if it's being mapped. However, removal will fail in some OS systems if the file (eg. by C++ file streams) and no delete share permission was granted to the file. But in most common cases file_mapping::remove is portable enough.

For more information about managed mapped file capabilities, see basic_managed_mapped_file class reference.


PrevUpHomeNext