Step 5 of 5

Custom allocators with std::pmr

new and malloc are general-purpose: they handle any size, any order of frees, from any thread. That generality costs. An allocation may take a lock, search free lists, or ask the operating system for more memory, and its time varies. Low-latency code (games, trading, audio) often forbids heap allocation in its hot loop entirely, and uses arenas and pools instead:

• An arena (monotonic buffer) hands out memory by moving a pointer forward through one big block, and frees everything at once when it's reset. Allocation is a few instructions. The "Write a memory allocator" project has you build one.
• A pool keeps a free list of fixed-size blocks, so allocating or freeing a block is a pop or a push. The same project builds one of these too.

C++17's polymorphic memory resources (std::pmr, header <memory_resource>) let standard containers use them. A std::pmr::vector<int> is a vector that asks a std::pmr::memory_resource*, chosen at run time, for its memory:

#include <array>
#include <cstddef>
#include <iostream>
#include <memory_resource>
#include <vector>

int main() {
    std::array<std::byte, 4096> buffer;                   // lives on the stack
    std::pmr::monotonic_buffer_resource arena(
        buffer.data(), buffer.size(),
        std::pmr::null_memory_resource());                // no fallback: never touch the heap
    std::pmr::vector<int> squares(&arena);
    squares.reserve(100);
    for (int i = 0; i < 100; i++) squares.push_back(i * i);
    std::pmr::vector<int> firsts(squares.begin(), squares.begin() + 10, &arena);
    std::cout << squares.back() << " " << firsts.size() << "\n";
}                                                         // the arena releases everything at once

9801 10

How it works

• monotonic_buffer_resource hands out pieces of buffer. Freeing an individual piece does nothing; all of it is released when the arena is destroyed. Perfect for "build some data, use it, throw it all away", like handling one request.
• The last argument is the upstream resource, used when the buffer runs out. null_memory_resource() refuses every request (it throws std::bad_alloc), which turns "we accidentally allocated on the heap" into an immediate failure instead of a silent slowdown.
• std::pmr::unsynchronized_pool_resource and synchronized_pool_resource (thread-safe) are pools, and new_delete_resource() is the normal heap.
• A std::pmr::vector<int> is a different type from std::vector<int> (their allocator types differ), so the two don't mix without copying. There are pmr versions of string, map and the other containers.
• You write your own resource by deriving from std::pmr::memory_resource and overriding three functions: do_allocate(bytes, alignment), do_deallocate(p, bytes, alignment) and do_is_equal(other).

Your task

Write CountingResource, a memory resource that passes every request on to an upstream resource (the normal heap by default) and counts: allocations() is the number of allocations so far, and bytes_in_use() is the number of bytes currently allocated and not yet freed. It's a tool for proving that code allocates as little as you think.

Challenge 2

A std::pmr::monotonic_buffer_resource hands out memory from a buffer you give it, which is very fast. With std::pmr::null_memory_resource() as its upstream, it throws std::bad_alloc instead of falling back to the heap once the buffer runs out. Write bool fits(std::size_t count): with a local 1024-byte buffer, build a std::pmr::vector<int> and reserve(count); return false if that throws.

Challenge 3

Write CappedResource, a memory resource that passes requests on to an upstream resource but refuses (throws std::bad_alloc) any allocation that would take the bytes in use over cap. Freed memory counts again, so it can be reused.

Previous: False sharing and alignment