.\" Automatically generated by Pandoc 2.10.1
.\"
.TH "LIBPMEM2" "7" "2020-10-28" "PMDK - pmem2 API version 1.0" "PMDK Programmer's Manual"
.hy
.\" SPDX-License-Identifier: BSD-3-Clause
.\" Copyright 2019-2020, Intel Corporation
.SH NAME
.PP
\f[B]libpmem2\f[R] - persistent memory support library (EXPERIMENTAL)
.SH SYNOPSIS
.IP
.nf
\f[C]
#include <libpmem2.h>
cc ... -lpmem2
\f[R]
.fi
.SH DESCRIPTION
.SH GRANULARITY
.PP
The \f[B]libpmem2\f[R] library introduces the concept of granularity
through which you may easily distinguish between different types of
reaching \f[I]power-fail protected domain\f[R] by data.
Data can reach this domain in different ways depending on the platform
capabilities.
.PP
Traditional block storage devices (SSD, HDD) must use system API calls
such as \f[C]msync()\f[R], \f[C]fsync()\f[R] on Linux, or
\f[C]FlushFileBuffers()\f[R],\f[C]FlushViewOfFile()\f[R] on Windows to
write data reliably.
Invoking these functions flushes the data to the medium with page
granularity.
In the \f[B]libpmem2\f[R] library, this type of flushing behavior is
called \f[B]PMEM2_GRANULARITY_PAGE\f[R].
.PP
In systems with persistent memory support, a \f[I]power-fail protected
domain\f[R] may cover different sets of resources: either the memory
controller or the memory controller and CPU caches.
In this regard, \f[B]libpmem2\f[R] distinguishes two types of
granularity for persistent memory:
\f[B]PMEM2_GRANULARITY_CACHE_LINE\f[R] and
\f[B]PMEM2_GRANULARITY_BYTE\f[R].
.PP
If the \f[I]power-fail protected domain\f[R] covers only the memory
controller, it is required to flush CPU caches, so the granularity type,
in this case, is called \f[B]PMEM2_GRANULARITY_CACHE_LINE\f[R].
Depending on the architecture, there are different types of machine
instructions for flushing \f[I]cache lines\f[R] (e.g., \f[I]CLWB\f[R],
\f[I]CLFLUSHOPT\f[R], \f[I]CLFLUSH\f[R] for Intel x86_64 architecture).
Usually, to ensure the ordering of stores, such instructions must be
followed by a barrier (e.g., \f[I]SFENCE\f[R]).
.PP
The third type of granularity \f[B]PMEM2_GRANULARITY_BYTE\f[R] applies
to platforms where \f[I]power-fail protected domain\f[R] covers both the
memory controller and CPU caches.
In such cases, cache flush instructions are no longer needed, and the
platform itself guarantees the persistence of data.
But barriers might still be required for ordering.
.PP
The library declares these granularity level in
\f[I]pmem2_granularity\f[R] enum, which the application must set in
\f[I]pmem2_config\f[R] to the appropriate level for a mapping a succeed.
The software should set this config parameter to a value that most
accurately represents the target hardware characteristics and the
storage patterns of the application.
For example, a database storage engine that operates on large logical
pages that reside either on SSDs or PMEM should set this value to
\f[B]PMEM2_GRANULARITY_PAGE\f[R].
.SH CAVEATS
.SH ENVIRONMENT
.PP
\f[B]libpmem2\f[R] can change its default behavior based on the
following environment variables.
These are primarily intended for testing and are generally not required.
.IP \[bu] 2
\f[B]PMEM2_FORCE_GRANULARITY\f[R]=\f[I]val\f[R]
.PP
Setting this environment variable to \f[I]val\f[R] forces
\f[B]libpmem2\f[R] to use persist method specific for forced granularity
and skip granularity autodetecting mechanism.
The concept of the granularity is described in \f[I]GRANULARITY\f[R]
section above.
This variable is intended for use during library testing.
.PP
The \f[I]val\f[R] argument accepts following text values:
.IP \[bu] 2
\f[B]BYTE\f[R] - force byte granularity.
.IP \[bu] 2
\f[B]CACHE_LINE\f[R] - force cache line granularity.
.IP \[bu] 2
\f[B]PAGE\f[R] - force page granularity.
.PP
Granularity values listed above are case-insensitive.
.RS
.PP
NOTE: The value of \f[B]PMEM2_FORCE_GRANULARITY\f[R] is not queried (and
cached) at library initialization time, but read during each
\f[B]pmem2_map\f[R](3) call.
.RE
.PP
This means that \f[B]PMEM2_FORCE_GRANULARITY\f[R] may still be set or
modified by the program until the first attempt to map a file.
.IP \[bu] 2
\f[B]PMEM_NO_CLWB\f[R]=1
.PP
Setting this environment variable to 1 forces \f[B]libpmem2\f[R] to
never issue the \f[B]CLWB\f[R] instruction on Intel hardware, falling
back to other cache flush instructions on that hardware instead
(\f[B]CLFLUSHOPT\f[R] or \f[B]CLFLUSH\f[R]).
Without this setting, \f[B]libpmem2\f[R] will always use the
\f[B]CLWB\f[R] instruction for flushing processor caches on platforms
that support this instruction.
This variable is intended for use during library testing, but may be
required for some rare cases when using \f[B]CLWB\f[R] has a negative
impact on performance.
.IP \[bu] 2
\f[B]PMEM_NO_CLFLUSHOPT\f[R]=1
.PP
Setting this environment variable to 1 forces \f[B]libpmem2\f[R] to
never issue the \f[B]CLFLUSHOPT\f[R] instruction on Intel hardware,
falling back to the \f[B]CLFLUSH\f[R] instructions instead.
Without this environment variable, \f[B]libpmem2\f[R] will always use
the \f[B]CLFLUSHOPT\f[R] instruction for flushing processor caches on
platforms that support the instruction, but where \f[B]CLWB\f[R] is not
available.
This variable is intended for use during library testing.
.IP \[bu] 2
\f[B]PMEM_NO_MOVNT\f[R]=1
.PP
Setting this environment variable to 1 forces \f[B]libpmem2\f[R] to
never use the \f[I]non-temporal\f[R] move instructions on Intel
hardware.
Without this environment variable, \f[B]libpmem2\f[R] will use the
non-temporal instructions for copying larger ranges to persistent memory
on platforms that support these instructions.
This variable is intended for use during library testing.
.IP \[bu] 2
\f[B]PMEM_MOVNT_THRESHOLD\f[R]=\f[I]val\f[R]
.PP
This environment variable allows overriding the minimum length of the
\f[I]pmem2_memmove_fn\f[R] operations, for which \f[B]libpmem2\f[R] uses
\f[I]non-temporal\f[R] move instructions.
Setting this environment variable to 0 forces \f[B]libpmem2\f[R] to
always use the \f[I]non-temporal\f[R] move instructions if available.
It has no effect if \f[B]PMEM_NO_MOVNT\f[R] is set to 1.
This variable is intended for use during library testing.
.SH DEBUGGING
.PP
Two versions of \f[B]libpmem2\f[R] are typically available on a
development system.
The normal version, accessed when a program is linked using the
\f[B]-lpmem2\f[R] option, is optimized for performance.
That version skips checks that impact performance and never logs any
trace information or performs any run-time assertions.
.PP
A second version of \f[B]libpmem2\f[R], accessed when a program uses the
libraries under \f[B]/usr/lib/pmdk_debug\f[R], contains run-time
assertions and trace points.
The typical way to access the debug version is to set the environment
variable \f[B]LD_LIBRARY_PATH\f[R] to \f[B]/usr/lib/pmdk_debug\f[R] or
\f[B]/usr/lib64/pmdk_debug\f[R], as appropriate.
Debugging output is controlled using the following environment
variables.
These variables have no effect on the non-debug version of the library.
.IP \[bu] 2
\f[B]PMEM2_LOG_LEVEL\f[R]
.PP
The value of \f[B]PMEM2_LOG_LEVEL\f[R] enables trace points in the debug
version of the library, as follows:
.IP \[bu] 2
\f[B]0\f[R] - This is the default level when \f[B]PMEM2_LOG_LEVEL\f[R]
is not set.
No log messages are emitted at this level.
.IP \[bu] 2
\f[B]1\f[R] - Additional details on any errors detected are logged, in
addition to returning the \f[I]errno\f[R]-based errors as usual.
The same information may be retrieved using \f[B]pmem2_errormsg\f[R]().
.IP \[bu] 2
\f[B]2\f[R] - A trace of basic operations is logged.
.IP \[bu] 2
\f[B]3\f[R] - Enables a very verbose amount of function call tracing in
the library.
.IP \[bu] 2
\f[B]4\f[R] - Enables voluminous and fairly obscure tracing information
that is likely only useful to the \f[B]libpmem2\f[R] developers.
.PP
Unless \f[B]PMEM2_LOG_FILE\f[R] is set, debugging output is written to
\f[I]stderr\f[R].
.IP \[bu] 2
\f[B]PMEM2_LOG_FILE\f[R]
.PP
Specifies the name of a file where all logging information should be
written.
If the last character in the name is \[lq]-\[rq], the \f[I]PID\f[R] of
the current process will be appended to the file name when the log file
is created.
If \f[B]PMEM2_LOG_FILE\f[R] is not set, output is written to
\f[I]stderr\f[R].
.SH EXAMPLE
.SH ACKNOWLEDGEMENTS
.PP
\f[B]libpmem2\f[R] builds on the persistent memory programming model
recommended by the SNIA NVM Programming Technical Work Group:
<https://snia.org/nvmp>
.SH SEE ALSO
.PP
\f[B]FlushFileBuffers\f[R](), \f[B]fsync\f[R](2), \f[B]msync\f[R](2),
\f[B]pmem2_config_set_required_store_granularity\f[R](3),
\f[B]pmem2_get_memset_fn\f[R](3),
\f[B]pmem2_map_get_store_granularity\f[R](3), \f[B]libpmemblk\f[R](7),
\f[B]libpmemlog\f[R](7), \f[B]libpmemobj\f[R](7) and
\f[B]<https://pmem.io>\f[R]
