Besides the hierarchy of objects and individual object attributes (see Object attributes), hwloc may also expose finer information about the hardware organization.
Distances
A machine with 4 CPUs may have identical links between every pairs of CPUs, or those CPUs could also only be connected through a ring. In the ring case, accessing the memory of nearby CPUs is slower than local memory, but it is also faster than accessing the memory of CPU on the opposite side of the ring. These deep details cannot be exposed in the hwloc hierarchy, that is why hwloc also exposes distances.
Distances are matrices of values between sets of objects, usually latencies or bandwidths. By default, hwloc tries to get a matrix of relative latencies between NUMA nodes when exposed by the hardware.
In the aforementioned ring case, the matrix could report 10 for latency between a NUMA node and itself, 20 for nearby nodes, and 30 for nodes that are opposites on the ring. Those are theoretical values exposed by hardware vendors (in the System Locality Distance Information Table (SLIT) in the ACPI) rather than physical latencies. They are mostly meant for comparing node relative distances.
Distances structures currently created by hwloc are:
- NUMALatency (Linux, Solaris, FreeBSD)
- This is the matrix of theoretical latencies described above.
Users may also specify their own matrices between any set of objects, even if these objects are of different types (e.g. bandwidths between GPUs and CPUs).
The entire API is located in hwloc/distances.h. See also Retrieve distances between objects, as well as Helpers for consulting distance matrices and Add or remove distances between objects.
Memory Attributes
Machines with heterogeneous memory, for instance high-bandwidth memory (HBM), normal memory (DDR), and/or high-capacity slow memory (such as non-volatile memory DIMMs, NVDIMMs) require applications to allocate buffers in the appropriate target memory depending on performance and capacity needs. Those target nodes may be exposed in the hwloc hierarchy as different memory children but there is a need for performance information to select the appropriate one.
hwloc memory attributes are designed to expose memory information such as latency, bandwidth, etc. Users may also specify their own attributes and values.
The memory attributes API is located in hwloc/memattrs.h, see Comparing memory node attributes for finding where to allocate on and Managing memory attributes for details.
CPU Kinds
Hybrid CPUs may contain different kinds of cores. The CPU kinds API in hwloc/cpukinds.h provides a way to list the sets of PUs in each kind and get some optional information about their hardware characteristics and efficiency.
If the operating system provides efficiency information (e.g. Windows 10), it is used to rank hwloc CPU kinds by efficiency. Otherwise, hwloc implements several heuristics based on frequencies and core types (see HWLOC_CPUKINDS_RANKING in Environment Variables).
Attributes include:
- FrequencyMaxMHz (Linux)
- The maximal operating frequency of the core, as reported by
cpufreq drivers on Linux.
- FrequencyBaseMHz (Linux)
- The base operating frequency of the core, as reported by some
cpufreq drivers on Linux (e.g. intel_pstate).
- CoreType (x86, Linux)
- A string describing the kind of core, currently
IntelAtom or IntelCore, as reported by the x86 CPUID instruction and future Linux kernels on some Intel processors.
- LinuxCPUType (Linux)
- The Linux-specific CPU type found in sysfs, such as
intel_atom_0, as reported by future Linux kernels on some Intel processors.
See Kinds of CPU cores for details.
