Short answer: pqm4 is an open-source library, benchmarking and testing framework for post-quantum key encapsulation and signature schemes on ARM Cortex-M4 microcontrollers. Its project page lists automated benchmarking for speed, stack usage and code size, and profiling of cycles spent in symmetric primitives. An Arm learning path describes how its build generates test, speed, stack and hashing binaries that run on a Cortex-M4 board or in QEMU. This post describes the tooling and quotes a few figures from the project's results file.
What binaries does the build system generate?
After building the pqm4 library, the system generates a set of binaries for each cryptographic scheme and implementation. These binaries are designed to verify correctness, measure performance, and analyze resource usage. The project page says the build system compiles six binaries for each implementation.
For example, when compiling the reference implementation of ML-KEM-768, the system creates specific executable files. One binary, such as crypto_kem_ml-kem-768_m4_test.bin, tests if the scheme works as expected. For Key Encapsulation Mechanisms (KEMs), this test verifies if Alice and Bob derive the same shared key. For signature schemes, it checks if a generated signature can be verified correctly. Several failure cases are also checked during this process to ensure robustness.
How does the framework measure speed and hashing?
The speed binary measures execution time in CPU cycles for each cryptographic operation. This metric allows developers to quantify the computational cost of key generation, encapsulation, and decapsulation. The project lists separate implementations: m4fspeed optimized for speed, m4fstack optimized for stack size, m4f for some schemes such as BIKE, and clean reference implementations from PQClean.
Additionally, a hashing binary measures how many cycles are spent in symmetric primitives such as SHA-2, SHA-3, and AES. This specific measurement reveals how much of the overall algorithm cost comes from hashing operations.
What metrics does the stack binary provide?
The stack binary measures peak stack memory usage for each operation. It reports this for each operation.
However, stack measurement might not work correctly on some boards due to platform-specific memory layout. Furthermore, memory allocated outside functions, such as public keys and ciphertexts, is not included in these measurements.
Example figures from the project's results file
The repository's benchmarks.csv, as read for this post, lists mean cycle counts for ML-KEM-768 with the m4fspeed implementation of 642,096 for key generation, 658,754 for encapsulation and 707,827 for decapsulation. For ML-KEM-1024 with m4fspeed it lists 1,018,976, 1,031,565 and 1,094,008. The same file lists the clean reference implementation of ML-KEM-1024 at 1,536,343, 1,708,071 and 2,020,327. These are the project's own measured values for one set of builds and may change as the repository is updated, so check the file for current numbers.
How are benchmark results stored and analyzed?
The benchmarks.py script runs speed and stack benchmarks and stores the results in a benchmarks/ directory. The output of running these benchmarks includes progress indicators showing the current scheme and implementation being tested. For instance, the output may display progress for ml-kem-768 using the m4fspeed implementation.
Results are saved to a benchmarks.csv file. This CSV file contains detailed numerical data for speed and memory evaluations. The speed evaluation section includes columns for key generation, encapsulation, and decapsulation, reporting mean, minimum, and maximum cycle counts. The memory evaluation section reports peak stack usage in bytes for each operation.
FAQ
Can pqm4 run benchmarks in QEMU?
Yes, the generated binaries can be used to verify correctness and measure performance on a Cortex-M4 board or in QEMU.
What does the speed binary measure?
The speed binary measures execution time in CPU cycles for each cryptographic operation. It quantifies the computational cost of key generation, encapsulation, and decapsulation, providing mean, minimum, and maximum cycle counts for performance analysis.
Does the stack binary include all memory usage?
No, the stack binary measures peak stack memory usage for each operation. Memory allocated outside functions, such as public keys and ciphertexts, is not included in these measurements. Platform-specific memory layouts may also affect measurement accuracy.
What symmetric primitives are profiled?
The framework profiles cycles spent in symmetric primitives such as SHA-2, SHA-3, and AES. This hashing binary measurement shows how much of the overall algorithm cost comes from these specific cryptographic components.
What data is stored in benchmarks.csv?
The benchmarks.csv file stores speed and memory evaluation results. It includes cycle counts for key generation, encapsulation, and decapsulation, as well as peak stack usage in bytes for each scheme and implementation tested.