Prns performance claims: an accounting of fudged measurements
1. The claim
The front page of the Prns website states, as a headline, that performance is "Measured, not just claimed", and that every number "comes from the published results in the repo, measured on real hardware by a harness you can run yourself" (docs/website/i18n/en-US.ftl). The README states "up to 89× the throughput" (README.md), and the on-network marketing page repeats "up to 89× the throughput, 48× smaller peak-memory footprint, and 33× the energy efficiency of stock RNS 1.4.2" (assets/nnpages/why_prns.mu). A comprehensive and multi-proged marketing rollout, which we will focus on in more detail later.
The "89×" is, in their own published tables, a single cell: single-packet throughput on macOS, measured at 37,400/s for Prns against 420/s for the reference (benchmarks/RESULTS-aarch64-apple-darwin.md). The same scenario measures 7.19× on Linux and 8.00× on Windows. The factor of twenty between hosts is not Prns — it is an invalid test reference harness collapsing to a different degree on each platform.
This is not an accident of hardware. The reason the test reference collapses so uniformly is that the reference is not RNS. It is a mislabeled, broken development shim, further hobbled by the interface and policy choices the harness itself imposes. Three stacked handicaps, each verifiable in the repository, account for essentially the entire claimed advantage of Prns. Either this was intentional, or more likely; nobody bothered to check anything.
It shall additionally be noted here, that Prns publishes no actual measurements of real-world performance or reliability metrics on heavily utilised nodes with hundreds of interfaces, or any benchmarks covering correctness of traffic handling, announce/PR flood rejection, et cetera; only extremely synthetic throughput benchmarks.
2. Handicap one: the "compiled RNS" reference is a defunct development shim
The benchmark harness loads the reference through benchmarks/reference/compiled_reference.py, which installs pyximport and asserts two things:
RNS.compiled == True, and
- that at least one RNS module was loaded from a native extension (
*.so / *.pyd).
Both checks are trivially satisfied, and their own published proof records show exactly what they captured. From the currently published suites:
benchmarks/results/aarch64-apple-darwin/suites/a52f5446-.../suite.json:
"cython": "3.2.8", "python": "3.13.13",
"native_module": "<repo>/.../pyximport/lib.macosx-26.0-arm64-cpython-313/RNS.cpython-313-darwin.so"
benchmarks/results/x86_64-unknown-linux-gnu/suites/c0595056-.../suite.json:
"native_module": "<repo>/.../pyximport/lib.linux-x86_64-cpython-313/RNS.cpython-313-x86_64-linux-gnu.so"
benchmarks/results/x86_64-pc-windows-msvc/suites/729ba968-.../suite.json:
"native_module": "~/.prns-oc/lib.win-amd64-cpython-313/RNS.cp313-win_amd64.pyd"
In every case the recorded native module is the top-level package only: RNS.cpython-313-<platform>.so. Nothing under RNS.Transport, RNS.Destination, RNS.Identity, RNS.Packet, RNS.Link or RNS.Resource ever appears — because nothing under those was ever compiled. This is the old CRNS development shim, whose pyximport-based "compilation" has been known for over a year to compile only the top-level __init__ module on modern Cython, leaving the entire actual protocol machinery in interpreted mode. It is a development utility that produces debug-oriented builds that are slower than plain interpreted RNS.
It was, in fact, deprecated and non-functional for more than a year before these benchmarks were published, and has only recently been repaired in the reference implementation itself — after the fact.
In short: "RNS 1.4.2 (compiled)" — the label used in every table, every chart, and every marketing page — is not a mode of RNS that exists, and is not faster than interpreted RNS. It is the opposite: a stale tool that makes RNS look far slower than it actually is.
The harness's own documentation (benchmarks/implementations/rns-1.4.2-compiled.json) says runs are "rejected unless RNS.compiled is true and native extension modules are loaded" — as if that constituted a meaningful verification. It verifies a flag and the presence of a .so. It never verifies what was compiled. It also never states, anywhere, what "compiled" means, that it is a development shim, or that it builds debug targets.
Interpreted RNS 1.5.4 on an ordinary mid-range laptop, single core, sustains roughly 30,000 deliveries/s for small single packets (including ephemeral-key decryption), about 175,000–280,000 packets/s in transit relay at 135–475 B payloads, and 10+ Gbps throughput at 16 KiB payloads. The "compiled" artifact in these benchmarks sustained 420–3,900 packets/s for its single-packet scenario, depending on host. That is the artifact being broken — not Python being slow. Any benchmark that claims to measure "Rust vs Python" while its Python side performs 10–100× worse than stock interpreted Python is measuring its own rigging, not engineering.
3. Handicap two: the reference was forced onto the slowest possible interface types
The harness does not use RNS's modern BackboneInterface on any platform, on either the endpoint or the relay side. The harness itself writes the reference's configuration (benchmarks/reference/participant_node.py):
type = TCPServerInterface (responder, and both sides of the relay)
type = TCPClientInterface (initiator)
type = UDPInterface (the abstract wire between the two hops)
TCPInterface is the legacy interface in RNS, maintained for stability and compatibility with older systems. BackboneInterface is the modern high-performance interface — and on macOS it is, moreover, not even usable in this configuration on RNS 1.4.2 without development branches (which Prns did not have access to, because they never asked or consulted RNS developers on anything before publishing their "benchmarks"). A fair modern-interface comparison was therefore structurally impossible in their setup, and none was even attempted.
The results tables never mention the interface module selection at all; the reader is left to assume the reference was configured with reasonable modern settings. It was not: It was configured to be as slow as possible.
4. Handicap three: the "default" rows compare different interface policies outright
The published tables state the policies openly, in a footnote, as if this was a virtue:
"This practical profile preserves each implementation's normal TCP policy: 500 Mbps for Prns and 10 Mbps for compiled RNS 1.4.2."
Their own tables (benchmarks/RESULTS-x86_64-unknown-linux-gnu.md):
| Scenario |
Prns policy / MTU |
RNS policy / MTU |
Published ratio |
| Raw transport throughput |
500 Mbps / 128 KiB |
10 Mbps / 8 KiB |
20.65× |
| Raw transport throughput (Windows) |
— |
— |
38.84× |
Raw transport throughput at a 10 Mbps MTU policy compared against a 500 Mbps MTU policy is not "20–39× faster". It is 20–39× more policy. When the harness equalizes the policy — the rows it labels "1 Gbps policy", in which both sides are explicitly configured identically — the published ratios collapse to roughly 2.4–8.4× across all hosts and scenarios. Keep in mind, the gains here are still measured against the broken baseline of section 2, but even so: nowhere near 89×, and consistent with the simple, uncontroversial observation that a native implementation tends to outperform an interpreted one in raw throughput — a point that needed no fudging to make.
Interestingly, a "high-performance" Rust implementation is apparently not really that much faster than the Python reference implementation - running in interpreted mode on a single CPU core.
5. What this means
- The "89×" headline is the worst-case cell of a broken measurement. It is one scenario, on one host, where a stale debug-oriented Cython shim and deliberate slow interface choices collapsed to 420 packets/s. The honest reading of their own cross-host data is that the multiplier is dominated by the reference artifact test harness' instability, not by Prns's speed.
- Every published ratio against "RNS 1.4.2 (compiled)" is void, because that artifact is not a valid measure of RNS on any axis: it is a deprecated dev shim, debug-targeted, top-level-only, on a Python version where it is known-broken, running a legacy interface at a fraction of the allowed policy.
- No methodology section anywhere addresses any of this. The writeups describe machines, durations, and accounting rules in detail — and omit most of the important methodology; the interface module used for the reference, the meaning of "compiled", the fact that it is a dev-only debug shim, and the policy asymmetry they themselves document in passing. That is not an oversight; that is selection of the one presentation that yields the marketing number.
- The front page says "Measured, not just claimed." On this evidence, the correct summary is: claimed, and arranged to look measured.
For anyone who wants an honest comparison: run stock interpreted RNS, current version, on BackboneInterface where available, at identical interface policy, with methodology published in advance. The results will be less flattering — and worth a lot more.