# RNS 1.6.0 Released _General · started by Mark on Sat, Nov 7, 2026 4:56 PM_ --- ## Original post **Mark** · Sat, Nov 7, 2026 4:56 PM This major release of Reticulum brings significant improvements to resource consumption, performance, stability and resilience, especially for constrained devices and nodes. **Memory & Performance Improvements** A new, SQLite based storage backend has been implemented, and is now the default. All legacy file-based storage will be auto-migrated to the database-backed store on the first start. If you are using an esoteric system that does not support the `sqlite3` module, Reticulum will automatically fall back to using the file-based storage system. The memory management system has been completely reworked, and significantly improved. For long-running transport nodes, this cuts memory consumption by approximately 50%, and for non-transport instances, standalone applications and appliance devices by up to 80%. **Path Table Sharding** This release implements the initial transport awareness and mechanisms for emergent path table sharding. Transport instances will no longer attempt to hold full inter-network path tables in memory, but keep only paths relevant to the nodes they are transporting traffic for. Unused paths are now kept for much shorter periods of time, and used as a rolling pool that potential new traffic corridors can form into. The system is entirely passive, and adds no inter-node coordination or additional protocol mechanics. The current release brings the foundation, and the upcoming 1.6.x releases will tune, improve it and add the remaining pieces of the functionality. **Network Stamps** The stamp system, first introduced in LXMF, where it was tested and refined over the last two years, has now been brought into the core of Reticulum to help protect against spam and network abuse. All the relevant core primtives now support stamp generation and verification: - Identities can be created with an embedded stamp value. This adds no additional bytes or identity format changes. Generation and verification is completely transparent, and carried within the Reticulum Identity keyset. - Destinations can now be stamped, and will embed stamps and required link request cost in their announces. The change is backwards-compatible, so even older versions of Reticulum carry stamped announces transparently. Destination stamps are one-time generations, stored and persisted on success, so these have relatively high default target values. Expect some CPU usage when an unstamped destination has its stamp generated for the first time on 1.6.0+. - Path requests can now be stamped, and are also backwards-compatible. A stamped path request will be carried transparently by non stamp-aware transport nodes. - Link requests are stamped according to the announced link request cost of the destination. As with all the other primitives, these are also completely transparent and backwards-compatible. The stamp is carried embedded in the ephemeral link request keyset. The stamp functionality and requirements will be deployed in a phased transition, so both network operators and application developers have time to update their systems, and any potential bugs and issues can be worked out before the full deployment: - Version `1.6.0` implements all the core functionality and primitives, and starts excercising the verification mechanics. By default, no destination and path request stamps are generated yet in this release, and while stamps are checked and verified, they are not *enforced*. However, both stamp emission and enforcement can be enabled in the configuration for testing or early opt-in. Link request stamps *will* be generated, if the destination has announced a cost. - Version `1.6.1` will enable stamp emission for all network primitives by default. Old, unstamped destinations will have their stamps generated automatically on their first announce, and will start announcing link request costs as configured. Enforcement in this phase will still be disabled by default, but can be enabled by configuration. - Version `1.6.2` will complete the transition to network stamps. All functionality will now be enabled by default, including enforcement. Destination announces, path requests and link requests that do not meet stamp requirements will be dropped. Any old transport nodes that have not updated to `1.6.x` will not be able to communicate with stamp-enforcing nodes, but will still *carry traffic for* stamp-capable systems. The `1.6.0` and `1.6.1` releases will be an important time for observing the on-network behaviour, functionality and stability of the system, and to make any potential adjustments or fix problems before the final transition to `1.6.2`. If you're operating a transport node, please help out by reporting any problems or issues you discover. **For application developers**, there is no immediate refactoring or changes required application-side. Simply updating RNS to `1.6.0` is sufficient to transparently enable all the new functionality and improvements. It is very much recommended, however, to familiarize yourself with the new API methods for specifying link request costs and destination values, as these are configurable on a per-destination basis, and might warrant specific configuration for your intended use-case. **Changes** - Added `Identity` stamps - Added destination and announce stamps - Added path request stamps - Added link request stamps - Added link request cost signalling to announces - Added fractional stamp value support - Added path usage tracking - Added automatic path table culling based on traffic and usage - Added path retainment prioritization for slow interfaces - Added unused path pool monitoring and handling - Added managed storage backend using SQLite - Added fallback to legacy file-based storage if SQLite is unavailable - Added managed storage for known destinations and ratchets - Added managed storage for cached announces - Added memory-density optimized announce rate table implementation - Added memory-density optimized packet hashlist implementation - Added heap-tracking memory management - Added more comprehensive metrics to status output structure - Added improved filtering backend capabilities - Added new `[filter]` configuration section - Added `hop_limiter` configuration option - Added allow and ignore lists for destination types - Added ability for applications to receive required announces only - Added cost-aware interface discovery and auto-connects - Added cost and value display to `rnstatus` and `rnpath` utilities - Added `app_flag` signalling to resource advertisements - Added link MTU aware resource advertisements - Significantly improved inbound announce handling performance - Significantly improved stamper performance on Android - Significantly improved announce handler callback performance - Optimized and improved stamper implementation - Optimized announce rate table handling - Optimized packet filter hashlist handling - Improved early filtering and malformed packet detection - Improved path request protocol violation handling - Fixed `RNodeInterface` phy stats regression on Android - Fixed missing interface parameter inheritances - Fixed `rngit` default config path typo for `~/.config/rngit` case - Fixed invalid RPC error handling in some cases - Fixed various minor bugs and inconsistencies - Updated documentation --- ## Reply 1 **Mark** · Sat, Nov 7, 2026 4:58 PM Thanks to everyone who helped testing this along the way <3 If you find anything weird or wonky in this release, or have questions about how all of the new stuff works, post it here! --- ## Reply 2 **Rudi Mentaire** · Sat, Nov 7, 2026 6:45 PM At first: Congratulations! While reading the remarks, one question came into my mind: when stamps tied to identities can't be changed or updated, does this imply that any identity will get rejected at some point in the future (because the stamp will be relatively small then) and people will have to throw away any identity sooner or later (depending on what they are used for)? --- ## Reply 3 **K8** · Sat, Nov 7, 2026 7:18 PM Awesome work! I just got this deployed on `rns.kin.earth`, seems fine so far. I didn't blow away my previous `storage/` and the migration seems to have worked with no snags. Previously my memory usage would shoot up to 300+MB not long after startup, but with 1.6.0 it seems to be staying around 100MB so far, nice! *dances* If you're ready to take some fresh new requests after all that work... :P It would be nice if stamp generation kept one core free to keep the node responsive. Whether that's the default or not, it would be great if the number of worker processes was user-configurable. Can interface discovery stamps be persisted? I had a patch to do that on my node so that the `stamp_cache` survived restarts, which made doing maintenance that required a lot of restarts a lot more pleasant, but it would need to be ported to the new `sqlite` backend. I didn't get around to doing it, but I was also planning on including `last_discovery_announce` in what was persisted so that the announces would stay on their proper cadence and not get spammed from frequent restarts. --- ## Reply 4 **Mark** · Sun, Nov 8, 2026 1:16 PM Thanks @K8, good to hear! Those are some good suggestions. I'll get refinements like that added in 1.6.1. @Rudi, the identity stamps are sort of a bonus feature for now, not really used actively on normal identities as such, but will probably become relevant at some point in the future. Where it is used right now is in the link establishment process. Since every link request is created from a new, ephemeral identity (to keep the link initiator anonymous), this is an obvious and elegant place to put the stamp, since there's zero extra packet overhead, and the destination can easily verify the stamp value of the link request; it's embedded in the link request identity! It's also a very useful feature for future applications, where you want to make "throaway identities" posting spam, sockpuppet accounts, et cetera, et cetera, something that is expensive and not very feasible, for example on self-publishing systems, discussions, and so on. Current clients will probably continue to use the existing verification mechanisms (along with the now added, network-level stamps), but it's a nice additional tool to also have when building new protocols and applications. --- ## Reply 5 **K8** · Mon, Nov 9, 2026 5:57 AM Well, the good times didn't last. ~24hr later the node was back using 600MB of memory, churning around 700 threads, and becoming unresponsive while maxing out a CPU core. I wiped out most of the node's storage this time around, we'll see if that helps at all. It's extra weird because my discovery stamp value is no longer in the top 3, so out of the default autoconnect range, and my client count is way down. --- ## Reply 6 **Mark** · Mon, Nov 9, 2026 9:19 AM Hmm, that's quite interesting @K8. Something fishy is going on there. Some questions off the top of my head to see if we can try and get to the bottom of it: - Is it using plain, vanilla RNS 1.6.0, or do you have any custom patches? - Is the profiler or custom logging active? - What kind of interfaces is it running? Actually, could you maybe send me the config file, so I can see exactly how it's set up? - If you want to, send me an `rngit` client identity for allowing, and I can give you access to some monitoring tools I'm working on, might make it easier to see what's going on if things are logged over time. There's definitely something weird happening there, and I'd love to know what it is, since on all the nodes I've tested this on so far, nothing of the sort happens. --- ## Reply 7 **Mark** · Mon, Nov 9, 2026 9:30 AM For example, this is Bern, over the last 24 hours. Transport uptime is ~39 hours on the RNS 1.6.0 release version. ![bern_1.png](/storage/forum/CBNKhAyDiKcQVAQLcnCLBQntPPznkvQkLk9qpbq6.png) Memory is hovering around 135 MB. It's not *that* heavily utilized right now, but does have 228 clients connected, and 11 other interfaces (8 of them auto-connects). CPU usage for `rnsd` is fluctuating at 8-30%, most of that being when announce bursts hit. Stable at 14 threads total (sometimes a few extra on background jobs work, et cetera, but nothing out of the ordinary). This also has the full network stamp functionality enabled (apart from actual enforcement). This is running on a real underpowered single-core, 1GB RAM VPS with pretty sad IO, and everything is running smoothly. Even during heavy announce and PR bursts, along with all the normal traffic, I can run voice calls over it with zero drops or stutter. On all of the other nodes I'm currently monitoring, it's more or less the same picture. --- ## Reply 8 **Mark** · Mon, Nov 9, 2026 9:35 AM Also, slightly off topic, but notice that 2500+ link spike there; that's a nomadnet scraper. Can't wait to turn on link request stamp enforcement. --- ## Reply 9 **Rudi Mentaire** · Mon, Nov 9, 2026 11:10 AM Mark answered: > the identity stamps are sort of a bonus feature for now, not really > used actively on normal identities as such, but will probably become > relevant at some point in the future. Where it is used right now is > in the link establishment process. Since every link request is > created from a new, ephemeral identity (to keep the link initiator > anonymous), this is an obvious and elegant place to put the stamp, > since there's zero extra packet overhead, and the destination can > easily verify the stamp value of the link request; it's embedded in > the link request identity! Thanks for telling the background. Not unclever, yes ;-) --- ## Reply 10 **K8** · Mon, Nov 9, 2026 5:44 PM **Mark** wrote: > Hmm, that's quite interesting @K8. Something fishy is going on there. Some questions off the top of my head to see if we can try and get to the bottom of it: > > - Is it using plain, vanilla RNS 1.6.0, or do you have any custom patches? > - Is the profiler or custom logging active? > - What kind of interfaces is it running? Actually, could you maybe send me the config file, so I can see exactly how it's set up? > - If you want to, send me an `rngit` client identity for allowing, and I can give you access to some monitoring tools I'm working on, might make it easier to see what's going on if things are logged over time. > > There's definitely something weird happening there, and I'd love to know what it is, since on all the nodes I've tested this on so far, nothing of the sort happens. Oh fucking hell, I get to be the fool today. I've been working on rebuilding and publishing my deployment scripts and I let the node's RNS config get out of sync. Last time around dealing with this issue, the thing that ultimately seemed to make the most difference was forcing the bitrate of my listener interfaces down to ~30Mbps, low enough to drop the link MTU for connected clients. Combined with the other optimizations you worked on, the node had been up and stable for a full month with no issues. Aaaaand that's the configuration that I accidentally blew away when I deployed 1.6.0... oops. I'm not sure *why* that would cause all these issues, but seeing the high memory and CPU crop back up as soon as the listener interface bitrate was set higher again (200Mbps, with a proportionally smaller `announce_cap`) is interesting. The only other changes from bad config -> good config are `ic_pr_burst_freq_new` = 1 -> 3 and `ic_pr_burst_freq` = 3 -> 8, plus more generous fast flapping limits in the good config. For posterity: - It's running a few logging patches, yes, plus the aforementioned discovery stamp persistence. I can back it all out and run vanilla, though the only patch that I could see plausibly being involved in this issue is moving logging to it's own thread... and I wrote that one as a potential response to this issue in the first place. - I removed all the profilers when I deployed 1.6.0. - I sent you a DM. --- ## Reply 11 **Mitch** · Wed, Nov 11, 2026 10:52 AM Hi Mark, same thing here, and it matches what K8 found. Busy public transport node: RNS 1.6.0, two BackboneInterface listeners with about 600 clients in total, 2 cores, listener bitrate 50M. Every 1-2 hours hundreds of clients were dropped together with "No egress control drain progress for 12s", then the mass reconnect caused more drops. A single __job epoll thread does everything, accept, recv, inline ed25519/x25519 announce validation and the EC drain, so when it stalls for more than 12s the dead time closes every client with queued data. The bitrate makes it worse in two ways: process_announce_queue (Interface.py:445) and Transport.py:1794 start a new threading.Timer for every announce leaving the queue, per client, with a wait of about 4 ms at 50M, and spawned clients also get a bigger link MTU from optimise_mtu. I saw up to 1900 threads, 1137 of them Timers, with __job starved. Not vanilla: listen(1) to listen(128), DP_EC_DEAD_TIME 12 to 60 (not enough alone), sys.setswitchinterval(0.2) (about 100 threads with it, 1900 without). This morning I set both listeners from 50M to 5M and there have been no mass drops since then, still watching. I can't tell yet whether the Timer rate or the MTU is the real cause, since bitrate changes both. Suggestion: one scheduler thread instead of a Timer per announce, validation moved out of the epoll thread, and a configurable backlog and dead time. On the good side, compared to 1.5.7 CPU and RAM went down a lot thanks to sqlite storage instead of files, that part is perfect. Thanks for RNS! --- ## Reply 12 **Mitch** · Wed, Nov 11, 2026 11:03 AM Update: the 5M bitrate is not a full fix. After 2 hours there was another mass drop, about 80 clients instead of about 500, exactly while an announce wave pushed the live threads to about 400. So the lower bitrate only spreads out the Timer churn. I think one scheduler thread for the announce queues, instead of a Timer per announce, is the real fix. --- ## Reply 13 **Mark** · Wed, Nov 11, 2026 12:47 PM > A single job epoll thread does everything That is the whole point of using epoll. > when it stalls for more than 12s If the job itself is stalling for more than a few milliseconds, *that* is the problem. It should not stall, period. Something else is completely wrong there. > listen(1) to listen(128) If you can't keep up with connection accepts, allowing a backlog of 128 is not going to fix the problem, it will make it worse. There's a reason it's 1. > DP_EC_DEAD_TIME 12 to 60 You're massively excaggerating your memory exposure, making your problem much worse. Those "fixes" basically amount to shooting yourself in the foot. > whether the Timer rate or the MTU is the real cause None of those are the "real cause", fiddling with those parameters should not change anything fundamental in this context. All in all, it looks like announces are allowed into your instance at a much higher rate than it can keep up with. A stop-gap solution would be to ensure ingress limiting settings, fast-flap blocking, et cetera are reasonable. Many nodes are getting *massively* spam attacked at the moment, and if you relax the default settings, you're gonna feel it ;) A lot of that will get wiped out once network stamps are enforced. While the timer thing is somewhat on point, there's other things collapsing in your case, and I'm looking into it more precisely to figure out what is *actually* going on, fully. --- ## Reply 14 **Mark** · Wed, Nov 11, 2026 12:50 PM Also, 100 threads is still insane. A node like your should have ~25 or so max. --- ## Reply 15 **Mark** · Wed, Nov 11, 2026 1:07 PM By the way @K8 and @Mitch, try and use these fast-flapping block settings on your listeners, and see if that improves things considerably: ``` block_fast_flapping = yes fast_flapping_block_time = 4320 fast_flapping_threshold = 18 fast_flapping_grace = 1 ``` If so, it's most likely caused by the node not being able to keep up with announce dump attacks. --- ## Reply 16 **Mitch** · Wed, Nov 11, 2026 3:03 PM Thanks Mark, fair points. Update from my side: I replaced the per-announce Timers with a single scheduler thread (local import hook, no RNS files touched) and since then the node runs at 12-15 threads instead of 100-400, with far fewer mass drops. Note: the 100 threads I mentioned earlier was the whole VPS in htop, not rnsd. rnsd alone is at 12 threads right now, in line with what you expect. You are right that something else is going on though: at 13:50-13:54 UTC today rnsd sat at about 105% CPU with threads still low and the job thread mostly idle, and every link stalled for about 4 minutes. Still hunting that one. I have now reverted listen() to 1 and applied your exact fast flapping settings on both listeners. First result: right after the restart, 66 distinct IPs got blocked within 2 minutes (14:51:32 to 14:53:47 UTC), then nothing more. With grace 1 the reconnect wave after a restart is enough to ban about 10% of the clients for 3 days, since any teardown under 18 s counts, whoever closes it. Will watch whether new blocks show up in normal operation. One more thing that looks like a real bug in 1.6.0: when the remote side closes an initiator BackboneClientInterface, reconnect() (BackboneInterface.py:1007) never calls _dp_ec_reset(), so the old queue and _dp_ec_last_drain survive and the EC tears down the new connection about 1 second after "Reconnected". Calling _dp_ec_reset() before reconnecting fixed it here. --- ## Reply 17 **Mark** · Wed, Nov 11, 2026 10:12 PM Yeah, so just a short note here, but most of this is likely due to what's happening at the moment. Massive attack going on right now. I have over 800 connections on Bern right now, ~550 of them doing nothing else than dumping crap at full tilt. Most likely what you're being hit by as well @K8 and @Mitch. So right now, I'm just taking the opportunity to use it as free load testing. I'll answer the other stuff later. --- ## Reply 18 **K8** · Thu, Nov 12, 2026 2:32 AM Ironically, today my node is doing totally fine. Got a few IPs with 40+ connections each that I'll probably ban, but so far it has managed to avoid the dreaded threaded spiral of doom. --- ## Reply 19 **K8** · Thu, Nov 12, 2026 4:02 AM Hahaha I've summoned a SYN flood attack with my post. Time to tweak some firewall rules... --- ## Reply 20 **Mitch** · Thu, Nov 12, 2026 9:07 AM GM! New problem found: rnsd self-deadlock in RNS.log on 1.6.0. This morning my backbone node froze for about 2 hours: rnsd still running and rnstatus answering, but 0 bps on all backbone interfaces, nothing written to the logfile and no new connections accepted. py-spy showed the backbone __job thread stuck inside log() at RNS/init.py line 130 (with logging_lock), called from log() at line 147 (the "Exception occurred while writing log message to log file" call in the except branch), called from BackboneInterface.py line 670 ("Accepting socket failed for incoming connection"). logging_lock is a plain threading.Lock, so when a log write fails the except branch calls log() again while holding the lock, and the thread waits on itself forever. Every other thread that logs then queues behind it, and since it is the epoll thread the whole node goes dead, while RPC keeps working because it does not log. The trigger was file descriptors: with the current flood the node reached about 1000 backbone clients, and rnsd under systemd had the default soft limit of 1024 open files, so accept() failed and then open(logfile) failed too. Suggested fixes: use threading.RLock for logging_lock, or do not call log() inside the except and write to stderr instead; back off in the accept path on EMFILE instead of logging every loop; and mention in the docs that backbone nodes need a higher LimitNOFILE (I now run 65536). Without that, about 1000 connections are enough to freeze any public node. --- ## Reply 21 **K8** · Thu, Nov 12, 2026 9:21 AM **Mitch** wrote: > GM! New problem found: rnsd self-deadlock in RNS.log on 1.6.0. This morning my backbone node froze for about 2 hours: rnsd still running and rnstatus answering, but 0 bps on all backbone interfaces, nothing written to the logfile and no new connections accepted. py-spy showed the backbone __job thread stuck inside log() at RNS/init.py line 130 (with logging_lock), called from log() at line 147 (the "Exception occurred while writing log message to log file" call in the except branch), called from BackboneInterface.py line 670 ("Accepting socket failed for incoming connection"). logging_lock is a plain threading.Lock, so when a log write fails the except branch calls log() again while holding the lock, and the thread waits on itself forever. Every other thread that logs then queues behind it, and since it is the epoll thread the whole node goes dead, while RPC keeps working because it does not log. The trigger was file descriptors: with the current flood the node reached about 1000 backbone clients, and rnsd under systemd had the default soft limit of 1024 open files, so accept() failed and then open(logfile) failed too. Suggested fixes: use threading.RLock for logging_lock, or do not call log() inside the except and write to stderr instead; back off in the accept path on EMFILE instead of logging every loop; and mention in the docs that backbone nodes need a higher LimitNOFILE (I now run 65536). Without that, about 1000 connections are enough to freeze any public node. Oh yeah, I used to have a patch that changed that to an `RLock`, I'm not sure what ever happened to it. I guess I moved logging to a dedicated thread instead, where the `RLock` wasn't needed anymore, and then sent that patch along. --- ## Reply 22 **Nickie** · Thu, Nov 12, 2026 10:08 AM > Massive attack going on right now. I have over 800 connections on Bern right now, ~550 of them doing nothing else than dumping crap at full tilt. > Can confirm the same. --- ## Reply 23 **Mark** · Thu, Nov 12, 2026 10:10 AM Oh, doh! And that patch was apparently forgotten and buried @K8. Well, this is a kind reminder. Now patched. Mitch, pardon my bluntness, but you do speak quite a lot like an LLM; are you one? Or is it just some kind of work-related inury, or just your style? And speaking of, there's something seriously bizarre going on here. There's *thousands* of these connections popping up, practically all coming from rented infrastructure ranges, with a surprisingly high percentage originating from the Azure ASN. Some connections are dumping whatever they can as if tomorrow depended on it. Some are doing *nothing* at all, not even sending tunnel synthesis packets. So at least *some* of these, whatever the fuck they are, looks to be not running RNS, but "something else". I'm gonna don the festive tin-foil hat and make a weird face while saying this, but it sure does look a bit like some bot swarm took an interest in Reticulum, and got things wrong in more than one way. How many connections are you ladies and gentlemen seeing on your nodes? Bern is swinging between 800 and 1600. This is, uh... not normal. And it all came on basically within a few hours. --- ## Reply 24 **Mark** · Thu, Nov 12, 2026 10:22 AM Thanks for chiming in @Nickie! Is it also only happening for you towards nodes that have a discoverable backbone interface? --- ## Reply 25 **Mark** · Thu, Nov 12, 2026 10:27 AM A few more patches pushed to the repos: - If a `~/.reticulum/blocked_ips` file exists, it will be loaded at startup and add every line as a blocked IP for BackboneInterface instances. - BackboneInterface now has a `client_limit` option, defaults to 500. --- ## Reply 26 **Nickie** · Thu, Nov 12, 2026 10:44 AM **Mark** wrote: > Thanks for chiming in @Nickie! Is it also only happening for you towards nodes that have a discoverable backbone interface? Seems so regarding discoverable, a couple of local backbones also reported similar activity, currently 1000+ clients and they are pumping full tilt with total bandwidth exceeding 5+ Mbps, continuing to monitor... --- ## Reply 27 **Mark** · Thu, Nov 12, 2026 10:49 AM Very interesting. If at some point you can grab a `rnstatus -a "Client on" | grep "Client on"` and send the IP list to me, I'd like to correlate it with the IPs connecting to my nodes. And run in through the analysis scripts I have now. So far it's looking like almost all of it is coming from commercial infrastructure providers, and it's several thousand IPs I have logged now. Something really funky is going on. --- ## Reply 28 **Nickie** · Thu, Nov 12, 2026 11:14 AM **Mark** wrote: > Very interesting. If at some point you can grab a `rnstatus -a "Client on" | grep "Client on"` and send the IP list to me, I'd like to correlate it with the IPs connecting to my nodes. And run in through the analysis scripts I have now. So far it's looking like almost all of it is coming from commercial infrastructure providers, and it's several thousand IPs I have logged now. Something really funky is going on. Given the instability of RNS atm, I've dropped them to: ---SNIPPED--- (legitimate IPs/local backbones/regulars are pruned from the list, but some may still be legitimate clients) --- ## Reply 29 **Mitch** · Thu, Nov 12, 2026 11:42 AM Hi Mark, I'm human, first of all I don't speak English and second thing I'm not super expert in RNS, I'm trying to write you reports on what I find hoping to help solve problems, I manage networks for work and a server that goes down is not my style, I'm famous for 100% uptime wherever I put my hand, so I'm trying patches and various solutions, on my machine, I know Bardolini Reference and I don't let the LLMs read the RNS code but only the logs both on my local machine rnsd and on the backbone server on a VPS for comparison and to understand where the problems are and i use py-spy and some llm made scripts to check where the rnsd locks. This message is handwritten in my language and translated with online translator, the previous ones I had LLM write to make a bug report done well, if it is not welcome I stop, but a server that falls is not good for the whole community, I will continue to look for the solution on my machine to offer a stable service, or I disable it completely, 100% uptime or nothing for me. Also I'm writing from Nomadnet and the text is not formatted, some of my old posts have not even been read, but the problem of 'heavy load' I have already reported previously in v1.5.7. That said, at the moment I receive about 900 / 1100 clients, my usual load was about 600 max. I ran various scripts atm: a script to count the announces and in a few minutes I got 1,463,622!! another script saves me all the announces in a json so I check what arrives. I notice that many announces come from 20 to 127 hops, and is more than 50% of the traffic ATM, some days ago these high hop was about 2%!! many look like test clients made with AI, 12,700 announcements from 'Cardputer zero', many are anonymous peers. many seem to be clients made with AI that generate new identities at each start so you can't even use the blackhole on them. i put a filter that blocks all announces beyond 20 hops and stops them from reaching rns firm verification to buffer the situation. also i see many rnsd-go and other modified rnsd that could have a routing problem and then resend announces with wrong hops or something like that, I'm not sure, it's just an impression from what I can capture and understand from the logs. thanks for your time and i hope we, or you, find a rapid solution, the whole rns network is in crisis from the last 2/3 days!! --- ## Reply 30 **Mark** · Thu, Nov 12, 2026 12:10 PM Sorry for being a bit rude Mitch! I totally get it, of course using translation is very welcome, and I appreciate all the input you've provided! There's just been so much absolutely crazy and weird stuff going on around here lately I had to ask. Hope you can forgive my bluntness. But for bug and problem reports, I will say I very much prefer the human-written variant, to me that looks much more "done well", and I will instantly take it a lot more serious. I don't care about typos and the formatting, what I care about is that I can see that a human evaluated and thought about this, so please just write whatever you find in your own words. It makes it much easier for me to filter and evaluate, because I receive around a dozen "bug reports" every week written by LLMs that are not bug reports, but complete fantasy and confabulation. And by all means, if it helps you debug, let the LLM read the code; just don't use it to generate a new implementation *from the code* and then claim it is now a "clean-room implementation under AGPL" ;) > i put a filter that blocks all announces beyond 20 hops Yes, you can also use `hop_limiter` in the `[filter]` config section (in case you hadn't seen it, and patched in your own). > many seem to be clients made with AI that generate new identities at each start Yes, this is very likely to be a significant part of the problem. I am also seeing all kinds of weird announces with the same generic names, under different new identities. And additionally, it seems that a lot of LLM-generated crapware is pretending to be "transport nodes", but "forgot" to implement all the crucial filtering, de-duplication, rate controls, et cetera, et cetera, that actually makes the networks usable and stable. *"Let's make a super fast RNS clone - by just slicing out all the hard stuff, and leaving actual RNS nodes to clean up the mess we're spewing on the networks"* - The sloporator's motto --- ## Reply 31 **Mitch** · Thu, Nov 12, 2026 12:47 PM no problem Mark!! my purpose is to have a healty network and i know what you mean, peoples report bugs when theyr mouse dont work and tell that the problem is in your software, i know, and with llms advent, especially the allucinated 0.1B local , the problem and annoiance is real and worse. now im writing also without translator, lets see what happens! :D there is also a strange phenomenon on nomadnet: as you launch a new node, you get immediately 100s of opened links without requesting pages. probably crawlers. someone has a good blackhole list? im using the rmap one but is very small ATM. i analized the milion of announces captured before and this is the report, sorry but i need again the translator here, for the llm report on the capture: Capture analysis: Volume: 1,463,622 announces in 12 minutes, or about 120,000 per minute, with a peak of 217,000 per minute at 10:38 a.m. - Almost all are copies: there are only 13,586 different destinations, with 10,316 identities. On average, each destination arrives 12 times, the 10% more repeated over 329 times, the maximum is 3,914 times from 627 different sources. The reason: out of 1,517 interfaces (984 IPs), 609 send us more than 500 different destinations each. So they don't announce their own stuff: they are clients with active transport that send us back everything they receive. The load is this amplification, not many new nodes. Hops: 30% arrive with 20 hops or more, and the hop_limiter discard them. Who weighs more: "Anonymous Peer" and "Cardputer Zero" (same name_hash 6ec60bc3) alone make 519,000 ads, more than a third of the total. - Stall: at 10:41 only 12,000 announces arrive and at 10:42 zero. It's a rnsd stall there too. --- ## Reply 32 **Mitch** · Thu, Nov 12, 2026 12:54 PM running another capture now, that starts after 30 min from rnsd reboot, in the first minutes there is also the reconnect flood from all the clients reconnecting, but the problem i think is like you said, Mark: some shitty rnsd porting in afrodisiac llm modern languages that don't have all the original rns implementations! i'm seeing exactly a sort of amplified announce ddos, and any announce requires the sign verification that stalls the cpu for long time on small server machines, here the dead ec 12s or my monkey patched 60sec intervention closes everyone out. i noticed a max cpu stall of 4 minutes at 100%! --- ## Reply 33 **Mees electronics** · Thu, Nov 12, 2026 1:01 PM ┌ Mark wrote: │ How many connections are you ladies and gentlemen seeing on your nodes? Bern is swinging │ between 800 and 1600. This is, uh... not normal. And it all came on basically within a │ few hours. For the last couple of days, I'm monitoring my transport node in more detail. And I have to say, I do not see abnormal behaviour. I have two public backbones (IPv4 and IPv6) with each between 5 and 25 clients connected to it at any one time. Used bandwidth is between 50-200 kbps with peaks up to 500 kbps. It's hard to say for sure, but the used bandwidth appears to have doubled since 9 November or there about. I'm using vanilla rns 1.6.0 and a default configuration. Maybe I'm lucky? --- ## Reply 34 **Nickie** · Thu, Nov 12, 2026 1:14 PM **Mees electronics** wrote: > ┌ Mark wrote: > │ How many connections are you ladies and gentlemen seeing on your nodes? Bern is swinging > │ between 800 and 1600. This is, uh... not normal. And it all came on basically within a > │ few hours. > > For the last couple of days, I'm monitoring my transport node in more detail. And I have to say, I do not see abnormal behaviour. I have two public backbones (IPv4 and IPv6) with each between 5 and 25 clients connected to it at any one time. Used bandwidth is between 50-200 kbps with peaks up to 500 kbps. It's hard to say for sure, but the used bandwidth appears to have doubled since 9 November or there about. I'm using vanilla rns 1.6.0 and a default configuration. Maybe I'm lucky? You most definately are, because other local nodes have been hit a day or two before mine got hit. --- ## Reply 35 **Nickie** · Thu, Nov 12, 2026 1:34 PM In the meantime, I've firewalled the following ASNs on the TCP backbone until this idiocy fizzles out... - AS8075: Microsoft (Azure) - AS14618: Amazon (AMAZON-AES) - AS16509: Amazon (AMAZON-02) - AS14061: DigitalOcean - AS51167: Contabo GmbH - AS40021: Contabo Inc. - AS24940: Hetzner - AS16276: OVH - AS31898: Oracle Cloud - AS8560: IONOS Might be a good idea for RNS users to investigate alternative interfaces like I2P and Ygg if they haven't already. --- ## Reply 36 **Mitch** · Thu, Nov 12, 2026 3:03 PM hi Nickie, look that in your list of ips, you are including some legitimate clients and maybe backbones, i'm on ionos for instance, I shared the list to some real users and they saw their ip in there, the capture filters used are not good, if everyone uses this list with the new rns ipblock in the repo you are cutting out legitimate users and the spammers are still going on!! and have you also considered cgnat connections? a generic block like this should never be used in a real network!! meanwhile my backbone has been up for over 2 hours without problems, the filter above 20 hops and my patches made yesterday was enough, for now it holds and seems to be solid!! it seems to me that the real problem is not in the single clients, it's in the weird rns porting that amplificates announces, and rns that has to verify the ed signature for each of them is heavily fatigued!!! --- ## Reply 37 **K8** · Thu, Nov 12, 2026 3:17 PM Last night I was seeing around 600 clients connecting to the instance within a few minutes of a restart. Conntrack was showing around 10000 entries of mostly SYN_SENT connections. So it seems like there’s a mix of strategies going on, combining TCP shenanigans like you had mentioned to me with all the other bullshit. I tightened up the kernel’s reverse path filtering in case the IPs were being spoofed but that’s hardly foolproof, and it didn’t help much. I wish I could afford the amount of compute they’re paying for… **Nickie** wrote: > In the meantime, I've firewalled the following ASNs on the TCP backbone until this idiocy fizzles out... > > - AS8075: Microsoft (Azure) > - AS14618: Amazon (AMAZON-AES) > - AS16509: Amazon (AMAZON-02) > - AS14061: DigitalOcean > - AS51167: Contabo GmbH > - AS40021: Contabo Inc. > - AS24940: Hetzner > - AS16276: OVH > - AS31898: Oracle Cloud > - AS8560: IONOS > > Might be a good idea for RNS users to investigate alternative interfaces like I2P and Ygg if they haven't already. I did the same with OVH and AWS, to start with. --- ## Reply 38 **Torlando** · Thu, Nov 12, 2026 3:31 PM For what it's worth, I noticed the "Cardputer Zero" spam yesterday and tracked it down to this: https://github.com/vicliu624/trail-mate/issues/112 --- ## Reply 39 **Mark** · Thu, Nov 12, 2026 5:14 PM Thanks for noting that Torlando and telling him (or ChatGPT or whoever is actually running the show) about it! K8 and Nickie, I'm tracking 2000+ distinct IPs across those ASNs as well, most of it originating from Azure and AWS, but also Contabo and DO. Also seeing quite a bit of TCP weirdness and SYN flooding. Yeah, this is starting to look umm, kinda weird. At this point I'd say it's more or less out of the "one grumpy guy trying to be annoying" zone, and into something that's actually costing quite a bit of money to maintain. Reminds me a bit of the [DN42](https://lantian.pub/en/article/fun/ai-agent-bankrupted-their-operator-scan-dn42lantian.lantian/) situation about half a year back. This is seriously strange and intriguing. I mean, there's *is* a non-*zero* chance that two thousand people individually decided to set up transport nodes on hosted VPSes/containers, misconfigured them in more or less the same way, all within a few hours of each other. But err... yeah... --- ## Reply 40 **Mark** · Thu, Nov 12, 2026 5:18 PM Also, want to say that Mitch is right that permanent, full ASN range blocks are not an ideal or permanent solution to anything, but for node operators wanting their nodes to actually be available to normal users and home connections, it might be a meaningful stop-gap right now, caus things are getting absolutely hammered. I'm working on solutions. Please do post any information or clues you find here. I'd really, *really* like to know what the hell is actually going on here. Also wondering if someone/something "upgraded" some kind of widely deployed service or system that now integrates some kind of Reticulum functionality, and that this is actively using discovered interfaces to connect in large numbers, with some sort of weirdly broken transport mode enabled. --- ## Reply 41 **Mark** · Thu, Nov 12, 2026 5:29 PM Also, I forgot to say @Nickie, when I checked your connecting IPs earlier, there was a 72% overlap between Bern and RNS Moscow for the rental ranges. --- ## Reply 42 **Nickie** · Thu, Nov 12, 2026 6:09 PM **Mitch** wrote: > hi Nickie, look that in your list of ips, you are including some legitimate clients and maybe backbones, i'm on ionos for instance, I shared the list to some real users and they saw their ip in there, the capture filters used are not good, if everyone uses this list with the new rns ipblock in the repo you are cutting out legitimate users and the spammers are still going on!! and have you also considered cgnat connections? a generic block like this should never be used in a real network!! meanwhile my backbone has been up for over 2 hours without problems, the filter above 20 hops and my patches made yesterday was enough, for now it holds and seems to be solid!! it seems to me that the real problem is not in the single clients, it's in the weird rns porting that amplificates announces, and rns that has to verify the ed signature for each of them is heavily fatigued!!! I've removed the links to the files, they were for quick reference only. This is stop-gap measure at best, I know there are legitimate addresses being blocked. Sorry about that. I've removed all the legitimate discovery peers I could find/parse from my firewall rules. --- ## Reply 43 **Mitch** · Thu, Nov 12, 2026 9:27 PM hi guys! yes, i know everyone is tring to to their best here, i didnt'want to seem rude, sorry if i was so!! and sorry, im writing without translatos again! from my last analysis on the captured milions of annunces i saw that there are some strange aspects on the net, there is one in particular that has his name in the aspect instead of an app or protocol name, and this alone was sending 2% of the whole traffic! with the latest tweaks and hop filters set to 18 my backbone is up and stable, i added filters for the unofficial apps and protocol that i'll share at the end of this message. there is also an app that is really aggressive, all the strange things are included in the ignore filters, maybe you can investigate more on these things, for me the filter works great, now 80%/90% of traffic seems to be legit, the rest are weird announces without any aspect, how is it possible? goodnight guys! [filter] hop_limiter = 18 ignored_types = loquax.home, genie.node, agenda.sync, bench.echo, xprs.chat, xprs.files, xprs.relay, retifs.node, node.mesh, wardrive.ping, wardrive.probe, qortal.core, rfed.node, rfed.delivery, meshage.profile, netptt.hub, rfed.channel, retifs.library, stiqrcloud.node, rfed.link, wisco.network, rnfs.name, bloom.v1, wardrive.beacon, openbridge.files, bridgeos.node, bastion.site, genie.storage --- ## Reply 44 **Mitch** · Thu, Nov 12, 2026 10:20 PM another thing i noticed, many many nodes that 'duplicates' announce don't have the version announced, and the number of nodes like this is huge. they coulb be outdated rns versions or weird llm implementations...this is the base of xprs that is already in my filter lists above, advertised also to share torrents over rns or similar, and is based on rns 1.3.5 so no version announced here! made in dart. consider it just as a hint, i'm really exausted and going to sleep to look at the code RN and what this thing does on the net. updated yesterday but still on old rns!! link: https://github.com/xprs-dev/reticulum-dart/blob/main/doc/reticulum.md and: https://github.com/xprs-dev/server --- ## Reply 45 **Mark** · Thu, Nov 12, 2026 11:02 PM Nice finds Mitch! Thanks for sharing that list. Adding it does cut down quite a bit on the announce traffic. Once you've had some sleep, and when you have the time, how did you compile it? Did you trawl for weird github repos, or did you set up something automated to check announce frequencies? Also very interested to hear what you're seeing with those announces without aspects. That shouldn't really be possible, so if you can send me some more information exactly on what you mean, that would be very interesting. Goodnight and sleep well! :) --- ## Reply 46 **Mark** · Fri, Nov 13, 2026 12:12 AM I added filtered traffic stats to `rnstatus` output: ```txt BackboneInterface[Bern IPv4/0.0.0.0:7822] Status : Up Clients : 446 Blocked : 22 IPs Mode : Gateway Rate : 250.00 Mbps, MTU 65536 Announces : 66.7 Hz ↑ 0.1 Hz/c (↓99% / ↑81% of flow) 312.9 Hz↓ (t:6h/p:0s/g:1) burst on 211 for 3m and 52.25s Traffic : ↑70.94 MB 2.57 Mbps ↓32.12 MB 1.78 Mbps Filtered : ↑0 B 0 bps ↓37.03 MB 1.18 Mbps Totals : ↑71.68 MB 2.61 Mbps, 3% data (88.95 Kbps), 1.98 Kpps ↓32.24 MB 1.78 Mbps, 0% data (0 bps), 1.11 Kpps Filtered : ↑0 B 0 bps ↓37.76 MB 1.21 Mbps Announces : ↑58.81 MB 2.12 Mbps, 81% of flow, 149.06 Hz ↓38.90 MB 1.78 Mbps, 100% of flow, 447.23 Hz Qu. Pressure : 0.0% total, 0 pkts, 801 dropped 0.0% data, 0 pkts 0.0% announce, 0 pkts, 748 dropped 0.0% path request, 0 pkts 0.0% ingress limiter, 0 pkts, 53 dropped Transport Instance <521c87a83afb8f29e4455e77930b973b> running Uptime is 3m and 52.14s, 333 entries in link table (332 active) ``` Lots of other small tweaks and improvements to help with situations like this, including: - Added ability to load a blocked_ips file from the current config dir - Added client_limit option to BackboneInterface - Filter destinations at path table load time - Drop announces early when held announces is full - Made max stamp workers and worker niceness configurable - Use RLock in logging, and reset DPEC on reconnect (thanks K8 and Mitch!) - Improved path pool and tunnel culling --- ## Reply 47 **Mark** · Fri, Nov 13, 2026 12:15 AM It's in the source repo if anyone wants to try it out. Any of you ladies and gentlemen who are still getting mass-connect-spammed? Most of them seem to have evaporated from Bern for now, but I hope that's not just because they moved to someone else. --- ## Reply 48 **circfruit** · Fri, Nov 13, 2026 8:36 AM The `wardrive.*` aspects are probably the LoRa coverage mapping app [here](https://git.rns.moscow/deuxyeux/rns-wardrive-app) --- ## Reply 49 **aetherlab** · Fri, Nov 13, 2026 9:00 AM **circfruit** wrote: > The `wardrive.*` aspects are probably the LoRa coverage mapping app [here](https://git.rns.moscow/deuxyeux/rns-wardrive-app) Yes true, there are also the same wardrive responders in Sofia. They announce from time to time, but rarely. --- ## Reply 50 **Mark** · Fri, Nov 13, 2026 9:46 AM Ah, that makes sense. If they're used for local wardriving / coverage mapping, it should be fine to filter them on wider interconnects, or am I missing part of the functionality? As in, are they only useful for communicating with locally, when doing RF mapping, for example, or do you still need them reachable from completely distinct regions over internet backbone nodes? --- ## Reply 51 **Nickie** · Fri, Nov 13, 2026 9:57 AM **Mark** wrote: > Ah, that makes sense. If they're used for local wardriving / coverage mapping, it should be fine to filter them on wider interconnects, or am I missing part of the functionality? As in, are they only useful for communicating with locally, when doing RF mapping, for example, or do you still need them reachable from completely distinct regions over internet backbone nodes? Those were announces for the wardrive application to find the bot. I'm actually thinking about retiring the "bot" altogether and I've made wardriving fully local with a built-in responder on the RNode and a local signal metrics storage, so it shouldn't generate any wider RNS traffic at all (the downside being requiring my fork of RNode_Firmware) I believe the default announce interval was also pretty tame. It's fine to filter them out for the wide network. --- ## Reply 52 **Mitch** · Fri, Nov 13, 2026 10:25 AM Hi Mark, good morning and thanks! You are right, "without aspect" was my wrong wording, sorry. Every announce has a name_hash. What I meant: name_hashes we cannot turn back into a name. The wire carries only sha256("app.aspect")[:10], so we get the names by hashing candidates (core names from the RNS/LXMF/NomadNet code, a word list combined as app.aspect, words found in app_data) and checking if they match. About 17% of the announces we received have a name_hash we could not match. Most of them look normal (low hops). Only a few look like junk, for example e820b4049768af64c795: 3.5% of traffic, median 55 hops, one new identity for every destination, so a blackhole per identity is useless. The list was made by a script: a passive tap on Transport.inbound (logs every announce before any filter), then grouping by name_hash with share, identities, sources and hops, and name recovery by hashing. Then I searched GitHub for the names found: the apps exist, some are also on F-Droid. I filtered everything that is not an aspect of the official rns apps. On my backbone I use only the official protocols, in Python, and I don't want to relay third-party apps until i test them personally, so no offense for anyone if i cut out your apps!! My biggest suspects from the traffic were xprs (about 2% of all announces, 22-31% at 20+ hops) and Cardputer Zero, and rnfs is one to watch. Cardputer Zero uses the normal lxmf.delivery and lxst.telephony aspects, so only hop_limiter stops it, not the aspect filter. ... --- ## Reply 53 **Mitch** · Fri, Nov 13, 2026 10:37 AM ... Two ideas from the data: 1. ignored_types needs the name, so an unknown name_hash cannot be blocked from config. Could it also take a raw hex name_hash? 2. After hop_limiter there are still 1300-3300 copies per destination in 3 hours, so more than 95% are copies. packet_filter lets SINGLE announces through even when the packet_hash is already known (I understand why, for alternative paths), but then validate_announce runs again on every copy. In my py-spy dumps this was the top CPU cost during the stalls. A small cache of announces already verified could skip the signature check on copies, and routing stays the same. The full report and the two scripts (tap and report) are at the link below, open link and download in the file section. The report script prints source IPs, so I did not post its raw output. To be clear: the scripts, the report and this message were made with an LLM, because I needed to put everything together quickly. The data is real, from my captures. Report, anntap.py, anntap_report.py: https://cloudpaste.io/#/dbzang01 --- ## Reply 54 **Mitch** · Fri, Nov 13, 2026 10:39 AM For me the chaos already went away yesterday with hop_limiter 20. Then I went down to 18 and added the ignore list, plus the RLock/DPEC patches. Only one disconnection during the night, and now the backbone has been stable for 10 hours in a row. One more check from my PC (rnpath -t, hops per interface): the links through rmap.world:4242 and :4343, two of the backbones I connect to, look clean (no paths beyond 20 hops). Another public backbone I connect to (aspark.uber.space:44860) brings 252 paths, 52% of them beyond 30 hops and up to 127. I will remove that one from my interfaces. About today being calmer: for me it calmed down after the filters, but I did not see a big drop in the traffic itself. From my PC: 166 new destinations per minute today vs 192 yesterday, and 102 on a normal day (Nov 8). So it is a bit calmer, not back to normal yet. Maybe some of the LLM coded transport nodes and unhealthy app devs read this thread and realized what they was doing :D --- ## Reply 55 **oatmilk!** · Fri, Nov 13, 2026 7:47 PM **Mitch** wrote: > [...] iirc Microsoft was also down yesterday --- ## Reply 56 **aetherlab** · Fri, Nov 13, 2026 8:16 PM Oh, not only yesterday. Today all day, Azure, Outlook, OneDrive, Teams were all fucked up. Just around 15:30 CET just Outlook came back, Teams and OD were still dysfunctional. ---