rns.recipes

Over the last three days, Aleph has been hit with over 200,000 page and download requests by the most sloppy piece of junk I have seen for a long time.

It doesn't even fucking re-use links. Every page download happens over a new link. Who the well "wrote" that? A fucking moron with an LLM, that's who. If it wasn't because this technology existed, I would have taken it as a DoS attempt, which in all practical sense, it is.

But today, even the most dim-witted idiot who'd like to imagine and play make-believe developer can slap together a piece of junk like this, without any thought about what they're actually doing, and still manage to feel they have "created" something, while in reality exhausting other peoples precious bandwidth resources (and their friggin' patience!).

If you want to block this kind of nonsense, you can add the null_ident hash to blocked_identities in your rngit config:

blocked_identities = d7db22f63b453c23bb0688dde565b7c1

Now, can someone use an LLM for something worthwile for once, and figure out who this fucker is? I don't have the time, compute, patience or temperament to do it, caus if I find that porridge-mind, lord knows I'll slap them with dried codfish and a wet umbrella till they repent their ways.

If you're short of a weekend project, download lc, and task it to search github for some crayon who "wrote" a "search engine" or something like that.

For the love of all that is sensible, this is getting tiresome.

Rant over

FOR NOW

Just checked my rngit instances stats, yep, same pattern. 683 Nomad downloads today. 0 fetches. Recursively downloads the entire repo dozens of times over through Nomad.

The reason I don't believe it's an attempted DoS is because this is the only repo hosted in my rngit that has this behavior. It's some type of automated crawling behavior that finds Nomad destinations from other pages and just recursively goes and downloads whatever is there.

My guess is it is any number of these vibecoded '''Nomad Search Engines'''
https://roogle.us/

https://roogle.us/dataset

Dataset

What Roogle has mapped of the Reticulum mesh - pages indexed, nodes seen, which of them carry the most weight, and what the crawler is doing right now.
Snapshot taken 4 hours ago · refreshed automatically every 15 minutes
30.8K
Text pages
5.1K
Nodes
389.6K
Distinct words
4.3M
Index entries
210.5K
Links
30.8K
Crawled
0
Queued
33.2K

Actually, yeah, that's it. Lol. I had them on my radar for awhile. You can see my rngit instance ouroboros git on here, for some reason.

This is crawling EVERYTHING ever linked with their vibecoded toy in the most inefficient, wasteful way possible

These figures are recomputed from the live crawl index every 15 minutes

Okay, I decided to turn this into a bit of an expedition. I temporarily turned on Active Countermeasures on rngit on Aleph, by blocking the null ident, so unauthenticated page node requests are blocked, and then rewiring the normal "You can't access this page, yadayada" response to a 32 megabyte garbage dump per page. Since normal visitors can't get to those pages anyway, being blocked at the frontpage already, with a nice notice to identify, only the scraper is hit.

So far, it's going swell:

[Notice] Served 5.01 GB so far

This is great fun. Since RNS compresses resources, each response is only like 1200 bytes or so lol, decompressed to the full 32MB at the idiots computer. He's currently downloading an eternal stream of tildes.

I'll read your messages now Zenith, haven't yet, just had to share the fun first.

Okay, what the actual fuck? Did they eat nails for breakfast? How stupid can people be? Yeah, let's crawl a low-bandwidth mesh that people are using over radio links EVERY 15 MINUTES, for an idiotic service THAT NOBODY USES.

They are literally abusing and causing damage to the entire network. I'm gonna continue filling up their drives with tildes. Hopefully that will send a message. Let me see if I can accelerate it a bit...

Hmm, a little better. My effective dump rate is around 500 megabytes per minute right now. Not sure if it's their end choking, caus the requests are slowing down when I raise the dump size.

The little RasPi is at around 18% CPU load serving this, but my /tmp mount is so small that I can't push a lot of concurrent large transfers. Considering moving this to a fast laptop and just handing them a gigabyte at a time.

I cannot conclusively say it's just """Roogle""". But they do seem to be aggressively indexing stuff recursively with no page height limit. Take these nodes they index for example.

Something like a .zim host or a Retipedia instance is going to have thousands of linked pages due to self-links.

2400 nodes indexed in just an hour.

The same guy who made this is the same genius behind the "Reticulum Phantom" abandoned slopware https://github.com/roogle-dev/reticulum-phantom/issues/3 with some of the worst design patterns possible.

They of course ignored my GH issue I made all the way back in April, and instead seem to be doing this slopware project instead.

One of the idents for """Roogle""" is 03d8bba81e12ee683e86ecfad7a039c1. I tricked it into indexing one of my pages. It indexes stuff as soon as it gets the announce it seems.And it also seems to identify to every node it indexes with a different set of idents.

Also interesting the source for "Roogle" isn't available at all. So God only knows what it's doing on the backend. nightmares.....

But there are probably a dozen more of these types of things. Also because Roogle is public-facing with no rate limiting, I wonder if it isn't their frontend being used to scrape stuff from Nomad too?

rnid -R -i 3c7c1581f440671594d201d485682320?

Going to be working on some crawler tarpit software like https://zadzmo.org/code/nepenthes/ that anyone with good enuff VPS or hardware specs and a whole lot of bandwidth can run to try and slow some of this nonsense down.

The crawler is trying to index AND download every file, for every commit, in every repository. I don't know if it's actually physically possible to be stupid enough to create something like this.

That's going to be millions of pages for a git node. For my repos, that would be several hundreds of gigabytes to download in aggregate. Fuck this crap, I am so done with it.

It's basically a DoS machine masquerading as a "search engine".

Mark wrote:

The crawler is trying to index AND download every file, for every commit, in every repository. I don't know if it's actually physically possible to be stupid enough to create something like this.

That's going to be millions of pages for a git node. For my repos, that would be several hundreds of gigabytes to download in aggregate. Fuck this crap, I am so done with it.

It's basically a DoS machine masquerading as a "search engine".

Idk if it's just "Roogle", definitely need more data points before fingerpointing. I do wonder if it could be something else that is trying to be actively malicious.

Are any of the abuse requests identing to the node?

Awesome Zenith, I'm going to deploy that like there's no fucking tomorrow.

Also:

https://github.com/roogle-dev/reticulum-phantom/issues/4

If I didn't express my frustration succinctly enough, anyone else is welcome to weigh in.

I can't conclusively say it's just Roogle. Or even if that's the only source responsible, or even responsible at all without enough data. So might not be a good idea to jump the gun and just blame that guy, although what he is doing certainly is dumb, with no way to opt out. And from the previous "Reticulum Phantom" nonsense not a good track record.

There are a number of these """search engines""", usually all vibecoded. And there is always the possibility it could be just purely malicious, too. Although it seems to reflect scraping more than some type of DoS.

You're very sensible, Zenith. But I'm beyond not pointing fingers right now. That thing is definitely crawling the entire network as much as it can, and my opinion on that is pretty much what I put on that GH issue. It's so fucking annoying seeing something I spent ten years of my life on, always with the intention of making something that was truly good, and an escape hatch from so much of the crap that's happening, being completely usurped, twisted and shat out to unrecognizability by every random attention seeking moron with an LLM.

I am so fucking done with it. It's weaponized narcissism for crying out loud. It's radioactive ignorance.

I work 12 hour days meticulously improving and refining Reticulum, and I can never catch a fucking break because people invent new ways to be idiots to everyone else every three and a half hours now.

It's saddening. Create a network where people are actually sharing their own personal thoughts, opinions, writing and all kinds of other stuff, and some idiots goes and scrapes it all at a 10-minute interval to showcase a copy of it on the internet.

There is also STRUDEL, which states on their about page:

Every node heard is indexed, probed and crawled. There is no submit form and no opt-out: public pages are public.

And:

reachable nodes have their pages fetched from /page/index.mu, following on-node links a few levels deep, into the full-text index. Re-crawled about daily; skipped early if the front page has not changed.

At least this may be a seemingly reasonable attempt for indexing/crawling? Much better than crawling so deep they're popping out on another continent.

Every node heard is indexed, probed and crawled. There is no submit form and no opt-out: public pages are public.

"Public pages are public" durrrr
https://reticulum.at/
runs "strudel"

[[RNS Austria]]
type = TCPClientInterface
enabled = yes
target_host = rns.reticulum.at
target_port = 4242
transport_identity = b618afe3b5b331be88cf39582f13b3b5

Might be worth blackholing that ^^^^

Seems like they have a 40 page limit per node? Based on their site directory and sorting by page count

88c4deae3cf034666955b45a6f9cc436:/page/index.mu
nomad-search-test

e59960b7f9011dcdd7c1a6f14ac2c6d3:/page/index.mu
NomadSearch (MichMesh testnet)

two more

Sorry if you now think I'm an idiot. Of course what they are doing is insane, and I'm very sorry this happens.

But I must admit that I sometimes used some of these. To find nodes in my region, learn about reticulum, or find interesting reticulum related sources. I'm new - and it worked.

I only wanted to respond to the "nobody uses" part... I didn't know these nodes cause such trouble, but I did.