RNS Logo

rns.recipes

Community Forum

Forum / General / Full git nodes over Reticulum - Help testing

Full git nodes over Reticulum - Help testing

Started by Mark bc7291552be7a58f... ·

Mark bc7291552be7a58f...

The latest source version of RNS (1.2.0) contains a new and useful addition: rngit and the git-remote-rns helper program. That means it's now possible to use git natively over Reticulum. See the "Using Reticulum on your system -> The rngit Program" section of the manual for details on how to use it.

This supports multiple groups/users and repositories hosted on the same node, with a full permission system for groups and individual repositories. No extra dependencies or bloat. Just needs git installed on the system. Then you can natively do: git clone rns://destination_hash/something/repo_name

And anything else that git allows you to do of course.

This is a functional, but of course early, and not very well tested preview. If you find this sort of stuff useful, please help testing it out, and provide any observations about bugs, problems, weird behavior, or of course also songs of rejoice and celebration, if it actually works wonderfully :)

Just a tip if you're not used to hosting git repositories: All the repos in the defined repo directories must be bare repositories! You can init a bare repository with git --bare init in empty folder. Or, if you already have a repo you want to try it out with, just copy the ".git" folder to for example "my_repo", then cd into that folder and run git config --bool core.bare true. You can now host the repo via rngit.

Enjoy!

Mark bc7291552be7a58f...

If you don't want to build the wheels manually from source, you can just do:

pip install git+https://github.com/markqvist/reticulum

To install the latest source version.

ozCleric 1747e565d5d11f4c...

this is fantastic Mark! I've been tunnelling TCP over RNS like a peasant to access a local repo

joakim b918e659eeedac9a...

Love it! Git over Reticulum was only a question of time :)

Torlando db3ffd2575469a78...

Fantastic! Can't wait to try this out

r8io

This is absolutely epic! Will try it as soon as possible.

Mark bc7291552be7a58f...

Thanks everyone :) It's been a long time coming, but very nice that it's possible now.

Haha, ozCleric :D Glad to hear you can lay down those barbaric ways now.

I think there's still some missing functionality for edge cases such as force pushes with refs missing locally but present on remote (in which case remote will just reject), but most everyday stuff should work as expected at this point. And only deltas are transferred, so it's pretty light-weight.

I'll have to try git over LoRa tomorrow, might be a world-first, unless anyone else beats me to it ;)

Mark bc7291552be7a58f...

If anyone wants a quick test, you can try cloning LXMF over Reticulum here:

git clone rns://7870891a2c2151d1da5564ce58177e89/reticulum/lxmf

It's just a test node I'm messing around with, so it may or may not be up, but feel free to give it a try.

greg 4c35f93aaa4ebe92...

Its super cool i have been testing over LoRa and it works pretty well

Mark bc7291552be7a58f...

Hah, very nice! You're hereby officially the first person in the world to have run git over LoRa then :)

r8io

Congratulations Greg for the first git over LoRa transmission :)

And again my highest praise to you Mark for providing the means to make such awesome feats possible :)

As a mere mortal I am unfortunately stuck with a git-remote-rns failed: Server refused list: Not found error (see "Test rngit" section below).

I don't want to turn this into a "help me" post. I am optimistic that I will eventually figure it out ... but anyway, here is what I did.

System Information

Platform:  Linux-6.1.0-44-amd64-x86_64 (Debian)
Reticulum: 1.2.0 from source
Python:    3.14.4 with GIL

Setup rngit

Install latest Reticulum from source

$ pip install git+https://github.com/markqvist/reticulum

Configure local only Reticulum

~/.reticulum/config

[reticulum]
  enable_transport = no
  share_instance = yes
  instance_name = local
  
[interfaces]
  [[Auto Server]]
    type = AutoInterface
    enabled = yes

Create server repository

$ mkdir ~/rngit
$ mkdir ~/rngit/server
$ mkdir ~/rngit/server/test.git
$ cd ~/rngit/server/test.git
$ git init --bare

Create local repository

$ mkdir ~/rngit/local
$ mkdir ~/rngit/local/test
$ cd ~/rngit/local/test

Initialize and configure local `test` repository:
$  git init  \
&& git config init.defaultBranch master \
&& git config user.name r8io \
&& git config user.email r8io@example.com \
&& git remote add rns rns://0ab9a453107fc87cc4e7418209734502/server/test

Show configuration
$ git config --local --list

Make initial commit to local repository

$ cd ~/rngit/local/test

Create a file:
$ echo "Commmit 1" > test.txt

Commit:
$ git add *
$ git status
$ git commit -m 'Initial commit'
$ git log

Start Reticulum

$ rnsd -v

Initialize rngit

$ rngit
Default config file created. Make any necessary changes in ~/.rngit/config and restart rngit.

Print rngit info

$ rngit -p
Git Peer Identity        : <32d4b47da0ccc32ac91f58917bfed41d>
Repository Node Identity : <9dc004dbea9902ac08cd07b6cec4172f>
Repositories Destination : <0ab9a453107fc87cc4e7418209734502>

Configure rgit

~/.rngit/config

[rngit]
  node_name = rngit_node

[repositories]
  internal = ~/rngit/server
  
[access]
  internal = rw:32d4b47da0ccc32ac91f58917bfed41d

Test rngit

Start rngit

$ rngit -vv
Reticulum running in interpreted mode
Connected to locally available Reticulum instance via: LocalInterface[rns/local]
Utilising cryptography backend "openssl, PyCA 46.0.7"
Configuration loaded from ~/.reticulum/config
Loaded 23 known destination from storage in 294.45µs
Loaded Transport Identity from storage
Starting Reticulum Git Node...
Repositories identity loaded from ~/.rngit/repositories_identity
Loading repositery group "internal"
Loaded 1 repository for group "internal"
Reticulum Git Node listening on <0ab9a453107fc87cc4e7418209734502>
Announcing repositories destination

Clone test repository from rngit_node

$ cd ~/rngit/local
$ git clone rns://0ab9a453107fc87cc4e7418209734502/server/test test-clone
Cloning into 'test'...
Link established with remote
git-remote-rns failed: Server refused list: Not found

The corresponding rngit log output

Link request includes MTU signalling
Incoming link request with mode AES_256_CBC
Validating link request <a8721d3cc93ca39bcd67e1825706f6ca>
Incoming link request <a8721d3cc93ca39bcd67e1825706f6ca> accepted on LocalInterface[rns/local]
Peer connected to <git.repositories.9dc004dbea9902ac08cd07b6cec4172f:0ab9a453107fc87cc4e7418209734502>
Peer identified as <32d4b47da0ccc32ac91f58917bfed41d> on <a8721d3cc93ca39bcd67e1825706f6ca>
Handling request <1ee5859e84a66accd39dfea97c2bfc6d> for: /git/list
List request from remote <32d4b47da0ccc32ac91f58917bfed41d>
Resolving server/test permission 1 for <32d4b47da0ccc32ac91f58917bfed41d>
Peer disconnected from <git.repositories.9dc004dbea9902ac08cd07b6cec4172f:0ab9a453107fc87cc4e7418209734502>
Cleaned 1 links

Push local test repository to rngit_node

$ cd ~/rngit/local/test
$ git push rns master
Link established with remote
git-remote-rns failed: Server refused list: Not found

The corresponding rngit log output

Link request includes MTU signalling
Incoming link request with mode AES_256_CBC
Validating link request <05f4a7c80634e42566e31a0b60745af4>
Incoming link request <05f4a7c80634e42566e31a0b60745af4> accepted on LocalInterface[rns/local]
Peer connected to <git.repositories.9dc004dbea9902ac08cd07b6cec4172f:0ab9a453107fc87cc4e7418209734502>
Peer identified as <32d4b47da0ccc32ac91f58917bfed41d> on <05f4a7c80634e42566e31a0b60745af4>
Handling request <3937e0a05672fa5541e55e133538c07b> for: /git/list
List request from remote <32d4b47da0ccc32ac91f58917bfed41d>
Resolving server/test permission 2 for <32d4b47da0ccc32ac91f58917bfed41d>
Peer disconnected from <git.repositories.9dc004dbea9902ac08cd07b6cec4172f:0ab9a453107fc87cc4e7418209734502>
Cleaned 1 links
Mark bc7291552be7a58f...

@r8io: You called your repository on the server "test.git" but you tried cloning "test". Names have to match. Try renaming the repository folder to "test", restart rngit and try again. Works here ;)

Also, grab the actual 1.2.0 release, caus it's out now (and lots of small fixes and efficiency improvements from before).

Have fun everyone, and take care of each other, caus I'm off to the woods again. Gonna climb some trees and roll in the dirt. Chase an elf or something. See y'all.

r8io

Thanks, will do! Take care as well.

Anonymous

Just tested it, worked fine for both cloning and pushing. Thanks a lot, Mark, this was a long-awaited feature!

joakim b918e659eeedac9a...
edited

Have fun everyone, and take care of each other, caus I'm off to the woods again. Gonna climb some trees and roll in the dirt. Chase an elf or something. See y'all.

Thanks for fixing issues and giving us rngit! Enjoy the tree climb, dirt roll and elf chase :)

r8io

Just want to report back that everything works as expected now. My above mentioned issue was, of course, entirely self inflicted :)

  1. As Mark mentioned above, the REPO_NAME in the URL must match the repository directory name on the server. Do NOT use the REPO_NAME.git naming convention on the server because the URL would have to use repo_name.git as well. In my case. use testand NOT test.git as the REPO_NAME.

  2. Use the GROUP_NAME from the ~/.rngit/config file and NOT the parent directory name of the repository in the URL. In my case, use internal and NOT server as the GROUP_NAME.

URL
  rns://DESTINATION_HASH/GROUP_NAME/REPO_NAME
Example
  $ git push rns://0ab9a453107fc87cc4e7418209734502/internal/test
~/.rngit/config

# repository groups
[repositories]
  internal = ~/rngit/server
  
[access]
  internal = rw:32d4b47da0ccc32ac91f58917bfed41d
  1. Note to future self: Cloning an empty bare repository does NOT work. Do a git push first.
Anonymous

penv/lib/python3.12/site-packages/RNS/Transport.py", line 2226, in synthesize_tunnel public_key = RNS.Transport.identity.get_public_key() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get_public_key'

when trying to clone, followed by failed to establish link.

Anonymous

Mark, I'm curious if the names git-remote-rns and rngit are inspired by the CLI of the same name developed since late March by Eeems on github? https://github.com/Eeems/git-remote-rns

Mark bc7291552be7a58f...

Git remote helpers must follow the naming convention git-remote-PROTOCOL. There's not really anything else to call it than git-remote-rns, since that's what git would expect. As for rngit, all the core tools included with RNS follow the rn naming convention, so again a pretty obvious choice. Git over Reticulum has been on the todo list for over two years, and I wrote a test implementation about a year ago to explore it, which was never finished due to other things being prioritized.

While working on the current implementation, I did discover Eeems' work, and considered whether to use that instead, or simply point to that as the recommended solution. But his implementation was not really designed to support many repositories, groups, multiple users, proper access permissions, extendability, and the other things I required for a full-fledged solution. It looked more like the proof-of-concept I had a year ago, so the right choice was to complete the current rngit work instead.

Just discovered that he's also the author of Oxide, though - very cool. I used that a lot back in the day when I had one of the early e-ink tablets :)

Post a Reply

Markdown

Supports Markdown: **bold**, *italic*, `code`, ```code blocks```, [links](url)

Log in to upload images

Proof of work verification for anonymous posting

Copied to clipboard