Full git nodes over Reticulum - Help testing
Started by Mark bc7291552be7a58f... ·
The latest source version of RNS (1.2.0) contains a new and useful addition: rngit and the git-remote-rns helper program. That means it's now possible to use git natively over Reticulum. See the "Using Reticulum on your system -> The rngit Program" section of the manual for details on how to use it.
This supports multiple groups/users and repositories hosted on the same node, with a full permission system for groups and individual repositories. No extra dependencies or bloat. Just needs git installed on the system. Then you can natively do: git clone rns://destination_hash/something/repo_name
And anything else that git allows you to do of course.
This is a functional, but of course early, and not very well tested preview. If you find this sort of stuff useful, please help testing it out, and provide any observations about bugs, problems, weird behavior, or of course also songs of rejoice and celebration, if it actually works wonderfully :)
Just a tip if you're not used to hosting git repositories: All the repos in the defined repo directories must be bare repositories! You can init a bare repository with git --bare init in empty folder. Or, if you already have a repo you want to try it out with, just copy the ".git" folder to for example "my_repo", then cd into that folder and run git config --bool core.bare true. You can now host the repo via rngit.
Enjoy!
If you don't want to build the wheels manually from source, you can just do:
pip install git+https://github.com/markqvist/reticulum
To install the latest source version.
this is fantastic Mark! I've been tunnelling TCP over RNS like a peasant to access a local repo
Fantastic! Can't wait to try this out
This is absolutely epic! Will try it as soon as possible.
Thanks everyone :) It's been a long time coming, but very nice that it's possible now.
Haha, ozCleric :D Glad to hear you can lay down those barbaric ways now.
I think there's still some missing functionality for edge cases such as force pushes with refs missing locally but present on remote (in which case remote will just reject), but most everyday stuff should work as expected at this point. And only deltas are transferred, so it's pretty light-weight.
I'll have to try git over LoRa tomorrow, might be a world-first, unless anyone else beats me to it ;)
If anyone wants a quick test, you can try cloning LXMF over Reticulum here:
git clone rns://7870891a2c2151d1da5564ce58177e89/reticulum/lxmf
It's just a test node I'm messing around with, so it may or may not be up, but feel free to give it a try.
Its super cool i have been testing over LoRa and it works pretty well
Hah, very nice! You're hereby officially the first person in the world to have run git over LoRa then :)
Congratulations Greg for the first git over LoRa transmission :)
And again my highest praise to you Mark for providing the means to make such awesome feats possible :)
As a mere mortal I am unfortunately stuck with a git-remote-rns failed: Server refused list: Not found error (see "Test rngit" section below).
I don't want to turn this into a "help me" post. I am optimistic that I will eventually figure it out ... but anyway, here is what I did.
System Information
Platform: Linux-6.1.0-44-amd64-x86_64 (Debian)
Reticulum: 1.2.0 from source
Python: 3.14.4 with GIL
Setup rngit
Install latest Reticulum from source
$ pip install git+https://github.com/markqvist/reticulum
Configure local only Reticulum
~/.reticulum/config
[reticulum]
enable_transport = no
share_instance = yes
instance_name = local
[interfaces]
[[Auto Server]]
type = AutoInterface
enabled = yes
Create server repository
$ mkdir ~/rngit
$ mkdir ~/rngit/server
$ mkdir ~/rngit/server/test.git
$ cd ~/rngit/server/test.git
$ git init --bare
Create local repository
$ mkdir ~/rngit/local
$ mkdir ~/rngit/local/test
$ cd ~/rngit/local/test
Initialize and configure local `test` repository:
$ git init \
&& git config init.defaultBranch master \
&& git config user.name r8io \
&& git config user.email r8io@example.com \
&& git remote add rns rns://0ab9a453107fc87cc4e7418209734502/server/test
Show configuration
$ git config --local --list
Make initial commit to local repository
$ cd ~/rngit/local/test
Create a file:
$ echo "Commmit 1" > test.txt
Commit:
$ git add *
$ git status
$ git commit -m 'Initial commit'
$ git log
Start Reticulum
$ rnsd -v
Initialize rngit
$ rngit
Default config file created. Make any necessary changes in ~/.rngit/config and restart rngit.
Print rngit info
$ rngit -p
Git Peer Identity : <32d4b47da0ccc32ac91f58917bfed41d>
Repository Node Identity : <9dc004dbea9902ac08cd07b6cec4172f>
Repositories Destination : <0ab9a453107fc87cc4e7418209734502>
Configure rgit
~/.rngit/config
[rngit]
node_name = rngit_node
[repositories]
internal = ~/rngit/server
[access]
internal = rw:32d4b47da0ccc32ac91f58917bfed41d
Test rngit
Start rngit
$ rngit -vv
Reticulum running in interpreted mode
Connected to locally available Reticulum instance via: LocalInterface[rns/local]
Utilising cryptography backend "openssl, PyCA 46.0.7"
Configuration loaded from ~/.reticulum/config
Loaded 23 known destination from storage in 294.45µs
Loaded Transport Identity from storage
Starting Reticulum Git Node...
Repositories identity loaded from ~/.rngit/repositories_identity
Loading repositery group "internal"
Loaded 1 repository for group "internal"
Reticulum Git Node listening on <0ab9a453107fc87cc4e7418209734502>
Announcing repositories destination
Clone test repository from rngit_node
$ cd ~/rngit/local
$ git clone rns://0ab9a453107fc87cc4e7418209734502/server/test test-clone
Cloning into 'test'...
Link established with remote
git-remote-rns failed: Server refused list: Not found
The corresponding rngit log output
Link request includes MTU signalling
Incoming link request with mode AES_256_CBC
Validating link request <a8721d3cc93ca39bcd67e1825706f6ca>
Incoming link request <a8721d3cc93ca39bcd67e1825706f6ca> accepted on LocalInterface[rns/local]
Peer connected to <git.repositories.9dc004dbea9902ac08cd07b6cec4172f:0ab9a453107fc87cc4e7418209734502>
Peer identified as <32d4b47da0ccc32ac91f58917bfed41d> on <a8721d3cc93ca39bcd67e1825706f6ca>
Handling request <1ee5859e84a66accd39dfea97c2bfc6d> for: /git/list
List request from remote <32d4b47da0ccc32ac91f58917bfed41d>
Resolving server/test permission 1 for <32d4b47da0ccc32ac91f58917bfed41d>
Peer disconnected from <git.repositories.9dc004dbea9902ac08cd07b6cec4172f:0ab9a453107fc87cc4e7418209734502>
Cleaned 1 links
Push local test repository to rngit_node
$ cd ~/rngit/local/test
$ git push rns master
Link established with remote
git-remote-rns failed: Server refused list: Not found
The corresponding rngit log output
Link request includes MTU signalling
Incoming link request with mode AES_256_CBC
Validating link request <05f4a7c80634e42566e31a0b60745af4>
Incoming link request <05f4a7c80634e42566e31a0b60745af4> accepted on LocalInterface[rns/local]
Peer connected to <git.repositories.9dc004dbea9902ac08cd07b6cec4172f:0ab9a453107fc87cc4e7418209734502>
Peer identified as <32d4b47da0ccc32ac91f58917bfed41d> on <05f4a7c80634e42566e31a0b60745af4>
Handling request <3937e0a05672fa5541e55e133538c07b> for: /git/list
List request from remote <32d4b47da0ccc32ac91f58917bfed41d>
Resolving server/test permission 2 for <32d4b47da0ccc32ac91f58917bfed41d>
Peer disconnected from <git.repositories.9dc004dbea9902ac08cd07b6cec4172f:0ab9a453107fc87cc4e7418209734502>
Cleaned 1 links
@r8io: You called your repository on the server "test.git" but you tried cloning "test". Names have to match. Try renaming the repository folder to "test", restart rngit and try again. Works here ;)
Also, grab the actual 1.2.0 release, caus it's out now (and lots of small fixes and efficiency improvements from before).
Have fun everyone, and take care of each other, caus I'm off to the woods again. Gonna climb some trees and roll in the dirt. Chase an elf or something. See y'all.
Thanks, will do! Take care as well.
Just tested it, worked fine for both cloning and pushing. Thanks a lot, Mark, this was a long-awaited feature!
Have fun everyone, and take care of each other, caus I'm off to the woods again. Gonna climb some trees and roll in the dirt. Chase an elf or something. See y'all.
Thanks for fixing issues and giving us rngit! Enjoy the tree climb, dirt roll and elf chase :)
Just want to report back that everything works as expected now. My above mentioned issue was, of course, entirely self inflicted :)
-
As Mark mentioned above, the
REPO_NAMEin the URL must match the repository directory name on the server. Do NOT use theREPO_NAME.gitnaming convention on the server because the URL would have to userepo_name.gitas well. In my case. usetestand NOTtest.gitas theREPO_NAME. -
Use the
GROUP_NAMEfrom the~/.rngit/configfile and NOT the parent directory name of the repository in the URL. In my case, useinternaland NOTserveras theGROUP_NAME.
URL
rns://DESTINATION_HASH/GROUP_NAME/REPO_NAME
Example
$ git push rns://0ab9a453107fc87cc4e7418209734502/internal/test
~/.rngit/config
# repository groups
[repositories]
internal = ~/rngit/server
[access]
internal = rw:32d4b47da0ccc32ac91f58917bfed41d
- Note to future self: Cloning an empty bare repository does NOT work. Do a
git pushfirst.
penv/lib/python3.12/site-packages/RNS/Transport.py", line 2226, in synthesize_tunnel public_key = RNS.Transport.identity.get_public_key() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get_public_key'
when trying to clone, followed by failed to establish link.
Mark, I'm curious if the names git-remote-rns and rngit are inspired by the CLI of the same name developed since late March by Eeems on github? https://github.com/Eeems/git-remote-rns
Git remote helpers must follow the naming convention git-remote-PROTOCOL. There's not really anything else to call it than git-remote-rns, since that's what git would expect. As for rngit, all the core tools included with RNS follow the rn naming convention, so again a pretty obvious choice. Git over Reticulum has been on the todo list for over two years, and I wrote a test implementation about a year ago to explore it, which was never finished due to other things being prioritized.
While working on the current implementation, I did discover Eeems' work, and considered whether to use that instead, or simply point to that as the recommended solution. But his implementation was not really designed to support many repositories, groups, multiple users, proper access permissions, extendability, and the other things I required for a full-fledged solution. It looked more like the proof-of-concept I had a year ago, so the right choice was to complete the current rngit work instead.
Just discovered that he's also the author of Oxide, though - very cool. I used that a lot back in the day when I had one of the early e-ink tablets :)