RNS Logo

rns.recipes

◈ 9ce92808be498e9e05590ff27cbfdfe4
NomadNet 1.4.0 released with experimental image support https://pypi.org/project/nomadnet/

So, this *is* possible, but...

Started by Mark bc7291552be7a58f... ·

A. Non
#21

Anonymous wrote:

Personally, I'm in favor, as long as it's used sparingly by NomadNet sites and not transmitted over slow links like LoRa, HF etc. unless a download is explicitly requested by the client, and caching and compression are smartly used.

Could be useful for Retipedia and forums (like this one), along with probably some other uses I can't think of

 

I use lynx to browse, it's still quite useful. It will display the alt text or caption for an image if available, perhaps enforcing a caption for images would be useful for nomad? Text only users would not be left out and it increases accessibility.

Anonymous
#22

I'm in favor of inline images upon request only. Maybe a built-in low-res preview as well?

Emru 227e1c4aecd3ae95...
#23

I'd say no
Let's keep TUI a TUI, but I think it would be nice in GUI nomadnet (of course with configuration option to only download images smaller than X, or something like that).

G6PHF eaf08697fa4f945d...
#24

aetherlab wrote:

r8io wrote:

@aetherlab, no I don't have an HF rig. Unfortunately I know next to nothing about radio stuff, but I have that on the list for my next life. Maybe I will take a class in Walhalla ;)

Oh sorry, I meant Mark's picture in Nomad! 73 de LZ1SWE
It's a Yaesu FT991a by the looks of it, which is one of my radios :)
IMG_3118_2.JPG

 

73
de G6PHF

Professor_Boop 482a4ba7592687ac...
#25

Hey, on a slight side tangent: I think JPEG XL might be a great match for Nomadnet because of its progressive decoding. You store a single file, and the nomadnet site can send just the first ~1%-ish of the codestream (the low-frequency frame) to give a decodable coarse image on slow links like LoRa, or deliver the remaining bytes for full quality for users on high-bandwidth links. Truncated prefixes stay decodable, so it also degrades gracefully if a transfer cuts off partway.

Zenith
#26

Professor_Boop wrote:

Hey, on a slight side tangent: I think JPEG XL might be a great match for Nomadnet because of its progressive decoding. You store a single file, and the nomadnet site can send just the first ~1%-ish of the codestream (the low-frequency frame) to give a decodable coarse image on slow links like LoRa, or deliver the remaining bytes for full quality for users on high-bandwidth links. Truncated prefixes stay decodable, so it also degrades gracefully if a transfer cuts off partway.

 

+1 for JXL!!!!!

d4e363fb 83f3e9497c4859cd...
#27

Inline images are not important to me (and would likely not work in tmux anyway). I'd rather see a download handler that can pipe image data to a (preferrably progressive) image viewer, AV media to mpv, or whatever else is configured.

K8 - Mobile d8a95477de4323a1...
#28

Lmao, I love that you actually did this. I think it would be great to have, but like others have said only if it's opt-in, degrades gracefully, and has good support for progressive loading. Most of the time I think the kind of interface I'd want is probably being able to select an image and have it open in a modal box, with only a link, or maybe a small preview or a micron text rendering displayed inline.

 

You were talking about wanting to make a game, right? So maybe there could be a sort of consent ladder where users can opt-in to levels of image display and interactivity, and the page creator is responsible for making sure that everything still functions at every level.

Mark 8dd57a7382268096...
#29

Ok, everyone gets what they want - more or less, lol.

 

  • Don't want image loading or display at all? Set image_loading to never in the config.
  • Want to only load images manually? Set to manual.
  • Want to load automatically if link conditions are suitable? auto is the answer.
  • Don't care about the bandwidth? always has you covered.

 

The whole thing is gated by the terminal actually being able to render images. If not, no images are ever fetched or attempted rendered. This is queried and determined dynamically on startup, so no user config is necessary. If image display is not possible, or disabled in the config, images render as alt-text placeholders instead. The alt-text is mandatory, it must be specified for an image tag to be valid.

 

No matter what kind of image loading method is specified, images will always load asynchronously after the main page is loaded, re-using the existing link if possible. Images are loaded sequentially, one at a time, in the order they appear on the page. Once an image begins loading, its placeholder changes to a dynamically updating progress indicator showing transfer progress and speed. Once the image is fully loaded, poof! It appears on the page!

 

Images are cached locally, and will only download if the local cache is empty. I haven't fully worked out the cache specification mechanics yet, so that is still a pending item. For now, you can manually clear and reload for a page with Ctrl-X.

 

While I haven't finished implementing this yet, the request format is prepared for using a per-link key, so images cannot be deep-linked from other nodes than the one serving them; that is, you will be able to specify that images are only served to clients viewing a page on your node, and not from somewhere else. This will probably be the default.

 

The current implementation adds zero extra dependencies or bloat to the program. This was a bit tricky, since webp decoding is not exactly a five-liner in Python (or any language). What I ended up doing was to create a decoding multi-backend proxy, that should be able to resolve a working webp decoder on more or less any operating system. The nice thing here is, that almost every OS in existence already has some way of decoding webp (one good thing about web-browsers, I guess), so in most cases we can just find the distro-installed libwebp.so (or dylib, or dll), and hook into that via ctypes.

 

So yeah, in most cases, it's just using the OS-native decoding through the C ABI. It also supports using common utilities such as image magick convert, ffmpeg, sips and such. Yes, PIL works too :)

 

If you have a terminal that does support image display with Kitty TGP, but it's still not working, you can run python -m nomadnet.ui.textui.images._selftest and post / send me the results.

 

From the beginning, my intention with this was to only allow webp for network transport, and judging the feedback, I think that's still a pretty safe bet to begin with. There's some good points about JPEG XL, hadn't thought about that actually, but I'm not sure if it actually will prove to be more bandwidth efficient than WebP at high compression. I'll have to run some tests on that at some point to determine it.

 

Either way, adding JPEG XL will be a down-the-road thing once the whole basic functionality is proven and the kinks worked out. If you knew the hoops I had to jump through just to get webp decoding working without adding massive dependencies and bloat, you'd understand ;) Adding progressive JPEG XL loading and rendering will be a massive undertaking. Not impossible, just massive.

 

Regarding progressive / bandwidth selective loading, it may actually be simpler and more optimal overall to allow sending a profile key on the request (similar to LXST profiles), which will dynamically downscale and increase compression level at the node, before sending (that was my initial plan). Same result, at lot less over mechanistic overhead. Anyway, I'm open to the idea and will keep both in mind. But for now, webp it is ;) The selective profile-based loading is not implemented yet, but the system is prepared for it. At this point, it's probably better to wrap up an initial release of this, and see how it works in the wild, adjust and improve, and bundle all of that stuff into the next iteration, when there's more real-world data. So let's consider the current state of affairs the initial, experimental base-line.

 

Having a per-image download / modal view button is a great idea, if you just want an easy way to download images instead of viewing the in-page. I haven't implemented this yet, but that should definitely be added.

 

Media serving is now built into nomadnet itself as well, of course, so the built-in page node can serve images for pages. See the relevant guide sections on how to create micron image tags. The "Display Test" guide entry also tests image display now, so you can check easily if it works.

 

For rngit, the page node has also been updated, so it can natively serve images within rendered documents, display them in the file tree browser, et cetera. This is in the latest source version on Aleph. There's still a few finer points missing here, for example auto-conversion of image tags in markdown documents, but all the fundamentals are working (you can see the RNS logo in the Reticulum repo readme now). There's an automatic media converter here, that will convert non-webp images to webp for transmission, so more or less all images/graphics should work transparently.

 

If you want to try it out, all the source is on Aleph now. I'll probably wrap up a release for nomadnet shortly, so it can be tested a bit more conveniently, but the rngit changes will have to be fetched from source until the next release.

 

I took a lot of care to try and design and implement this in a way that is robust, degrades neatly, makes sensible use of resources, and doesn't destroy functionality or the experience for users who don't want or need it, while keeping nomadnet itself as light-weight as possible. That being said, there's bound to still be weirdness, bugs or edge cases I haven't considered, so I think it's best to let this have a real spin in the wild to see how it goes. Consider everything experimental regarding to this; we'll adjust and correct if something turns out idiotic.

Mark 8dd57a7382268096...
#30

Aaand, there's a wheel. 1.4.0 on Aleph and pip :)

Mark 8dd57a7382268096...
#31

If you want to try it out on a live node that actually serves images, you can try browsing some of the repos on Aleph that has images in their trees, for example reticulum/docs/source/graphics or something like that :)

burger ace748e1c4e3fd4e...
edited #32

[removed]

Anonymous
#33

It might be a good idea to use the QOI format.
It's especially useful for microcontrollers.
https://qoiformat.org

Zenith
edited #34

Just an FYI that WebP and AVIF don't always compress better than PNG and JPEG. In about 85% of cases they do, but not always. Btw I was working on this exact same problem about 2 months ago for sending images via JXL and raster data over HF radio as a demo for my project, modem73. No Reticulum in that mix though.
image.png

 

 

https://www.tempertemper.net/blog/avif-and-webp-are-not-always-better-than-png-and-jpg

Zenith Admin
edited #36

Also it would be really nice if instead of just showing:

 

image.png

 

We could have something like:
image.png

 

 

Edit:
Nvm, I just realized that the file size isn't known until we download the remote image. Any way that might be possible without requests? Would it make sense to cache that info in the page itself? Or have it as something that can be passed to the tag?

Anonymous
#37

Not asking to change the WebP default, but a node-side profile that can emit a QOI (or even raw) thumbnail for constrained clients would let standalone hardware actually render images inline instead of alt-text only.

K8 a361065d039738f6...
#38

This is working perfectly in iTerm2, I had no idea it actually supported the Kitty image protocol.

Post a Reply

Supports Markdown: **bold**, *italic*, `code`, ```code blocks```, [links](url)

Log in to upload images

Quote
Copied to clipboard