Knowledge Base

Preserving for the future: Shell scripts, AoC, and more

My Luanti open-source contributions so far

I'm not here to list all my mods that I use in my Luanti Mineclonia world, but I am going to talk about the work I've done to try to help the ecosystem.

I wrote a very small mod named list_to_file that dumps active users to a file, so I can use server-side scripts to render that into a dashboard for myself. That's a separate topic for a separate time. I also brought in a few functions from the forum that list the items in their respective files: items-nodes.txt, items-tools.txt, and items-crafts.txt. Small stuff that average users wouldn't need. I was struggling with the names of nodes for giving myself "mapserver:poi_green" for example. Shoulda gone to creative mode, too late now.

I found mod inventory_pouches ideal for my small server, and I had to fix a few bugs, including the dye colors for Mineclonia, and losing contents upon server restart (which defeats most of the purpose of such a mod). That fix took me 3 hours of intense debugging, only to realize the only change that was necessary was changing a lua loop for i, stack_table in ipairs(inv_table) to pairs(inv_table). That's it. One character. I added tons of debugging and helper functions to narrow down the problem. I wonder if the lua/luanti syntax changed or if this mod has always been broken.

And then I hacked mod inventory_icon to include the inventory pouches. That took less work than debugging the pouches to persist across game restarts, although I'm still not 100% certain the icon placement in the HUD is bulletproof. But hopefully people will find my changes useful.

Appendix

A quick way to view what is a git repo in that directory.

[luanti@server4|/home/luanti/.minetest/mods]$ find . -maxdepth 2 -iname '.git' -printf '%h\n' | while read dir ; do ( cd "${dir}" ; git remote -v ; ) | sed -r -e "s@^@${dir} @" ; done | column -t | grep fetch
./dumpnodes          origin       https://github.com/Montandalar/dumpnodes              (fetch)
./list_to_file       gitlab       https://gitlab.com/bgstack15/list_to_file.git         (fetch)
./list_to_file       origin       http://server3/git/list_to_file                       (fetch)
./inventory_pouches  github       https://github.com/JamesClarke7283/inventory_pouches  (fetch)
./inventory_pouches  github_mine  https://github.com/bgstack15/inventory_pouches        (fetch)
./inventory_pouches  origin       http://server3/git/inventory_pouches                  (fetch)
./inventory_icon     codeberg     https://codeberg.org/Wuzzy/minetest_inventory_icon    (fetch)
./inventory_icon     origin       http://server3/git/minetest_inventory_icon            (fetch)

Luanti in docker-compose

I migrated my Luanti Mineclonia world from my desktop to a dedicated server. Thankfully, it is very easy to do that! I love Free Software (free as in libre).

I set it up in docker-compose, and I merely copied my entire .minetest directory to the service account. So it had all my worlds, modes, games, and (the three different categories in the ContentDB for Luanti) already.

Then I set up my service in a docker-compose.yml, and also the venerable mapserver in a separate compose file. Here's what I've got so far.

# File: /home/luanti/game/docker-compose.yml
# Reference:
#    https://github.com/linuxserver/docker-luanti
#    https://github.com/minetest-mapserver/mapserver/blob/master/docker-compose.yml
#    https://github.com/minetest-mapserver/mapserver/blob/master/doc/install.md more useful
---
version: "3.5"
services:
  luanti:
    image: lscr.io/linuxserver/luanti:latest
    container_name: luanti
    environment:
      - PUID=1009
      - PGID=1009
      - TZ=Etc/UTC
      - "CLI_ARGS=--gameid mineclonia --worldname world1"
    volumes:
      - /home/luanti/.minetest:/config/.minetest
    ports:
      - 30000:30000/udp
    restart: unless-stopped
...

Yes, I used the linuxserver.io build of a docker image because it came up first in my search. It also took me a while to learn that it configured to use config file .minetest/main-config/minetest.conf. No biggie, once you know about it.

And then my mapserver is separate, so I can bring it up and down while I'm testing.

# Reference:
#    https://github.com/minetest-mapserver/mapserver/blob/master/docker-compose.yml
#    https://github.com/minetest-mapserver/mapserver/blob/master/doc/install.md more useful
---
version: "3.5"
services:
  mapserver:
    image: ghcr.io/minetest-mapserver/mapserver
    restart: always
    networks:
      - default
    volumes:
      - /home/luanti/.minetest/worlds/world1:/minetest
    working_dir: "/minetest"
    ports:
      - 8086:8080/tcp
...

I had to open up the firewall on the docker host (server4).

sudo firewall-cmd --add-port=8086/tcp --permanent
sudo firewall-cmd --reload

On my https reverse proxy, for which I still don't have websockets quite right but it is somehow functional enough:

# 2025-03-07-6 08:02 for luanti game mineclonia world world1
RewriteEngine on
RewriteRule ^/map-world1$ /map-world1/ [R,L] # not working, but whatever
<Location "/map-world1/">
   ProxyPreserveHost On
   ProxyPass        http://server4:8086/ retry=20 connectiontimeout=300 timeout=300 upgrade=websocket
   ProxyPassReverse http://server4:8086/
   RequestHeader    set X-Script-Name /map-world1
   RequestHeader    set Upgrade websocket
   RequestHeader    set Connection upgrade
</Location>

Once I ran the docker-compose up -d for mapserver, it generated the worlds/world1/mapserver.json which I customized for my needs.

{
    "configversion": 1,
    "port": 8080,
    "enableprometheus": true,
    "enablerendering": true,
    "enablesearch": true,
    "enableinitialrendering": true,
    "enabletransparency": false,
    "enablemediarepository": false,
    "webdev": false,
    "webapi": {
        "enablemapblock": false,
        "secretkey": "IUEXAMPLELTdht3n"
    },
    "layers": [
        {
            "id": 0,
            "name": "below1",
            "from": -8,
            "to": -7
        },
        {
            "id": 1,
            "name": "below2",
            "from": -7,
            "to": -6
        },
        {
            "id": 2,
            "name": "below3",
            "from": -6,
            "to": -5
        },
        {
            "id": 3,
            "name": "below4",
            "from": -5,
            "to": -4
        },
        {
            "id": 4,
            "name": "below5",
            "from": -4,
            "to": -3
        },
        {
            "id": 5,
            "name": "below6",
            "from": -3,
            "to": -2
        },
        {
            "id": 8,
            "name": "below7",
            "from": -2,
            "to": -1
        },
        {
            "id": 6,
            "name": "Ground",
            "from": -1,
            "to": 10
        },
        {
            "id": 7,
            "name": "Sky",
            "from": 11,
            "to": 24
        }
    ],
    "renderingfetchlimit": 500,
    "renderingjobs": 8,
    "renderingqueue": 10,
    "incrementalrenderingtimer": "5s",
    "mapobjects": {
        "areas": true,
        "bones": false,
        "protector": true,
        "xpprotector": true,
        "privprotector": true,
        "technic_quarry": false,
        "technic_switch": false,
        "technic_anchor": false,
        "technic_reactor": false,
        "luacontroller": false,
        "digiterms": false,
        "digilines": false,
        "travelnet": false,
        "mapserver_player": true,
        "mapserver_poi": true,
        "mapserver_label": true,
        "mapserver_trainline": false,
        "mapserver_border": true,
        "tileserverlegacy": true,
        "mission": false,
        "jumpdrive": false,
        "smartshop": false,
        "fancyvend": false,
        "atm": false,
        "train": false,
        "trainsignal": false,
        "minecart": false,
        "locator": false,
        "signs": true,
        "mapserver_airutils": false,
        "phonograph": false,
        "um_area_forsale": false
    },
    "mapblockaccessor": {
        "expiretime": "10s",
        "purgetime": "15s",
        "maxitems": 50
    },
    "defaultoverlays": [
        "mapserver_poi",
        "mapserver_label",
        "mapserver_player"
    ],
    "skins": {
        "enableskinsdb": false,
        "skinspath": ""
    },
    "worldpath": "./",
    "datapath": "./",
    "colorstxtpath": "./"
}

The initial map tile generation took a while (a few hours?) on my hardware, for a 160MB map.sqlite. It's worth noting that the from and to values in the layers are not node coordinates; they are the mapblock coordinates. You just divide the node coordinates by 16 to get the mapblock coordinates. So in a Mineclonia mapgen v7 world with node Y coord -128 as bedrock, the bottom is just -8. And my colors.txt which shows grass as an ugly brown (for some lame reason) came from my Luanti mapping notes.

I was hoping to investigate adding mod respawn support to mod mapserver but since poi was already supported and is good enough, I didn't bother. I can live with adding crafted blocks in the world for the map to show them.

I haven't yet solved the problem of world backups. I suppose I would want to exclude all the mapserver tile files.

Installing Luanti flatpak

I set up a user with Luanti on Windows. They had a great link on the front page. The download asset has the luanti.exe inside the bin/ subdirectory.

The bad news: This is the latest and greatest version, 5.11.0 as of the time of this writing. But the Devuan (Debian) package of luanti is still on 5.10.0. So I had to ponder the best way to get this client connected to my instance.

I decided to research the flatpak, and Devuan has supported flatpak for years now (since Debian level 10, however long ago that was). The install instructions were very easy.

sudo apt-get install -y flatpak

Then add the flathub repository.

flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo

I didn't even have to restart, like the instructions said. I wonder what I'm messing up then by skipping that. Oh, well. Then I was off to install:

flatpak install flathub net.minetest.Minetest

Apparently to run takes a special invocation:

flatpak run net.minetest.Minetest

One one system where I did reboot, the xdg menu generator for fluxbox did parse the /var/lib/flatpak/exports/share/applications/net.minetest.Minetest.desktop which is nice! It looks like it has updated this env var:

XDG_DATA_DIRS=/home/public/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share

I've used one AppImage (for DarkTable which was OK. And this one flatpak I've just tried is OK. I wouldn't want to use this sort of tech for everything, but for a frivolous game that isn't a business-related workflow, I can live with it.

Luanti mapping notes

I have some partially-complete notes about how I built a series of static images of the maps of my world in Luanti Mineclonia.

Building maps of my world

  1. Install minetestmapper

    sudo apt-get install minetestmapper
    
  2. Download https://github.com/luanti-org/minetestmapper/blob/master/util/generate_colorstxt.py

  3. Install mod dumpnodes.

    cd ~/.minetest/mods
    git clone https://github.com/Montandalar/dumpnodes
    
  4. Ran the game, loaded my world, and ran /dumpnodes which saves to ~/.minetest/worlds/world1/nodes.txt

  5. Generate colors.txt.

    cd ~/.minetest/worlds/world1
    python3 ~/Downloads/generate_colorstxt.py --game ~/.minetest/games/mineclonia
    
  6. Run the map printout.

    1. Manual step

      minetestmapper -i ~/.minetest/worlds/world1 -o ~/foo1.png
      
    2. Shell script to generate each individual layer.

      #!/bin/bash
      # Startdate: 2025-02-24-2 21:41
      # purpose: make set of maps for each layer of Mineclonia world
      WORLD="${WORLD:-${1}}"
      #OUTPUT="${OUTPUT:-${2}}"
      MAX="${MAX:-128}"
      MIN="${MIN:--128}"
      for word in $( seq ${MIN} ${MAX} ) ;
      do
         outname="$( printf '%04d' "${word}" )"
         next="$( printf '%s\n' "${word}-1" | bc )"
         minetestmapper -i "${HOME}/.minetest/worlds/${WORLD}" -o "${HOME}/Downloads/maps/map${outname}.png" --max-y "${word}" --min-y "${word}"
      done

Map of my world

Someday I want to replicate some small mapping program that I used for OG minecraft back in 2012, which let you use a slider to scroll through the layers of a world. I used it to look for diamond and interesting structures. It was awesome, and I can't find it in my old files or notes or the Internet.

I tried Luanti (tafka minetest)

I recently tried Luanti, which is really an engine for various voxel games. The one "game" I tried in it is of course one of the Minecraft clones. I picked Mineclonia. I enjoyed the in-game content browser, but it had some flaws.

The download speed of the website is slow, below 100kB per second so it can take some time, and even it failed a few times. The in-game downloader doesn't indicate what went wrong other than "failed." If run from a console, the console might show that the website timeout was reached. You can visit the download link for a game/mod/texture pack, and move the downloaded zip file to ~/.minetest. Then explode the zip inside a directory named the correct thing: game or mods. I did not try any texture packs.

The game is supposed to be a clone of Minecraft, and the last time I played that was back in 2012. So there's so many new features. There's traders wandering around, and way more flowers and wildlife than I remember.

After dying a few times, I learned that I just needed to modify the settings for this "game" to allow "keep inventory."

It's been an enjoyable experience, digging the Mines of Moria again. I wasn't in the Free Software world back then, but now I am, and Luanti scratches this itch! I was also impressed that running a headless server was as easy as running luanti --server. It chooses the default (only) game and gets it started. And backing up the world is of course as easy as targzing the directory. There's more complicated steps for bupping a live world, but my 0200 cron job will never interfere with my playing.

And I remember futzing around with Java. In Windows, that was annoying. It's still mildly frustrating in a GNU+Linux world, but Luanti doesn't even need java. I call that an improvement as well. Just apt-get install luanti.

For the Free Software netizen with a desire to avoid creepers and find diamond ore, go try Luanti.

I have not been paid for saying any this. If anybody wants to change that, let me know!

Finding the best tokens to money ratio from mission items in Lego Universe

In the self-hostable Darkflame Lego Universe game server, you can earn 2 types of currency: money, and faction tokens. In the late game, you accumulate these in spades, but only money is still the most useful. So a lot of advanced players are interested in determining the most efficient way to convert faction tokens into money.

The way to do this scientifically is within the database. All the mission objects are stored in the CDServer.sqlite file, so we can use sqlite3 for this.

I have a whole methodology of what I did, in heading Narrative below. It's very lengthy, and really just my scratch pad combined with the steps I took to become familiar with the database.

tl;dr

All the experienced Lego Universe players are right: Castle Model Pack 3. You can buy this for 10 tokens at Honor Accolade in Nexus Tower, and sell the items from it for 798 money, which makes the token:money ratio 1:79.

Fun picture showing ratio of money to tokens at 79:1

Sql query

You need at least sqlite 3.39 for the FULL JOIN operation. My narrative explains the same query without it, but you're not going to like it.

Technically this query is still incomplete because it shows some of the "small number of random options for rewards" but they also aren't available at Honor Accolade.

SELECT i.mid, IFNULL(i.tokens,0)+IFNULL(p.tokens,0) tokens, IFNULL(i.money,0)+IFNULL(p.money,0) money, ((IFNULL(i.money,0)+IFNULL(p.money,0))/10)/(IFNULL(i.tokens,0)+IFNULL(p.tokens,0)) moneypertoken, i.oid, i.name, p.o2count, p.o2id, p.o2names
FROM (
    SELECT m.id mid, SUM(IFNULL(ic.commendationCost,0)) AS tokens, SUM(IFNULL(ic.baseValue,0)) money, o.id oid, o.name name, NULL, NULL, NULL
    FROM Objects o
    JOIN ComponentsRegistry cr ON cr.id = o.id
    JOIN ItemComponent ic ON ic.id = cr.component_id
    JOIN Missions m ON o.id IN (m.reward_item1, m.reward_item2, m.reward_item3, m.reward_item4)
    WHERE cr.component_type IN (11)
    --AND m.id IN(427,14,17,82,633)
    GROUP BY o.name
) i --items
FULL JOIN (
    SELECT mid, SUM(IFNULL(tokens,0))+SUM(IFNULL(tokens2,0)) tokens, SUM(IFNULL(baseValue,0))+SUM(IFNULL(baseValue2,0)) money, oid, SUBSTR(MAX(oname),1,300) name, o2namecount o2count, SUBSTR(GROUP_CONCAT(o2id),1,300) o2id, SUBSTR(GROUP_CONCAT(o2name),1,300) o2names
    FROM (
        SELECT m.id mid, cr.component_id as crid, ic.baseValue, ic.commendationCost AS tokens, o.id oid, o.name oname, pc.LootMatrixIndex, SUM(ic2.baseValue) baseValue2, SUM(ic2.commendationCost) as tokens2, SUBSTR(GROUP_CONCAT(o2.id),1,300) o2id, SUBSTR(GROUP_CONCAT(o2.name),1,300) o2name, COUNT(o2.name) o2namecount, GROUP_CONCAT(ic2.baseValue) o2_baseValues
        FROM Objects o
        FULL JOIN ComponentsRegistry cr ON cr.id = o.id
        FULL JOIN ItemComponent ic ON ic.id = cr.component_id
        FULL JOIN PackageComponent pc on pc.id = cr.component_id
        FULL JOIN LootMatrix lm ON lm.LootMatrixIndex = pc.LootMatrixIndex
        FULL JOIN LootTable lt ON lt.LootTableIndex = lm.LootTableIndex
        FULL JOIN Objects o2 ON lt.itemid = o2.id
        FULL JOIN ComponentsRegistry cr2 ON cr2.id = o2.id
        FULL JOIN ItemComponent ic2 ON ic2.id = cr2.component_id
        FULL JOIN Missions m ON o.id IN (m.reward_item1, m.reward_item2, m.reward_item3, m.reward_item4)
        WHERE cr.component_type = 53
        AND cr2.component_type = 11
        --AND m.id IN (427,14,17,82,633,1254)
        AND lm.percent > 0.98 AND lm.minToDrop >=1
        AND o.id > -1
        GROUP BY o.id
    )
    GROUP BY oid
) p --packages
ON i.oid = p.oid
ORDER BY moneypertoken DESC
;

Inferior sql query

The query without full joins and is slightly more flawed, is this one. You can run this on CentOS 7 sqlite3 (less than 3.39).

Read more…

html collapsible div without javascript

I wanted to have some collapsible contents in html without using javascript. I was fine using css, but a brief Internet search revealed this gem: css - Collapse without javascript - Stack Overflow

Just use the detail tag. That's it.

<detail><summary>Instructions</summary>Install the application underneath /usr/bin, and the desktop icon in /usr/share/applications.</detail>

Let's see if what I put below works in the final rendered web page.

InstructionsInstall the application underneath /usr/bin, and the desktop icon in /usr/share/applications.

References

In-line.

Adding Let's Encrypt root certs to Android 6

Old Android devices do not have the current Let's Encrypt root certificates installed, e.g., Android < 7.0.

You might need to enable a screen lock password or pin (Settings -> Security -> Screen lock -> PIN). You can remove it after installing these certs.

Install these certificates for "VPN and apps." Note that old Android might want to download these as a .crt file extension, in order to prompt you to install them as a trusted root, which is left as an exercise for the reader.

  1. ISRG Root X1 https://letsencrypt.org/certs/isrgrootx1.pem
  2. Let's Encrypt R3 https://letsencrypt.org/certs/lets-encrypt-r3.pem

References

Weblinks

  1. Ripped directly from How-to install a root certificate on Android 6.0 devices? – Geolantis.360 Knowledgebase
  2. Extending Android Device Compatibility for Let's Encrypt Certificates - Let's Encrypt

carps-cups driver works great for Canon ImageCLASS D320

I have a desktop Linux user who needed to print to his old Canon ImageCLASS D320 printer. CUPS doesn't have any built-in drivers for this, so I had to search the Internet like a noob. But, my ability to use what I found very much depends on my experience and expertise! And thankfully, the project I found, carps-cups was well-designed and took only a small amount of expertise. A standard make && sudo make install later, we could print to the printer!

Further poking after the fact, I should have tried building a dpkg. It was a single user, in a non-scaling environment, so I didn't bother. But if the printer were on my main network (well, it would be behind the one cups server), I would have investigated a dpkg.

FreeIPA on Devuan and sss ssh knownhosts proxy

After this month's updates to my operating systems, I have found something frustrating regarding sssd/freeipa.

Firstly, I had to udpate my package to depend on dnsutils | bind9-dnsutils, which was also shared with upstream Debian package by somebody else shortly afterwards. Seriously, I beat them by 1 hour!

Secondly, after updating, I would get a bogus error when using ssh client.

$ ssh server3

******************************************************************************
Your system is configured to use the obsolete tool sss_ssh_knownhostsproxy.
Please read the sss_ssh_knownhosts(1) man page to learn about its replacement.
******************************************************************************

Connection closed by UNKNOWN port 65535

Apparently file /etc/ssh/ssh_config.d/04-ipa.conf which is not directly owned by package freeipa-client (because it depends on freeipa being configured; i.e., the host being joined to a domain) contains a now-deprecated piece of information. I had to dig around to find info about this. It was back in version 4.12.0. The FreeIPA team wanted to replace sss_ssh_knownhostsproxy with sss_ssh_knownhosts (bug #9536). They solved their upgrade processes, presumably for the Fedora/Enterprise Linux world, but whatever fixes they did apparently don't run client-side on a Devuan/Debian freeipa client.

So I had to poke around to learn exactly what my 04-ipa.conf should look like. I found it:

# IPA-related configuration changes to ssh_config
# Last-modified: 2025-02-02-1 22:25 bgstack15
# References:
#    https://github.com/freeipa/freeipa/pull/7345/files
#    https://www.freeipa.org/release-notes/4-12-0.html
PubkeyAuthentication yes
GlobalKnownHostsFile /var/lib/sss/pubconf/known_hosts
#VerifyHostKeyDNS yes

Match exec true
   KnownHostsCommand /usr/bin/sss_ssh_knownhosts %H

# Deprecated as of v4.12.0
# assumes that if a user does not have shell (/sbin/nologin),
# this will return nonzero exit code and proxy command will be ignored
#Match exec true
#   ProxyCommand /usr/bin/sss_ssh_knownhostsproxy -p %p %h

I wonder if this is something worth reporting to the Debian package. I'm not entirely certain how the "upgrade logic" is applied; I only was researching the end-effect in the config file. I just want my ssh client to work! A brief examination of my EL8 equivalent shows it's still on freeipa 4.9, and EL9 equivalent is on 4.12.2 but there's no /usr/bin/sss_ssh_knownhosts binary and that old sss_ssh_knownhostsproxy works correctly.

So I will just distribute this /etc/ssh/ssh_config.d/04-ipa.conf to all my Devuan systems and call it a day.

#!/bin/sh
# Startdate: 2025-02-03-2 08:38
# Purpose: install the freeipa >= 4.12.0 /etc/ssh/ssh_config.d/04-ipa.conf file
sudo install -m 0644 -o root -g root /mnt/public/Support/Platforms/devuan/04-ipa.conf /etc/ssh/ssh_config.d/04-ipa.conf

References

All links in-line.