• 1 Post
  • 9 Comments
Joined 1 year ago
cake
Cake day: June 20th, 2025

help-circle

  • I approach it the same as I did with Docker, one user per container.

    I started with rootless but networking within Podman is moot with multiple users per container. And one user for all containers to get networking has to lead to subUID clashes (and thus escape vectors) - unless someone can explain how not…

    But root Podman is just as secure anyway, and easier, so I just roll with UserNS=auto and use idmap= on the volumes to enable writing as the specified user for the container. And networking in Podman works because it’s one user space. By default UserNS=auto gives 1024 subUIDs to a container. I had to up that to the expected max of 65535 for Frigate to work (example bit I use with idmap, UserNS with the size and group mapping, and GroupAdd):

    Volume=/mnt/camraz:/media/frigate:rw,noexec,nosuid,nodev,Z,idmap=uids=@1111-0-1;gids=@1111-0-1#1020-1020-1  
    UserNS=auto:size=65535,uidmapping=105:@105:1,gidmapping=105:@105:1  
    GroupAdd=105  
    

    So what’s going on here is on the Volume= I have the camera folder mounted to where Frigate wants it, and I specify that it accesses that volume with idmap= and frigate’s UID/GID 1111 and the camera group I made at GID 1020.

    The order for the idmap is @${hostUID}-${containerUID}-${num2map}. The @ symbol means that 1111 is an absolute host UID; drop that and 1111 is pulled from the containers subUID/subGID list. Most containers run as root (0) so you just map to that user.

    Side quest: For containers that have one of those mini-hypervisors like LinuxServers images and their “S6” thing, I worked around them starting with a root user but then requiring moving to a non-root user by specifying PUID=1 and PGID=1 then setting idmap=uids=@1999-0-2;gids=@1999-0-2 and reserving 1999 as username containerNameStartup and 2000 as containerName. This idmap= maps host 1999 to container 0 but maps two UID/GIDs consecutively, so it also maps host 2000 to container 1 keeping everything lined up perfectly - while LinuxServer images still don’t get host root like they so desperately want.

    GroupAdd=105 adds in the group 105 to the container, which is notably not the host group 105. UserNS is needed to complete that. This just makes that group exist in the container.

    UserNS=auto:size=65535,gidmapping=105:@105:1 accomplishes the subUID/subGID size allocation to the expected default max of 65535 (because Frigate needs it) and the group connection from host 105 to container 105. Note that this syntax helpfully uses the opposite order as idmap=. It goes ${containerUID}:@${hostUID}:${num2map} and uses colons instead of dashes.

    Every other container is cool with the default 1024 (just UserNS=auto). The subUIDs are pulled from a non-existent user named containers that you need to enable for Podman root to work with UserNS, and it has like 2 million billion or something with their recommended setup, so it’s good on subUIDs/subGIDs.

    Command to set up root Podman with UserNS=auto:

    sudo echo "containers:2147483647:2147483648" >> /etc/subuid  
    sudo echo "containers:2147483647:2147483648" >> /etc/subgid  
    

    You can also have UserNS=auto and run the container as a specific host user directly instead of “nobody” with:

    UserNS=auto:uidmapping=0:@1111:1,gidmapping=0:@1111:1

    Same idea as how I mapped in the host group 105 above. Where the root user 0 is mapped to the host (via @) user 1111 so the container runs as the user. I don’t use that because the idmap= works perfectly and then the container doesn’t even run as the user that controls the files directly. But maybe there will be a use for that sometime(?).

    And I do have a fuckton of users; Debian once complained it ran out of numbers or something after like 20 users, so I just ran the first thing I found to make the UID limit some really big number, and I never thought about it again! No idea what it was…

    **Edit: ** I cleaned it up and added examples

    Also I mostly work with UIDs not users per-say, so I specify UID 1111 has access to Frigate files. I just make a user named frigate to make it a bit easier for me. When you do ls -l it says frigate instead of 1111; life is easier.

    Overall, it has been a lot of work to divine these things because the documentation and useful examples for Podman are confusing and non-existent. I think I’ve gotten most of this idmap= and UserNS= stuff working from reading GitHub issues that drop little tidbits. But after I get it working as I want it, it’s solid, and I can now rinse-and-repeat in the future. No Docker daemon to fuck with or get hacked, and it’s reliable since I just let systemd start these quadlets up.

    I’m not 100% done, but I’m close. Outstanding issue right now is that on server restart sometimes some Podman networks don’t actually start up correctly and then containers that want them fail to start because of that (I have to restart the Podman network via systemctl to get them to exist in Podman). But eventually I’ll find the stupid GitHub issue where someone mentions something relevant for that, and then I’ll be golden!




  • I have this setup. Upfront, I would not recommend Proxmox, the update methods are annoying. The better way is straight Debian with Incus installed, then you get straightforward stable Debian updates automatically - they won’t break anything and you’re secure. Sometime I’ll redo it - I haven’t because, of course, it is my router and when its down I don’t have internet! So foreboding and on the back burner.

    Also also Proxmox’s GUI leaves a lot to be desired (for me, it looks like ass and is confusing), Incus is nicer for VM control and Cockpit is nicer for host control. After typing all that I realize I’m a hater at this point

    I haven’t really noticed downtime issues cause of Proxmox updates cause I just do it when nothing is happening. And Proxmox hasn’t bricked itself, though I am wary of it because that has happened to others due to their rolling release update style.

    I’ve got a Dell Wyse 5070 Extended with a 2 port Intel NIC in it. I pass both ports through leaving the built-in port for managing Proxmox.

    Here are my notes:

    Set NIC PCIe Passthrough for Network Card

    nano /etc/default/grub

    • Edit this line by adding intel_iommu=on to get

    GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"

    update-grub

    nano /etc/modules

    • Add these lines
    vfio  
    vfio_iommu_type1  
    vfio_pci  
    vfio_virqfd  
    

    update-initramfs -u -k all

    reboot

    Click on 2nd level thing named router on the left side vertical bar hierarchy thing and then click in the top right the blue Create VM button.

    • General tab
      • Name: OPNsense
      • Start at boot: checked
      • Start/Shutdown order: 1
      • Startup delay: 15
    • OS tab
      • Use media: DVD version (usb might work) of OPNsense.iso
    • System tab
      • Machine: q35
      • Bios: OVMF (UEFI)
        • Storage: local-lvm
        • UNCHECK Pre-enroll Keys (HATE)
    • Hard Disk tab
      • Disk size (GiB): 15
      • Discard: checked
      • SSD emulation: checked
    • CPU tab
      • Cores: 4
      • Type: host {makes it not moveable between diff CPU types but will theoretically allow for more speed}
    • Memory tab
      • Memory (MiB): 2048
      • Minimum memory (MiB): 512
    • Network tab
      • No network device: checked
    • Confirm tab
      • Do not start on creation
    • After creation, go to Hardware tab in the 2nd left vertical list on the browser page and click add
    • Click PCI Device
      • Device: ...01:00.0 I350 Gigabit... & ...01:00.1 I350 Gigabit... (1st & 2nd ones)
      • PCI-Express: checked

    Go to the Console tab in the 2nd left vertical list on the browser page and hit enter to get to a command line in the OPNsense VM

    !Add expand storage via command line!

    And lastly, during setup I have these notes

    It will choose wrong (WAN gets igb1 and LAN gets igb0 -> we want WAN gets igb0 and LAN gets igb1)  
    Default User: root, PW: opnsense (they don't tell you anywhere, you don't have internet b/c this is your new router, fuck em)  
    **Access at 192.168.1.1 via pluging an ethernet cable into the 1st port in a set of forwarded ports**  
    *Note that we will move it so the 1st port is the WAN (can't access OPNsense from the WAN port for safety), so after following this you access via 2nd port*  
    

    So watch out for those things. Not sure quite what I mean by the 1st and 2nd port things, may be related to on setup it had the order of the ports I wanted wrong so they’re switched till setup is complete and it reboots.

    I don’t remember doing this at this point, but maybe this info dump will help!



  • I do see Authentik can apparently act as a reverse proxy, so it can sit at the very front. But I’d lose out on caddy + crowdsec then…

    I’ll have to do some reading if caddy can actually just forward to caddy, and where the TLS is terminated and all that.

    And I do use the internal setting now, but I need it off if I want to publish the port on the LAN so that the VM can see the ports on the LAN. But if I can have WAN caddy do the auth check and forward along good stuff to the LAN caddy, then I’ll only need to publish LAN caddy’s port and that’s not the worst at all.

    Thanks for the ideas, I’ll try to cook “caddy (DMZ) -> auth OIDC (DMZ) -> caddy (LAN) -> services (LAN)”!




  • Containers lower the bar since the developer doesn’t need to make their program work on every system - just the container’s system.

    Price we pay for more programs. And they bring boons like read-only, rootless, limited capabilities, and constrained perf limits (esp if you use Podman with Quadlets).

    And don’t feel trapped - the Dockerfile is a recipe to build that program. Probably want to do it in an LXC container since it’ll want to use /data for its data or something. But the LXC container can also be run as a user but the program thinks it’s root. Plenty of security abounds!

    I think it’s worth the price and you’re not trapped. They’re trapped with you and your robust Quadlet files