It’s been a while! Let us know what you’re currently working on, what exploded, what cool stuff you found, or just brag that everything is running fine unattended.
I haven’t been doing a lot, renovate auto upgrades for patches and sometimes minor updates work great so far. I got involved in my union to spin up a mastodon server soon, so there’s a new project, but I already run mastodon so it shouldn’t be much of a deal, I just need to get comfortable with the infrastructure. I also set up ntfy but only sent some test notifications and didn’t configure any app yet - planned for Element Chat.
[Meta] I’m not sure how to apply the new tags. In the recent versions of this thread, people sometimes posted about their ollama setup or the like, but I’m not mentioning AI in the post body (except this meta thingy) and it shouldn’t be much of the discussion, but it’s an open post. Maybe curbstickle can chime in to let me know if I should apply one of them.
You must log in or register to comment.
I found the best thing to deal with a device that won’t work…
I’m trying to get the 3.5" adafruit touchscreen working and it’s being stubborn…
So… I went outside and split a large pile of logs.
Problem still not fixed, but I feel a hellofalot better
Not working on anything particular, but I’m going on a road trip on Tuesday to this regional airport that is giving away two racks perfect for my use case. Deep enough for my larger machines, sound proofed and 42U tall.
Finally my hardware can have a proper rack instead of being in a stack/pile in this harrypotteresque locker under my stairs.
Sound proofed… wow… does that also mean heat can’t escape?
Vent fans at the top and the bottom. But it can be next to my work desk without the hardware getting annoying.
Ah… nice
Giving away? As in free? Luckyyy
Free except from the drive to get there. But the drive is a lot cheaper than those racks would cost.
Not to brag but everything’s working fine, clean as a whistle. Upgraded forgejo to the latest version. I’m still helping maintain the awesome-selfhosted list, and get inspiration from selfh.st a lot (big shoutout!) Hope to make sure some postgres dbs are up to date at some point, all of my dockhand instances that i set up have been a big help :)
I set up oauth for my sites
[AIT] I know this isn’t everyone’s cup of tea, but I’m excited about Deepseek V4 flash. It’s (for me) the perfect size and architecture to self-host an LLM.
My box (and brain) are chugging through a queue:
-
Figure out why my swap is going crazy, and how to ban processes from it [Done].
-
Figure out why Code OSS is unhappy [Partially Done].
-
Make an ik_llama.cpp iMatrix for Deepseek V4 [Done].
-
Figure out why quantization isn’t working [Done].
-
Make a test IQ2_KL/MXFP4_R8 quant to see how it does squeezed onto my box [in progress].
-
Test. Tune. Inevitably troubleshoot the dozen other things that go wrong. Figure out how much spare RAM that leaves me.
-
Make a higher quality IQ3_KT quantization. This will take all night on my CPU.
-
KLD test both of them vs the full precision, to quantify quantization loss. Likely an overnight task, too.
-
Try merging the new model release with the base model, 50/50, for a less “deep fried” model. imatrix, quant, test.
The goal is to host it on a single RTX 3090, Ryzen 7000 with 128GB RAM, for anyone curious. Though I may try smaller models too, like Laguna S1.
Deepseek V4 flash
What are the advantages of Deepseek V4 flash?
-
It’s just under 300B, trained at FP4; absolutely the perfect size for servers with 128GB-192GB CPU RAM to spare.
-
Its fast. I’m getting 17 tokens/sec on a single RTX 3090 GPU, all experts offloaded to RAM; for a 300B model this smart, that’s crazy fast.
-
Its attention mechanism is cutting edge, good for long context without too much processing time.
-
The benchmarks for coding/agenic usage are absolutely bonkers, within margin of error of frontier models or Deepseek Pro in some cases: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731
-
…Though I don’t put much stock in benchmarks. And I haven’t tested it enough to tell you if it lives up to that hype for specific use cases.
-
Deepseek also publishes its base model. That means I can “unfry” the model with a merge if I have to.
I am afraid of the the model being “overfit” to coding and agenic stuff.
For reference, my previous favorite model was Xiaomi MiMo V2.5 310B. It benched well, but it also feels “smart” outside of benchmarks, like in knowledge of trivia without tool usage/internet access or comprehension of weird questions.
Sweet.
-
-
spent some time last week adding metadata to my mp3 music obtained with yt-dlp. used musicbrainz Picard.
I’m actually really surprised how comprehensive that database is, and how easy it was to manually tag the tracks that didn’t get picked up automatically.
I finally got my immich docker running again. I lost all my meta data in the process but oh well, it works again.
what happened to make it stop running?
I think my Postgres docker became unsupported. I think Immich needs Postgres with vector to run and mine did not have vector. I tried to save my data but blew it up in the process. Nothing of great value was lost and I keep all my images elsewhere in my nas for just this reason. I do kinda wish Immich could just look at a directory rather than keeping its own file structure.
immich supports external libraries, but i’m not sure if you get all the functionalities when using them. i just got started with immich so i’m still learning the ropes. currently debating whether i want to migrate my photo archive to immich or just leave it as an external library.
Oh rad. I did not know.
got a busted Lenovo Ideapad 520ISK out of a dumpster, missing screen but otherwise perfectly functional. the thing has i5-7200u, 8 GB, two SATA ports inside and this prompted me to look up converting the thing into a server, specifically figuring out how to run 3.5" drives off it; I’m very anti-friendly towards using USB solutions for such a task.
searching was arduous, both web and yotube, and except one lonely forum post (remember those?) didn’t find anyone attempting let alone succeeding in running it thus. the 2.5" and 3.5" connectors are identical (there are micro SATA to SATA adapters for the DVD port) but the drives will need 12 V that the 2.5" ports ain’t got.
what I’ve concluded is that all that’s needed is a M/F SATA extension cable which I have trouble sourcing locally; the power will come from a separate ATX PSU for the time being, although I’m sure I can pull 12 V from somewhere on the board.
don’t got no 3d printing nor fabricating options, so it’ll likely look super-janky, but that’s not a huge concern for me now. I’ll do a write-up if I succeed.
I have been on vacation and not a single alert has come through. I count that as a win.
[AIT] Well, let’s see…nothing really new on the server. Still waiting for something interesting besides *.arr. I did provision a new tower with the following specs:
- CPU: Ryzen 5 5600X
- GPU: ASUS ROG RX 580
- RAM: Came with 32 GB Corsair Vengeance RGB DDR4 / dropped another 96 GB in it.
- SSD: WD Black NVMe M.2 iirc it’s about a 1tb
- Asus motherboard
- Working on acquiring a GeForce RTX 5090 32GB GDDR7 mounted on a OCuLink external dock (tentative - may change)
This, obviously, will be an local AI machine
Nice. Did you have to sell a house to afford the parts?
LOL Not quite, however, I am choking on the 5090 but as I understand, that’s the better option, and 32 GB seems to be the sweetspot for AI. I actually picked up the tower and original RAM for less than $500 USD.
First of all, I mean zero offense with any purchase decision. A 5090 is very good.
…But if I were paying that kind of money, I’d probably get a 4090 and a new motherboard/CPU instead. Maybe a used DRR4 threadripper system.
Hybrid (CPU + GPU) inference is where it’s at these days. It opens up a whole world of huge MoE models, whereas on an 5090 you are stuck with Qwen 27B.
Having a fast CPU, with lots of RAM channels, with full PCIe bandwidth is much more important for that than having a 5090, where a 3090 or 4090 will get the job done.
Even if pure speed is your primary concern, you can tune a sparse 120B model (like Laguna) to run almost as fast as Qwen 27B on a 5090, and get at-least-good results.
It’s more finicky and involved, though. For sure.
Running an LLM on a 5090 is a task. Hybrid CPU + GPU inference is a hobby.
Hybrid (CPU + GPU) inference is where it’s at these days. It opens up a whole world of huge MoE models, whereas on an 5090 you are stuck with Qwen 27B.
Why you do this to me? LOL Injecting your options. But seriously thank you for the advice. I’m really green in the AI arena. So I’m trying to feel my way around, trying not to spend money on equipment I’ll regret later.
Be aware that you pay a significant performance penalty for going over occulink.
The hit modest for fully offloaded dense models (like Qwen 27B), but dramatic for hybrid inference of big MoEs.
Even my old 3090 got a noticeable performance gain going from a PCIe 3.0 x16 riser to a PCIe 4.0 one.
Be aware that you pay a significant performance penalty for going over occulink.
And that’s why it’s tentative. The reason I’m leaning towards the dock is that, while the 5090 will fit, physically in the case, I am worried about heat dissipation. Even now, when I fire up a moderate model and run it through some paces, I can see the temps rising fairly quickly. So, if I’m going to drop that kind of cabbage on a 5090, the one thing I don’t want to do is watch it turn into a shrink-i-dink.
Duct it!
I have a 400W 3090 with zero case fans.
Its always cool, because I bought like $10 of weather stripping to “seal” its intakes against the edge of an SFF case. So it’s always sucking in ambient air:
You can also severely undervolt a 5090 to like 250-300W, with very little performance impact. Honestly its stock speeds are kind of crazy.
…But again, you aren’t gaining much over a 4090. A 4090 + DDR4 threadripper would be way faster than a 5090 + occulink, for many reasons. And TR CPUs are pretty reasonably priced compared to GPUs these days.
But even if you do go 5090, I’d highly recommend finding some way to shove it in the case and duct some air into its intakes. Its going to be way faster on a PCIe slot.
You could even get a riser and mount it somewhere else in the case, theoretically.
Shit I deleted my post. Yes I have been enjoying some greenery so you just have to hear me out. I have an old countertop ice maker that stripped it’s gears. The refrigeration loop is fine. Pretty wild how small it is. I was wondering…what if I provisioned some brackets on which to mount a muffin fan of unknown CFM, to blow across the coils thru several sheets of mesh of varying gauges to collect moisture drained off to an unknown location, and duct that indirectly into the computer, like a miniature Liebert server room cooling unit. I worry about the moisture tho. Constructing a desiccant wheel seems improbable. I could reheat the cool air upstream or mix warmer air in with the stream to so the outlet air doesn’t stay at a very low dew point and “rewet” downstream components. Perhaps with a small filament heat coil. HMMmmmmm
Unfortunately condensation is, indeed, an intractable issue there.
Sometimes people put a heat pump in water cooling loops, where condensation along the pipe can be mitigated with insulation, but even that’s a little risky with a 5090 these days.
Enjoy your lovely greenery!
Shit I deleted my post. Yes I have been enjoying some greenery so you just have to hear me out. I have an old countertop ice maker that stripped it’s gears. The refrigeration loop is fine. Pretty wild how small it is. I was wondering…what if I provisioned some brackets on which to mount a muffin fan of unknown CFM, to blow across the coils thru several sheets of mesh of varying gauges to collect moisture drained off to an unknown location, and duct that indirectly into the computer, like a miniature Liebert server room cooling unit. I worry about the moisture tho. Constructing a desiccant wheel seems improbable. I could reheat the cool air upstream or mix warmer air in with the stream to keep the so the outlet air doesn’t stay at a very low dew point and “rewet” downstream components. Perhaps with a small filament heat coil. HMMmmmmm
deleted by creator
Finally think I got everything installed on my Raspberry Pi 4 that I want there, thankfully.
I’ve been trying to document my journey with selfhosting this time better than previous attempts, as far as writing down what worked and didn’t work. It’s coupled with me trying to write little explanation guides for my partner about what service does what, and how to use them. They’re really excited to actually use them, but I want to get the guides done first, which has been a challenge.
Kinda surprised Grimmory is taking up so much RAM but if it becomes an issue I’ll put it on my other, beefier server. It was the last service I was struggling with, because every time I would upload a book, something would go wrong. An example being that one time I uploaded a book and it went into every single folder after, then deleting one copy deleted all the copies.
Admittedly, I’m not terribly well experienced so it was probably an issue on my end. Still learning!
Speaking of, my next project to tackle is overhauling my more powerful server and what it’s running. Since I use a split setup where I store media on my NAS, I’ve had difficulties with databases hating that aspect. Now that I know a bit more than I did when I started, and I have nowhere else to ask (thank you for having this community, by the way, it seems a lot more newbie friendly than some others) I was wondering if I could map the database of a service to be on the server itself, while having the media mapped to my NAS? From my research, I think this is possible, but I’m not 100% sure.
Hope everyone else’s experiences are going well!
Check out BookOrbit if you find Grimmory too hefty in the RAM reqs. It uses very little RAM for me. I think it also has import from Grimmory as well.
As for the database Q, if you’re using docker/Podman it’ll be very easy to point the database folder (if included in the image, like Jellyfin does) to your local filesystem and the media to your NAS. Same idea if the container uses a Postgres container, just spread across two containers in that case.
First, thank you much for suggesting BookOrbit! I actually just installed it because I was having a lot of issues with Grimmory making dupes of books in every folder and couldn’t figure out why it was doing it. Haven’t had any issues with BookOrbit and I really like that it’s got achievements. Plus, the less RAM usage is nice.
I’m using docker and docker compose files! Thank you for answering, the last time I took a crack at selfhosting, I really felt lost and screwed a lot up, so I wasn’t sure if that was even possible.
The docker compose file is great, clear once you get used to all its little sections. For volumes I only use what they call “bind mounts” which are where you have a folder on your system connected to the folder in the container. As opposed to the “docker volumes” which are internal to docker.
And with those it’s super easy to just be like in the volume subsection:
- /path/to/local/drive:/container/database:rw,noexec,nosuid,nodev,Z- /path/to/network/drive:/container/media:rw,noexec,nosuid,nodev,ZThe
:rw,noexec,nosuid,nodev,Zat the end is a great extra security thing that never causes problems. rw means read-write, you can switch it to ro for read-only if you’ve got something you want the container to only read from but not be able to modify. I use that for jellyfin’s media since I don’t want it doing anything but reading it. The noexec means don’t let executables be run from the folder, never should happen so it just prevents a sick hack from being put in the folder and run. I forget what the others do but they’ve never been a bother. And Z means only one process can access the volume, you can switch it to lower case z to let multiple processes access the volume - and I’m not sure it does anything without SELinux going which I think only fedora does by default right now.Enjoy the info dump!
Thank you again! I actually really appreciate the infodump and it was well explained. I’ve made notes on this in my selfhosting documentation so I can reference it later! I didn’t even know the difference between “docker volumes” and “bind mounts” really.
Discovered you can self host Joplin and I can’t wait to set it up in the next days.
Also I tried some alternatives to ActualBudget, because I didn’t need to many functionalities. Long story short, I went back to it cause none offers a neat interface and multiple accounts at the same time.
https://lemmy.world/post/49663813
My old post from my old account.
Since then, I’ve been moving my PC (not server) from arch to nix, and discovered that I was relying on fundamentally broken DNS resolution behaviour on netbird.
Fixed that and also changed my Caddy config to use FQDNs as well.
Apart from that, added Gollum as a wiki server, and the experience has been perfectly amazing!
Still waiting to get my own mini-PC and use that, current shitbox is a desktop in my college.
After a lot of trial and error, I managed to get rsgain working with Lidarr on my raspberry pi. I might actually be fully satisfied with my music piracy/streaming setup now (until I find the next silly addition :3 ).
Mealie, recipe manager and Social to Mealie. Scrapes recipes from instagram reels, tiktok, youtube shorts.
Bar Assistant, bar and cocktail management. You can add your ingredients and it shows what can you make, you can share a menu to your friends, its cool
I’m really enjoying mealie. I just wish there was an easier way to input nutritional information
