I need a kubernetes cluster with high availability, load balancing and horizontal pod autoscaling, because that is something I want to learn. I don't care that it's just for wife's home-made dog collars webshop.
I don't get this; a Pi isn't even in the same conversation as an old rackmount server you can get for free. You couldn't stuff half the compute, ram and storage into a Pi or a dozen Pis for 10X the cost of grabbing something off eBay for a hundred bucks.
That's if the Rpi Foundation is deigning to let us peasants even buy them these days.
I have an old rackmount server I got for free. Dual Xeon X5650s, 192GB of RAM, four 8TB HDDs, and a pair of 250GB SSDs. I can only use it in the basement because it’s too loud to run anywhere else, but even then, it’s currently off because it trips its circuit breaker under heavy load.
A power strip full of Pis in a k3s cluster doesn’t do that. I used a 2GB model 4 for the control plane and 3Bs as the workers.
If it trips circuit breakers it's using a terrifying amount of power. Honestly with energy prices I am starting to think old stuff is actually becoming a bad value in some cases.
Yeah that's basically it for me. I have a collection of dev boards, old hardware and stuff other people were tossing out set up for a variety of purposes (Kubernetes clusters, two build farms, network boot, etc.). None of it is because I feel I "need" any of that for self hosting. In practice two old desktops with a bunch of drives would be perfectly capable of providing everything I need including redundancy. I have all that stuff because I'm learning and experimenting.
Switched from a raspberry pi 3 to a second hand x86 thin client (lenovo thinkcentre m920q) because raspberry pi 4 were not available at the time. Made me learn proxmox and a bunch of other cool stuff my raspi couldn't handle.
I'm rooting for ARM / RISC-V to become more popular in desktop computing / servers though.
Apple Silicon Macs do a great job with virtualization. Outside of them there's just no nice high end hardware that's well suited for something like proxmox. It's either low end SBC, or the hyper proprietary ARM servers that I don't think we can even buy.
Modern Android phones include a hw-accelerated hypervisor. In Android 16, there will be a feature to run a full Linux VM through what Google calls protected Kernel VM (pKVM).
Qualcomm has their own implementation called Gunyah
There are some Raspi competitors offering SBCs with RISC-V chips, there is even a RISC-V Mainboard for the framework laptops, but the last time I checked they sadly didn't reach the performance levels of comparable ARM chips.
I did similar once the pi4s were hard to get and expensive. A used x86 mini pc was cheaper and magnitudes more powerful. It runs all my server needs. I’m a simple person: homebridge, plex server, retro game library.
Ive had a slightly higher failure rate with the Dells, but the sample size is too small to be relevant.
The Lenovos more often than others ive found outfitted with a dGPU which comes in handy in some scenarios, but I think that comes down more on which enterprises more often purchase Lenovos and want the dGPU, and that its just what ive come across in the used/decommissioned territory.
Yeah, I expect there is a compromise somewhere but 10+ year old stuff which I can easily find on eBay probably just isn't worth it for anything that is going to be on most/all of the time. Better to go for more modern lower end stuff to use less power. 250w will cost you like £50 a month to run
A i5-6500 has a TDP of 65W while a i5-13600K has a TDP of 150W.
If you get something modern that has the performance of a i5-6500 it will be a little bit more efficient. The key is that more performance uses more power.
TDP ≠ power draw. TDP is literally the Thermal Design Power aka what is the amount of thermal load a system designer should account for. Yes it can give you a rough and dirty idea of maximum power draw, but real world power draw can be entirely different because that depends on load.
For example, if your i5-6500 runs at 50-70% load while the newer processor only runs at 20-30% load due to IPC and instruction improvements the newer processor might very well use less power over the course of month than the older one despite the newer one being capable of drawing more
You're also comparing a 4c4t part to one with 14c/20t not to mention comparing a mass market part to a gaming specific part. The 6600k (which is targeting the same market segment as the 13600k) has a 91w TDP. Go compare your 6500 to the i5-13500 except again it's still comparing apples to oranges when you just look at raw specs and TDP ≠ real world power consumption
Yes, but also no. Older hardware is less power efficient, which is a cost in its own right, but also decreases backup runtime during power failure, and generates more noise and heat. It also lacks modern accelerated computing, like ai cores or hardware video encoders or decoders, if you are running those appd. Not to mention lack of nvme support, or a good NIC.
For me a good compromise is to recycle hardware upgrades every 4-5 years. A 19 year old computer? I would not bother.
Yeah what I’ve always done is use the previous gaming/workstation PC as a server.
I just finished moving my basic stuff over to newer old hardware that’s only 6-7 years old, to have lots of room to grow and add to it. It’s a 9700k (8c/8t) with 32GB of ram and even a GTX 1080 for the occasional video transcode. It’s obviously overkill right now, but I plan to make it last a very long time.
This is why rack mounts were made. Hell, I've seen a lot of custom builds where people have mapped out the server on their wall and it takes up no floor space. Something like this: https://i.xno.dev/kG9Wx.jpg
A rack takes up as much space as a fridge though, and mounting things to the wall is risky. You better make sure you really got it into the stud in the wall. Also, don't do that if you live in an earthquake zone.
Only full size racks. You don't need to buy a full size rack. You can get very small racks these days that are smaller than a little chest cooler. And why are you under the impression that you have to mount it on the wall?
Correct. If space is such a big problem for you that it's unconscionable to use a 4U mini rack (which again, like what the fuck), then mounting hardware on the wall is a completely valid option. It'll take up zero floor space.
That's what Iam aiming for at the next hardware update. I don't have the space for a server rack and a SFF desktop would also not fit into my home, so a miniPC it'll be. I cannot wait to move to x86.
I'm not sure if I'm alone in this but I have a terrible aversion to transcoding. I know the loss of quality is probably not that huge (depending on the original codec) but I just can't bring myself to get past it.
As a result I have a tiny arm based box with a 2tb SSD and I'm happy out.
I think the pi zero might have a place, don't have one but at least it is actually cheap. Used an old laptop before, now running stuff on my PC which I feel was a bit of a mistake in a way but it started as one process and more appeared over time.
At some point would like to move it into a VM tbh, then I could copy the VM to a mini PC at a later date. Or easily copy it when reinstalling the OS on my main PC. VM for convenience and separating it from my general PC usage.
Pi zero 2 runs my home automation smoothly and few of it's sisters are in a desk drawer in case of equipment failure.
It has quite enough oomph to run Home Assistant with zigbee hat.
Nah, they are underpowered for their current price.
When you could get your hand on a raspberry pi 3 for 35€ 10 years ago it was a bargain. But now it's not the case.
Honestly Raspberry Pis are pretty underpowered as hosts for more than a handful of super basic services, and given they consume 20-30w at minimum you're easily getting into used office desktop territory where you can get a ton more performance right out of the gate.
The real value in the raspberry pi is in the GPIO and the cohesive ecosystem of accessories to plug into said GPIO. You can do so many cool automations and controls using just an RPi (especially if combined with something that can't be accomplished more cheaply and easily with an ESP32) but as a server host they're pretty crap in comparison to a decade old business PC off of eBay
I've found that a pi is good enough, computationally, but not reliability wise.
A lot of things like advanced light control goes through my host, so any lockups or crashes are bad. My pi held up for about 18 months before it began to play up. I've found a small NUC system has higher reliability for the same price and power usage.
Even just the choice of programming language makes a big difference. Running a JVM language or NodeJS, Python, Ruby etc., you can be bottlenecked by a Pi. Meanwhile, Rust or C/C++ will use barely a fraction of those resources.
See, I don't pay for the electric bill to keep my collection of old enterprise equipment running because I need the performance. I keep them running because I have no resistance to the power of blinkenlights.
My home server is literally made from garbage left over from other PCs. The motherboard is currently some piece of junk from a prefab PC with a custom power socket, so I got to make my own adapter from scratch.
I spend all day at work exploring the inside of the k8s sausage factory so I'm inured to the horrors and can fix basically anything that breaks. The way k8s handles ingress and service discovery makes it absolutely worth it to me. The fact that I can create an HTTPProxy and have external-dns automagically expose it via DNS is really nice. I never have to worry about port conflicts, and I can upgrade my shit whenever with no (or minimal) downtime, which is nice for smart home stuff. Most of what I run tends to be singleton statefulsets or single-leader deployments managed with leases, and I only do horizontal for minimal HA, not at all for perf. If something gives me more trouble running in HA than it does in singleton mode then it's being run as a singleton.
k8s is a complex system with priorities that diverge from what is ideal for usage at home, but it can be really nice. There are certain things that just get their own VM (Home Assistant is a big one) because they don't containerize/k8serize well though.
Yup, same here. being able to skip all the networking and DNS hassle and have it automated for you is so nice.
Having databases fully managed with cnpg is AMAZING
I just have renovate set to auto update my argocd, so everything just runs itself with zero issues. Only the occasional stateful container that has breaking changes in a minor version.
If something OOMs or crashes, it all just self heals, I never need to worry about it. I don't have any HPAs (nor cluster scaling obv), though I do have some HA stuff set up just to reduce restart times and help keep the databases happy.
The main issue with Kubernetes is that a lot of self-hosted software makes bad design decisions that actively make kubernetes harder, eg sqlite instead of postgres and secrets stored in large config files. The other big issue is that documentation only supports docker compose and not kubernetes 90% of the time so you have to know how to write yaml and read documentation.
Moving my hass from a statefulset to kubevirt sounds tempting. Did you have better reliability/ergonomics? I have been looking into moving my Hass automation to NodeRed, so that I can GitOps it all, since NodeRed supports git syncing.
The only problem I've had with Raspberry Pi is that some apps want to write a lot of stuff to "disk", and the default "disk" on a Pi is a MicroSD card which dies if you keep writing things to it. Sure, you can always plug something into a USB slot, but that adds a bit of friction to the whole process.
Oh, also, I wish it were easy to power a whole bunch of Pi units. Each one needing its own wall wart is a bit annoying, and I've had iffy results using weaker, less steady power supplies with multiple ports intended for things like phones.
Yeah, but then you have to get a kind of case that can handle a Pi plus that hat. It's a good idea, it's just a bit more fiddly than just the typical booting from the SD card and doing everything that way.
The amount of writes required to kill an SSD aren't going to be seen in the real world on a timescale of less than 10 years unless you're really doing something wild that you shouldn't be.
An SD card might fail after it's full capacity being written a handful of times, SSDs can survive that several hundred times over. Seriously look up the terrabytes written specs for various storage mediums and calculate out the daily amount of writes. Oftentimes with SSDs you'd have to literally write a terrabytes of data a week to actually see a problem
Not within the computer's lifetime. Consumer-grade SSDs are generally rated for 3000-5000 write cycles or more, and contain some kind of wear levelling mechanism to distribute write operations over the entire physical medium to reduce the chance of individual block failures. The first SSD I ever bought is still going strong as my server's root filesystem.
@Tangent5280@spongeborgcubepants I think the days of worrying about SSDs failing like that during its expected lifespan are over for all practical applications.
Most SD cards aren't really suitable for the kind of workload an operating system generates (that being mostly random i/o). Make sure to get a reputable A2 (application class 2) rated card, they aren't that expensive but perform way better.
Raspberry Pi themselves launched a card recently, I haven't tried that one but it's probably a good choice too.
I think the Raspberry Pi Linux releases mount things onto a ram drive, so the typical IO doesn't touch the SD Card. But, if you run another OS (which sometimes is the easiest way to get other software running) it tends to just treat the SD Card like an HDD/SSD.
That's definitely not true, Raspberry Pi OS works and acts like a normal Debian installation per default - with root mounted rw and all.
Other than that, there isn't much "treating like an HDD/SSD" going on, it just writes to flash when an application requests it does. If the underlying storage is an eeprom, an sdcard nvme storage doesn't really change anything here.
Docker is so bad. I don't think a lot of you young bloods understand that. The system is so incredibly fragmented. Tools like Portainer are great, but they're a super pain in the ass to use with tools/software that include a dockerfile vs a compose file. There's no interoperability between the two which makes it insurmountably time-consuming and stupid to deal with certain projects because they're made for a specific build environment which is just antithetical to good computing.
Like right now, I have Portainer up. I want to test out Coolify. I check out templates? Damn, not there. Now I gotta add my own template manually. Ok, cool. Half way done. Oops. It expects a docker-compose.yml. The Coolify repository only has a Dockerfile. Damn, now I have to make a custom template. Oh well, not a big deal. Plop in the Dockerfile from the repository, and click "deploy." OOPS! ERROR: "failed to deploy a stack: service "soketi" has neither an image nor a build context specified: invalid compose project." Well fuck... Ok, whatever. Not the biggest of deals. Let me search for an image of "soketi" using dockerhub. Well fuck. There are 3 images which haven't been updated in several years. Awesome. Which one do I need? The echo-server? The network-watcher? PWS?
Like, do you see the issue here? There's nothing about docker that's straightforward at all. It fails in so many aspects it's insane that its so popular.
Maybe I’m an idiot (high possibility) but I couldn’t even get the instance of homebridge docker to work. However, it had zero problems running on hypervisor. You aren’t alone.
I'm coming to appreciate Hyper-V more and more to be honest. It's a very mature virtualization environment. The only issue I have with it is the inability to do GPU-passthrough. Once they figure that one out, I probably won't bother with anything else.
So close. Started on raspberry pi. Went for a cluster with dpckrt swarm. Finished with a nas and a 10years old game computer as a mediacenter. (That the electricity bill whoch made me stop the cluster)
It's basically a Kubernetes cluster, which you can run locally on your PC. Really useful for playing around with Kubernetes before you move to a 'proper' environment.
Absolutely the best way to learn though. The number of places I've walked into that had no clue about containers or even a vpc and thought Google drive was an API is too damn high.
I have actually had to write something that used the Google drive API for a friend's company once and it was... Unpleasant. Counterintuitive. Woefully inconsistent. My solution worked but it sucked and I am a bit ashamed of it
Do you run Docker in a VM or on the host node? I'm running a lot of LXC at home on Proxmox but sometimes it'd be nice to run Docker stuff easily as well.
It's kind of redundant but I run Docker in minimal debian LXCs. I like the speed of LXCs but I still love the reproducibility of Docker so I combine them lol. I do run regular VMs with docker for systems that are doing more than one thing
Thanks for the tip, I'll have to give that a try this weekend. I assumed it would require a full VM but never thought to try it in LXC, nice if it's that simple and I now feel kind of silly for asking haha. And the reproducibility and ease of deployment is indeed a big plus of Docker compared to LXC. It would be nice if Proxmox could add native integration for Docker at some point.
One thing that's super important is to enable container nesting in the LXC settings. You can't run docker otherwise. (Besides that one setting, it's definitely as simple as that) And very much agreed @ the docker support in PM. That'd be the dream.
It doesn't work for me since all of my network equipment is 19" and there's no point in having two racks but having a 10" standard is still a great idea!
Wait, you can host a website on a raspberry pi !?
But is it really cheaper than shared hosting, for instance? And even then, quality-wise, it cannot be that good, can it?
You can host a website on a lot less, even. But it entirely depends on what you're hosting and the load. Basically anything can host a bunch of static pages, so if your site is just that, basically anything will do. You could probably even do a WordPress site with the right caching plugins and serve a reasonable amount of traffic. The first limit you'll hit realistically is your uplink, not your webserver CPU.
@MrMobius@hanke depends what you're doing. A Raspberry Pi can be surprisingly powerful. Can run NAS, websites, proxies, things like NextCloud, PiHole.
Cheaper than paying someone for hosting a small server? Maybe. But there's pros and cons of both.
Do you run Docker in a VM or on the host node? I'm running a lot of LXC at home on Proxmox but sometimes it'd be nice to run Docker stuff easily as well.
You're correct that nesting namespaces is unlikely to introduce measurable performance degradation. For performance, I was thinking mostly in the nested virtual network stack adding latency. Both docker and lxc run their own virtual interfaces.
There's also the issue of running nested apparmor, selinux, and/or seccomp checks on processes in the child containers. I know that single instances of those are often enough to kill performance on highly latency sensitive applications (SAP netweaver is the example that comes to mind) so I would imagine two instances of those checks would exacerbate those concerns.
Thanks for the tip, for some reason I assumed I couldn't run docker in LXC but never actually tried... I prefer to avoid the overhead of a full VM and I find LXCs way easier to manage from the host system. Guess I'll have something to test this weekend. Cheers!
I have been in for a couple months now, Proxmox cluster with two machines.
Self built pc that was my daily driver for a while, rtx 3080ti 32gb ram, ryzen 7 3700x, runs the heavy stuff like a Mac VM, LLM stuff, game servers
Rando open box mini pc I picked up on a whim from Bestbuy, Intel 300 (didn't even know these existed...) with igpu, 32gb of ram, hosts my dhcp/dns main traefik instance and all the light services like dozzle and such.
Works out nicely as I crash the first one too often and the DHCP going down was unacceptable, wish I got a slightly better cpu for the minipc but meh, maybe I can upgrade it later.
Certainly tempted by one, any particular ones to look for? UK here, currys out of stock on an MSI one and Amazon is full of names I have never heard of and half the time it comes with Windows pro which is just a waste of money buying with the hardware.
I got the ASRock motherboard with the n100 cpu and built a PC with it. Right now, I just have it running two 3 TB hhds in a zfs mirror and it's working nicely.
I wanted to get a raspberry pi at first but it was stupidly expensive in my country and, for what it offered compared to a normal x86 system, it was not worth it.
I'm actually just about to start up my server again on a rp4. It's been like 5 years since I've used it. Is dietpi still the best way to go about making a Plex media server/bare bones desktop environment that I can access with 'no-machine'?
I sear no machine just broke my autoboot setup one day and I never got around to fixing it. What do you nerds think?
I'm not interested in video streaming, just hosting my music collection and audiobooks. I remember FTP being a pain to transfer music files from my phone
I bought a decade old Z840 and it's great for VMs, Plex, Arr stack, and a few other services but it is so overkill with 2 GPUs. I think what I should've done was buy a couple of used desktops or laptops to expand the my homelab as I needed.
I ran lots of containers on a Pi 4 but recently purchased two cheap Chinese mini PC's with 16GB RAM and an SSD. They're so much faster and only a bit dearer than a Pi. I run Proxmox on both.
Absolutely nothing wrong with the Pi though. The Pi 4 lives on with a USB drive attached. I have NFS configured on it to backup my Proxmox VMs to it. It also hosts all the media for Jellyfin.
And yes, pi’s are waaaaaay overpriced now. Get a used mini pc with a 6500T in it for server needs. It’s more than enough and maxes out at 65W. Most of the time it idles near 12W.
Yes, you can optimize a lot. Especially with Linux. I did the same and even started to replace program that did too much, bloated, with my own programs. To speed up the development I did it with AI and Cursor.
It's just fun to play with, there is no "need".
Yeah, I enjoyed my time with k3s setup at home as well, but right now I don't really want nor need that 😄
I need a kubernetes cluster with high availability, load balancing and horizontal pod autoscaling, because that is something I want to learn. I don't care that it's just for wife's home-made dog collars webshop.
This is the way
You can do it on a handful of Raspberry Pis rather than one, then.
Imagine, if you will, a Beowulf cluster of Raspberry Pis!
A man of culture, I see!
I don't get this; a Pi isn't even in the same conversation as an old rackmount server you can get for free. You couldn't stuff half the compute, ram and storage into a Pi or a dozen Pis for 10X the cost of grabbing something off eBay for a hundred bucks.
That's if the Rpi Foundation is deigning to let us peasants even buy them these days.
I have an old rackmount server I got for free. Dual Xeon X5650s, 192GB of RAM, four 8TB HDDs, and a pair of 250GB SSDs. I can only use it in the basement because it’s too loud to run anywhere else, but even then, it’s currently off because it trips its circuit breaker under heavy load.
A power strip full of Pis in a k3s cluster doesn’t do that. I used a 2GB model 4 for the control plane and 3Bs as the workers.
Why do you think you got it for free ;)
Eh, it was good when I got it. Who am I to turn down a free dual socket server though? :)
Useful in winter I suppose!
But I have a heat pump that is more efficient
If it trips circuit breakers it's using a terrifying amount of power. Honestly with energy prices I am starting to think old stuff is actually becoming a bad value in some cases.
The problem is that server will probably use more electricity, it'll be clunky to store, and it's going to be loud as fuck.
Yeah that's basically it for me. I have a collection of dev boards, old hardware and stuff other people were tossing out set up for a variety of purposes (Kubernetes clusters, two build farms, network boot, etc.). None of it is because I feel I "need" any of that for self hosting. In practice two old desktops with a bunch of drives would be perfectly capable of providing everything I need including redundancy. I have all that stuff because I'm learning and experimenting.
Switched from a raspberry pi 3 to a second hand x86 thin client (lenovo thinkcentre m920q) because raspberry pi 4 were not available at the time. Made me learn proxmox and a bunch of other cool stuff my raspi couldn't handle.
I'm rooting for ARM / RISC-V to become more popular in desktop computing / servers though.
I've always liked riscv. Just the idea of literally everything on the device being open source is a fun idea. Manuals to everything.
Just because the ISA is open source doesn't mean that the end product or even the design will be open source.
RISC-V is licensed permissively, giving anyone the right to make a proprietary (or FOSS) RISC-V processor.
Often times, you'll see mostly open source cores, but then some extention is proprietary.
Waiting for proxmox-arm becoming a thing (I know there's some community versions trying it but I'm not sure how reliable they are)
The hardware virtualisation available for arm just isn’t there yet
Apple Silicon Macs do a great job with virtualization. Outside of them there's just no nice high end hardware that's well suited for something like proxmox. It's either low end SBC, or the hyper proprietary ARM servers that I don't think we can even buy.
Modern Android phones include a hw-accelerated hypervisor. In Android 16, there will be a feature to run a full Linux VM through what Google calls protected Kernel VM (pKVM).
Qualcomm has their own implementation called Gunyah
Those are heavily customised, we’re talking raspberry pi’s here
Is there RISC-V hardware already? I thought the specification was still under development.
Very much so, not quite ready for prime time maybe, but you can play with it, StarFive is quite well-known for their chips in this space for example
Ok great! Time to get me some presents.
There are some Raspi competitors offering SBCs with RISC-V chips, there is even a RISC-V Mainboard for the framework laptops, but the last time I checked they sadly didn't reach the performance levels of comparable ARM chips.
Psh if I'm writing at that level I don't need the performance yet. Thanks!
I did similar once the pi4s were hard to get and expensive. A used x86 mini pc was cheaper and magnitudes more powerful. It runs all my server needs. I’m a simple person: homebridge, plex server, retro game library.
This struggle usually takes place over a weekend.
This guy selfhosts
i think the best choice is a cheap used pc or laptop, or server. Reduces electric waste. I also host my own server on a 19 year old Dell Insprion 1300
Think centre tiny here
Low consumption, two ddr4 slots, one 2.5" slot and one nvme slot! Lots of outside slots.
Costed less used than a new pi too. They have gotten too expensive IMO.
Pi has gotten crazy expensive.
Same mentality but HP Elitedesk Minis
Just add dell micro to the list and you have what I run - 9 tiny/mini/micro PCs run everything here. Though I may move a few things to a VPS soon.
Edit:
How would you class them, if you think you could/would/should? I'm so impressed with the thinkcentre tiny I wonder if it can get better at all.
Mostly equitable.
Ive had a slightly higher failure rate with the Dells, but the sample size is too small to be relevant.
The Lenovos more often than others ive found outfitted with a dGPU which comes in handy in some scenarios, but I think that comes down more on which enterprises more often purchase Lenovos and want the dGPU, and that its just what ive come across in the used/decommissioned territory.
Short answer - they are basically all the same.
Thanks!
ServeTheHome has a series "tiny mini micro" for exactly this reason.
lenovo thinkcentre m910q supremacy
Yesss I have a m910q as my main with (IIRC) a 6500T 4 cores.
And a m710 with the CD contraption for backup (the CD is just for fun, the PC is the backup) :-p
A lot of older equipment actually wastes more electricity.
But it will cut down on electronic waste.
Yeah, I expect there is a compromise somewhere but 10+ year old stuff which I can easily find on eBay probably just isn't worth it for anything that is going to be on most/all of the time. Better to go for more modern lower end stuff to use less power. 250w will cost you like £50 a month to run
not always. especially laptops
Not necessarily.
A i5-6500 has a TDP of 65W while a i5-13600K has a TDP of 150W.
If you get something modern that has the performance of a i5-6500 it will be a little bit more efficient. The key is that more performance uses more power.
If you buy a high watt CPU, that's on you. Ryzen 7 also came out in 2022 and had many 65 watt cpus that could outperform an i5-6500.
What should you look for if you want closer to pi energy use, like under 20w
TDP ≠ power draw. TDP is literally the Thermal Design Power aka what is the amount of thermal load a system designer should account for. Yes it can give you a rough and dirty idea of maximum power draw, but real world power draw can be entirely different because that depends on load.
For example, if your i5-6500 runs at 50-70% load while the newer processor only runs at 20-30% load due to IPC and instruction improvements the newer processor might very well use less power over the course of month than the older one despite the newer one being capable of drawing more
You're also comparing a 4c4t part to one with 14c/20t not to mention comparing a mass market part to a gaming specific part. The 6600k (which is targeting the same market segment as the 13600k) has a 91w TDP. Go compare your 6500 to the i5-13500 except again it's still comparing apples to oranges when you just look at raw specs and TDP ≠ real world power consumption
Yes, but also no. Older hardware is less power efficient, which is a cost in its own right, but also decreases backup runtime during power failure, and generates more noise and heat. It also lacks modern accelerated computing, like ai cores or hardware video encoders or decoders, if you are running those appd. Not to mention lack of nvme support, or a good NIC.
For me a good compromise is to recycle hardware upgrades every 4-5 years. A 19 year old computer? I would not bother.
Bro, I am just hosting a WordPress backup, an RSS reader, and a few Python scripts
Wordpress needs the latest GPU these days, didn't you know? /s
I have a Lenovo M710q with a i3 7100T that uses 3W at idle. I'm not mining bitcoin, server is idle 23h a day if not more.
my 19 year old laptop runs the web server just fine, and only needs 450 mb ram even with many security modules. it produces minimal noise
Yeah what I’ve always done is use the previous gaming/workstation PC as a server.
I just finished moving my basic stuff over to newer old hardware that’s only 6-7 years old, to have lots of room to grow and add to it. It’s a 9700k (8c/8t) with 32GB of ram and even a GTX 1080 for the occasional video transcode. It’s obviously overkill right now, but I plan to make it last a very long time.
exactly, you can still do HA and LB on old gear, even K8s if you must
They take up so much space though.
This is why rack mounts were made. Hell, I've seen a lot of custom builds where people have mapped out the server on their wall and it takes up no floor space. Something like this: https://i.xno.dev/kG9Wx.jpg
Oh I love that!
A rack takes up as much space as a fridge though, and mounting things to the wall is risky. You better make sure you really got it into the stud in the wall. Also, don't do that if you live in an earthquake zone.
Only full size racks. You don't need to buy a full size rack. You can get very small racks these days that are smaller than a little chest cooler. And why are you under the impression that you have to mount it on the wall?
You shared a picture of a computer mounted on a wall.
Correct. If space is such a big problem for you that it's unconscionable to use a 4U mini rack (which again, like what the fuck), then mounting hardware on the wall is a completely valid option. It'll take up zero floor space.
That is very cool
A mini PC is a good middle ground. Mostly for the video transcode and machine learning power.
Yeah, a mini PC... or if you already have one, why not 5 mini PCs?
That's what Iam aiming for at the next hardware update. I don't have the space for a server rack and a SFF desktop would also not fit into my home, so a miniPC it'll be. I cannot wait to move to x86.
I'm not sure if I'm alone in this but I have a terrible aversion to transcoding. I know the loss of quality is probably not that huge (depending on the original codec) but I just can't bring myself to get past it.
As a result I have a tiny arm based box with a 2tb SSD and I'm happy out.
You want to avoid it everywhere possible of course.
But when the GF tries to use Jellyfin on whatever random device that doesn't have the codec support to play it, it is nice to have.
Yeah that makes a lot of sense in fairness.
Trash.guides can help ya setup profiles so it won't ever transcode unless the the above reason happens
I think any mini-pc/old laptop is better, and probably cheaper than a raspberry pi nowadays
I would have disagreed with you when Pis were like $50 and chaining 3 Pis together with a hard drive was a fun project to do self hosting.
Now to get to the beefiest raspberry pi, it's $120. And in the range, yeah, for price and reliability, use a mini-pc/laptop.
I think the pi zero might have a place, don't have one but at least it is actually cheap. Used an old laptop before, now running stuff on my PC which I feel was a bit of a mistake in a way but it started as one process and more appeared over time.
At some point would like to move it into a VM tbh, then I could copy the VM to a mini PC at a later date. Or easily copy it when reinstalling the OS on my main PC. VM for convenience and separating it from my general PC usage.
Pi zero 2 runs my home automation smoothly and few of it's sisters are in a desk drawer in case of equipment failure. It has quite enough oomph to run Home Assistant with zigbee hat.
Yeah honestly i can't get anything done with a raspberry. Maybe i host too many services ?
Nah, they are underpowered for their current price. When you could get your hand on a raspberry pi 3 for 35€ 10 years ago it was a bargain. But now it's not the case.
Honestly Raspberry Pis are pretty underpowered as hosts for more than a handful of super basic services, and given they consume 20-30w at minimum you're easily getting into used office desktop territory where you can get a ton more performance right out of the gate.
The real value in the raspberry pi is in the GPIO and the cohesive ecosystem of accessories to plug into said GPIO. You can do so many cool automations and controls using just an RPi (especially if combined with something that can't be accomplished more cheaply and easily with an ESP32) but as a server host they're pretty crap in comparison to a decade old business PC off of eBay
I've got 3x RPi 4/5 with external drives and other USB running ~10 containers at 4-6W each. How did you make them draw 20W?
I've found that a pi is good enough, computationally, but not reliability wise.
A lot of things like advanced light control goes through my host, so any lockups or crashes are bad. My pi held up for about 18 months before it began to play up. I've found a small NUC system has higher reliability for the same price and power usage.
Kubernetes is designed to improve reliability
That doesn't help against hardware thermal runaway. The pi would overheat its own ram chips and hard lock up. A simple power cycle fixed it.
Even just the choice of programming language makes a big difference. Running a JVM language or NodeJS, Python, Ruby etc., you can be bottlenecked by a Pi. Meanwhile, Rust or C/C++ will use barely a fraction of those resources.
Right? I just spin up another process on my home server. No need to get more hardware involved for something that's inherently a software problem.
See, I don't pay for the electric bill to keep my collection of old enterprise equipment running because I need the performance. I keep them running because I have no resistance to the power of blinkenlights.
Yup, a pi is enough for me.
Well... 5 Pis and an ancient NUC running proxmox are enough for me. And a DS920+... and an old laptop running docker are enough for me.
With Linux any old computer from yesteryear can become a quick server. That's what I do, just make sure you got backups.
My home server is literally made from garbage left over from other PCs. The motherboard is currently some piece of junk from a prefab PC with a custom power socket, so I got to make my own adapter from scratch.
Floor PCs FTW
Yup! When I built a gaming comphter last year my old desktop became my first dive into linux. Probably overkill, but ive been having a blast with it.
Nice!
I've literally tossed working computers of the exact same model as my home server into the ewaste bin at work. One person's trash and all
I had to buy a lenovo thinkcentre mini because was cheaper than a brandnew raspberry pi.
I spend all day at work exploring the inside of the k8s sausage factory so I'm inured to the horrors and can fix basically anything that breaks. The way k8s handles ingress and service discovery makes it absolutely worth it to me. The fact that I can create an
HTTPProxyand have external-dns automagically expose it via DNS is really nice. I never have to worry about port conflicts, and I can upgrade my shit whenever with no (or minimal) downtime, which is nice for smart home stuff. Most of what I run tends to be singleton statefulsets or single-leader deployments managed with leases, and I only do horizontal for minimal HA, not at all for perf. If something gives me more trouble running in HA than it does in singleton mode then it's being run as a singleton.k8s is a complex system with priorities that diverge from what is ideal for usage at home, but it can be really nice. There are certain things that just get their own VM (Home Assistant is a big one) because they don't containerize/k8serize well though.
Yup, same here. being able to skip all the networking and DNS hassle and have it automated for you is so nice.
Having databases fully managed with cnpg is AMAZING
I just have renovate set to auto update my argocd, so everything just runs itself with zero issues. Only the occasional stateful container that has breaking changes in a minor version.
If something OOMs or crashes, it all just self heals, I never need to worry about it. I don't have any HPAs (nor cluster scaling obv), though I do have some HA stuff set up just to reduce restart times and help keep the databases happy.
The main issue with Kubernetes is that a lot of self-hosted software makes bad design decisions that actively make kubernetes harder, eg sqlite instead of postgres and secrets stored in large config files. The other big issue is that documentation only supports docker compose and not kubernetes 90% of the time so you have to know how to write yaml and read documentation.
Moving my hass from a statefulset to kubevirt sounds tempting. Did you have better reliability/ergonomics? I have been looking into moving my Hass automation to NodeRed, so that I can GitOps it all, since NodeRed supports git syncing.
The only problem I've had with Raspberry Pi is that some apps want to write a lot of stuff to "disk", and the default "disk" on a Pi is a MicroSD card which dies if you keep writing things to it. Sure, you can always plug something into a USB slot, but that adds a bit of friction to the whole process.
Oh, also, I wish it were easy to power a whole bunch of Pi units. Each one needing its own wall wart is a bit annoying, and I've had iffy results using weaker, less steady power supplies with multiple ports intended for things like phones.
I really recommend a HAT with SSD, totally worth the investment.
Yeah, but then you have to get a kind of case that can handle a Pi plus that hat. It's a good idea, it's just a bit more fiddly than just the typical booting from the SD card and doing everything that way.
Wouldn't an SSD run into problems down the line with too many Writes?
In my experience, that concern is way outdated.
Theoretically, yes, but I suspect the manufacturing quality of SD cards is a lot lower than SSDs
The amount of writes required to kill an SSD aren't going to be seen in the real world on a timescale of less than 10 years unless you're really doing something wild that you shouldn't be.
An SD card might fail after it's full capacity being written a handful of times, SSDs can survive that several hundred times over. Seriously look up the terrabytes written specs for various storage mediums and calculate out the daily amount of writes. Oftentimes with SSDs you'd have to literally write a terrabytes of data a week to actually see a problem
Not within the computer's lifetime. Consumer-grade SSDs are generally rated for 3000-5000 write cycles or more, and contain some kind of wear levelling mechanism to distribute write operations over the entire physical medium to reduce the chance of individual block failures. The first SSD I ever bought is still going strong as my server's root filesystem.
@Tangent5280 @spongeborgcubepants I think the days of worrying about SSDs failing like that during its expected lifespan are over for all practical applications.
I ended up just buying an industrial mSD card. Has yet to fail.
Most SD cards aren't really suitable for the kind of workload an operating system generates (that being mostly random i/o). Make sure to get a reputable A2 (application class 2) rated card, they aren't that expensive but perform way better.
Raspberry Pi themselves launched a card recently, I haven't tried that one but it's probably a good choice too.
I think the Raspberry Pi Linux releases mount things onto a ram drive, so the typical IO doesn't touch the SD Card. But, if you run another OS (which sometimes is the easiest way to get other software running) it tends to just treat the SD Card like an HDD/SSD.
That's definitely not true, Raspberry Pi OS works and acts like a normal Debian installation per default - with root mounted rw and all.
Other than that, there isn't much "treating like an HDD/SSD" going on, it just writes to flash when an application requests it does. If the underlying storage is an eeprom, an sdcard nvme storage doesn't really change anything here.
Or you learn proxmox and running everything as a VM
Just learn docker
Docker is so bad. I don't think a lot of you young bloods understand that. The system is so incredibly fragmented. Tools like Portainer are great, but they're a super pain in the ass to use with tools/software that include a dockerfile vs a compose file. There's no interoperability between the two which makes it insurmountably time-consuming and stupid to deal with certain projects because they're made for a specific build environment which is just antithetical to good computing.
Like right now, I have Portainer up. I want to test out Coolify. I check out templates? Damn, not there. Now I gotta add my own template manually. Ok, cool. Half way done. Oops. It expects a docker-compose.yml. The Coolify repository only has a Dockerfile. Damn, now I have to make a custom template. Oh well, not a big deal. Plop in the Dockerfile from the repository, and click "deploy." OOPS! ERROR: "failed to deploy a stack: service "soketi" has neither an image nor a build context specified: invalid compose project." Well fuck... Ok, whatever. Not the biggest of deals. Let me search for an image of "soketi" using dockerhub. Well fuck. There are 3 images which haven't been updated in several years. Awesome. Which one do I need? The echo-server? The network-watcher? PWS?
Like, do you see the issue here? There's nothing about docker that's straightforward at all. It fails in so many aspects it's insane that its so popular.
Docker got popular because of dockerfiles. It really sucks at service management though which is why people run Kubernetes in production, not Docker.
Maybe I’m an idiot (high possibility) but I couldn’t even get the instance of homebridge docker to work. However, it had zero problems running on hypervisor. You aren’t alone.
I'm coming to appreciate Hyper-V more and more to be honest. It's a very mature virtualization environment. The only issue I have with it is the inability to do GPU-passthrough. Once they figure that one out, I probably won't bother with anything else.
So close. Started on raspberry pi. Went for a cluster with dpckrt swarm. Finished with a nas and a 10years old game computer as a mediacenter. (That the electricity bill whoch made me stop the cluster)
Ha ha
Under-complicated -> over-complicated -> under-complicated.
There's a 'just right' that I think you skipped through.
rPi with k3s installed
None of the power, all of the hassle 💪💪
kbnts?
That's not a typo: https://k3s.io/
It's basically a Kubernetes cluster, which you can run locally on your PC. Really useful for playing around with Kubernetes before you move to a 'proper' environment.
That's neat, and also gives me another reason to hate numeronyms 😭
Only the good ones for me
How does that compare to microk8s? I have been using that for a while and like the plugins.
I have literally been on this exact journey. Mind you I'm on NixOS across two boxes so not quite a raspi... Perhaps my downsizing is not yet complete
Same, in fact you can also went down in RPi models. Basically the more you know, the less you need, e.g. going from Plex to Kodi to minidlna...
Absolutely the best way to learn though. The number of places I've walked into that had no clue about containers or even a vpc and thought Google drive was an API is too damn high.
I have actually had to write something that used the Google drive API for a friend's company once and it was... Unpleasant. Counterintuitive. Woefully inconsistent. My solution worked but it sucked and I am a bit ashamed of it
horror
I still like vms on digital ocean. I guess I'm a seething soydev.
I use a DO droplet with docker compose. Filthy dev here too. Much cheaper overtime than buying and hosting home server equipment.
Some stuff you want to have in your house though, like storage and stuff like home assistant.
I'd never trust DO with my vintage europorn collection
But those aren't in your house. No good for things like media servers.
I love my little k3s box and having all my config in git
As a developer and not a sysadmin, I refuse to learn anything more than docker. It's good enough for me 😤
Edit: on a more serious note, proxmox with docker containers has been more than enough for me
Do you run Docker in a VM or on the host node? I'm running a lot of LXC at home on Proxmox but sometimes it'd be nice to run Docker stuff easily as well.
It's kind of redundant but I run Docker in minimal debian LXCs. I like the speed of LXCs but I still love the reproducibility of Docker so I combine them lol. I do run regular VMs with docker for systems that are doing more than one thing
Thanks for the tip, I'll have to give that a try this weekend. I assumed it would require a full VM but never thought to try it in LXC, nice if it's that simple and I now feel kind of silly for asking haha. And the reproducibility and ease of deployment is indeed a big plus of Docker compared to LXC. It would be nice if Proxmox could add native integration for Docker at some point.
One thing that's super important is to enable container nesting in the LXC settings. You can't run docker otherwise. (Besides that one setting, it's definitely as simple as that) And very much agreed @ the docker support in PM. That'd be the dream.
Have fun with the testing though!
I've met sysadmins who say that unironically 😅
I've been enjoying Jeff Geerling's ongoing experiments with his 10" Raspberry Pi mini rack.
It doesn't work for me since all of my network equipment is 19" and there's no point in having two racks but having a 10" standard is still a great idea!
Off-topic but can you tell me why your name is red? I use Voyager and your name is red, it should mean something, right?
It denotes me as an admin. I wish it wouldn't.
Oooh it makes sense now. Saw your comment a few times and I was always wondering why the red name. Thanks!
Random mixed parts -> 4u unraid server with switch -> random fedora Optiplex that never fails unlike 4u server
Unraid is very unstable and hard to repair when it breaks
That’s very my experience as well. It’s just a worse version of Linux with nice mixed drive support.
Wait, you can host a website on a raspberry pi !? But is it really cheaper than shared hosting, for instance? And even then, quality-wise, it cannot be that good, can it?
Same as a 4x CPU with 8GB ram VPS.
Unless bandwidth is a limiting factor.
But the quality of a website is about code. Not about hardware
You see, bits sent from an x86 have 10% more antioxidants....
I had a website serve me oxidised bits. My computer BSODd and now I have herpes.
Need those antioxidants
My understanding is raspberrypi.com is hosted on raspberry pis. It's a Linux computer; it can do anything a Linux computer can do.
You can host a website on a lot less, even. But it entirely depends on what you're hosting and the load. Basically anything can host a bunch of static pages, so if your site is just that, basically anything will do. You could probably even do a WordPress site with the right caching plugins and serve a reasonable amount of traffic. The first limit you'll hit realistically is your uplink, not your webserver CPU.
You can definitely run a low traffic website with a Pi. You can run Minecraft Servers and such on Pis. Especially on Pi4s.
@MrMobius @hanke depends what you're doing. A Raspberry Pi can be surprisingly powerful. Can run NAS, websites, proxies, things like NextCloud, PiHole.
Cheaper than paying someone for hosting a small server? Maybe. But there's pros and cons of both.
Why not rasberry pi with kubernetes?
The HAT-ability of RPi makes them enough for me. You can add sata ports, PCIe, and more with a simple HAT.
any recommendations on hats for sata?
Been running one from Radxa for a while. Just make sure to check the power requirements of your drives.
Do you run Docker in a VM or on the host node? I'm running a lot of LXC at home on Proxmox but sometimes it'd be nice to run Docker stuff easily as well.
Just create an LXC container to run your dockers, all you have to do is make sure you run the LXC as privileged and enable nesting.
There are security performance and capability concerns with that approach, apparmor on the first layer lxc probably being the most annoying.
If you want to isolate your docker sandbox from your main host, you should use a vm not a container.
OP's already running LXC on the host, so... Namespaces are namespaces...
I don't see what performance issues there would be with that.
You're correct that nesting namespaces is unlikely to introduce measurable performance degradation. For performance, I was thinking mostly in the nested virtual network stack adding latency. Both docker and lxc run their own virtual interfaces.
There's also the issue of running nested apparmor, selinux, and/or seccomp checks on processes in the child containers. I know that single instances of those are often enough to kill performance on highly latency sensitive applications (SAP netweaver is the example that comes to mind) so I would imagine two instances of those checks would exacerbate those concerns.
Thanks for the tip, for some reason I assumed I couldn't run docker in LXC but never actually tried... I prefer to avoid the overhead of a full VM and I find LXCs way easier to manage from the host system. Guess I'll have something to test this weekend. Cheers!
I have been in for a couple months now, Proxmox cluster with two machines.
Works out nicely as I crash the first one too often and the DHCP going down was unacceptable, wish I got a slightly better cpu for the minipc but meh, maybe I can upgrade it later.
To be fair, also love the mini pc's and having a larger NAS. For me the PoE capabilities of the Pi's are definitely the reason I use them
a pie is neat. thats it. does it have enough ram for hosting & running all your containers? no.
three raspberries pi running k3s is good enough for me
An n100 PC is much better than that crapberry pi
jesus christ what a nice burn
https://en.m.wikipedia.org/wiki/List_of_burn_centers_in_the_United_States
Certainly tempted by one, any particular ones to look for? UK here, currys out of stock on an MSI one and Amazon is full of names I have never heard of and half the time it comes with Windows pro which is just a waste of money buying with the hardware.
I got the ASRock motherboard with the n100 cpu and built a PC with it. Right now, I just have it running two 3 TB hhds in a zfs mirror and it's working nicely.
I wanted to get a raspberry pi at first but it was stupidly expensive in my country and, for what it offered compared to a normal x86 system, it was not worth it.
N100 is two years old now. If you're going to suggest a mini PC, at least suggest one with a current gen CPU.
I was thinking more about a motherboard with that cpu, but we can go with the n150 then
I am seeing N100 mini PCs for a lot less than N150s though
I'm actually just about to start up my server again on a rp4. It's been like 5 years since I've used it. Is dietpi still the best way to go about making a Plex media server/bare bones desktop environment that I can access with 'no-machine'?
I sear no machine just broke my autoboot setup one day and I never got around to fixing it. What do you nerds think?
I'm not interested in video streaming, just hosting my music collection and audiobooks. I remember FTP being a pain to transfer music files from my phone
I bought a decade old Z840 and it's great for VMs, Plex, Arr stack, and a few other services but it is so overkill with 2 GPUs. I think what I should've done was buy a couple of used desktops or laptops to expand the my homelab as I needed.
Unraid FTW.
If you think about it, the kubernetes nodes often are only raspberry pis specwise. 2-4 cores, 8-16gb of ram
I ran lots of containers on a Pi 4 but recently purchased two cheap Chinese mini PC's with 16GB RAM and an SSD. They're so much faster and only a bit dearer than a Pi. I run Proxmox on both.
Absolutely nothing wrong with the Pi though. The Pi 4 lives on with a USB drive attached. I have NFS configured on it to backup my Proxmox VMs to it. It also hosts all the media for Jellyfin.
Ain't nobody got time for that
Raspberry Pis are way overhyped and overpriced.
Also this is totally wrong. Once you start it just keeps growing unless there is some other factor.
You’re just in the middle of the curve still …
And yes, pi’s are waaaaaay overpriced now. Get a used mini pc with a 6500T in it for server needs. It’s more than enough and maxes out at 65W. Most of the time it idles near 12W.
Yes, you can optimize a lot. Especially with Linux. I did the same and even started to replace program that did too much, bloated, with my own programs. To speed up the development I did it with AI and Cursor.