The issue isn't Docker vs Podman vs k8s vs LXC vs others. They all use OCI images to create your container/pod/etc. This new limit impacts all containerization solutions, not just Docker. EDIT: removed LXC as it does not support OCI

Instead, the issue is Docker Hub vs Quay vs GHCR vs others. It's about where the OCI images are stored and pulled from. If the project maintainer hosts the OCI images on Docker Hub, then you will be impacted by this regardless of how you use the OCI images.

Some options include:

• For projects that do not store images on Docker Hub, continue using the images as normal
• Become a paid Docker member to avoid this limit
• When a project uses multiple container registries, use one that is not Docker Hub
• For projects that have community or 3rd party maintained images on registries other than Docker Hub, use the community or 3rd party maintained images
• For projects that are open source and/or have instructions on building OCI images, build the images locally and bypass the need for a container registry
• For projects you control, store your images on other image registries instead of (or in addition to) Docker Hub
• Use an image tag that is updated less frequently
• Rotate the order of pulled images from Docker Hub so that each image has an opportunity to update
• Pull images from Docker Hub less frequently
• For images that are used by multiple users/machine under your supervision, create an image cache or image registry of images that will be used by your users/machines to mitigate the number of pulls from Docker Hub
• Encourage project maintainers to store images on image registries other than Docker Hub (or at least provide additional options beyond Docker Hub)
• Do not use OCI images and either use VM or bare metal installations
• Use alternative software solutions that store images on registries other than Docker Hub

I do not think what I would want as a replacement exists (yet). My main requirements are:

• Only FOSS software and firmware
• Similar level of "casting" compatibility/ubiquity as the discontinued Chromecast
• Easy navigation and/or great UI/UX
• Can be controlled with a stand alone remote control, phone/tablet/laptop, and remote services like Home Assistant
• As portable and low powered as the discontinued Chromecast (or no less portable than a small mini-pc)
• Ability to turn on/off the TV, switch inputs, and control the volume
• Ability to install apps/plugins to directly on the device (maybe even things like Lutris, Moonlight, or something similar for gaming)
  • Ideally, the apps would be as well maintained and provide similar levels of quality as something like an Android TV or Apple TV
• (bonus) Ability to store media locally for offline playback

I think the closest I have seen is LibreELEC + Kodi on a RaspberryPi or mini-pc. It's still not quite there for my tastes though. Hopefully the recent Chromecast announcement will lead to more/better alternatives in the coming months!

This would only work if there is no other traffic on the port being used (eg: port 22). If both the host SSH service and Forgejo SSH service expect traffic on port 22, then this would not work since server name indication (SNI) is not provided with SSH traffic and Caddy would not be able to identify the appropriate destination for multiple SSH services traffic.

The thing that makes casting so appealing for me is how ubiquitous it is. It eliminates situations with guests where they would recommend a show/movie only to find out that I can't easily play the content because it's only available on a streaming service that the guest pays for and I do not. As long as the guest brought a device and connected it to my WiFi, it more than likely could be casted without having to install another app and/or sign up for a new service (or have the guest login with their account).

I am becoming less optimistic about it though. I just do not think that the level of ubiquity that Chromecast reached even 10 years ago will be matched with a FOSS alternative. Developers would need to incorporate it into their apps, websites, etc. or it would need to be compatible with existing solutions. I doubt Google will open Chromecast up enough so other options can be fully compatible with it. Additionally, without the backing of a major corporation, I do not see developers taking the time to make their content compatible with another casting option.

Ah, you're right. I'll edit my comment.

incus may be an option for you though. It supports both LXC/LXD and OCI (although not nearly as well as Docker/Podman/Kubernetes - I don't think it supports any compose files).

The steps below are high level, but should provide an outline of how to accomplish what you're asking for without having to associate your IP address to any domains nor publicly exposing your reverse proxy and the services behind the reverse proxy. I assume since you're running Proxmox that you already have all necessary hardware and would be capable of completing each of the steps. There are more thorough guides available online for most of the steps if you get stuck on any of them.

1. Purchase a domain name from a domain name registrar
2. Configure the domain to use a DNS provider (eg: Cloudflare, Duck DNS, GoDaddy, Hetzner, DigitalOcean, etc.) that supports wild card domain challenges
3. Use NginxProxyManager, Traefik, or some other reverse proxy that supports automatic certificate renewals and wildcard certificates
4. Configure both the DNS provider and the reverse proxy to use the wildcard domain challenge
5. Setup a local DNS server (eg: PiHole, AdGuardHome, Blocky, etc.) and configure your firewall/router to use the DNS server as your DNS resolver
6. Configure your reverse proxy to serve your services via domains with a subdomain (eg: service1.domain.com, service2.domain.com, etc.) and turn on http (port 80) to https (port 443) redirects as necessary
7. Configure your DNS server to point your services' subdomains to the IP address of your reverse proxy
8. Access to your services from anywhere on your network using the domain name and https when applicable
9. (Optional) Setup a VPN (eg: OpenVPN, WireGuard, Tailscale, Netbird, etc.) within your network and connect your devices to your VPN whenever you are away from your network so you can still securely access your services remotely without directly exposing any of the services to the internet

Depending on the network's setup, having Pihole fail or unavailable could leave the network completely broken until Pihole becomes available again. Configuring the network to have at least one backup DNS server is therefore extremely important.

I also recommend having redundant and/or highly available Pihole instances running on different hardware if possible. It may also be a good idea to have an additional external DNS server (eg: 1.1.1.1, 8.8.8.8, 9.9.9.9, etc.) configured as a last resort backup in the event that all the Pihole instances are unavailable (or misconfigured).

I think I can avoid using metallb since I'm already using Kube-VIP.

I do not think that I am using Virtual IPs correctly as I tried setting it up, but it did not work. I assume the Virtual IP would just become the Load Balancer IP and I would need to configure Traefik and Kube-VIP (as well as my network) to use the Virtual IPs?

I guess I need to dig in a little deeper. I am currently only using Kube-VIP to provide a single IP address for the control plane. I think I may have it configured wrong though since that same IP address is the single load balancer IP used by Traefik.

I have struggled finding good documentation, hints, tutorials, etc. setting up Kube-VIP with Virtual IPs. Is there anything that you are aware of that might provide some assistant in setting that up correctly?

I feel silly for not realizing that the SSH config would be used by Git!

I thought if Forgejo's SSH service listened to a non-standard port that you would have to do commands with the port in the command similar to below (following your example). I guess I assumed Git did not directly use the client's SSH service.

git pull [email protected]:1234:user/project.git

people often seem to have a misinformed idea that the first item on your dns server list would be preferred and that is very much not the case

I did not know that. TIL that I am people!

Do you know if it's always this way? For example, you mentioned this is how it works for DNS on a laptop, but would it behave differently if DNS is configured at the network firewall/router? I tried searching for more info confirming this, but did not find information indicating how accurate this is.

Agreed! I am concerned though that even if a viable casting alternative started gaining momentum that Google would essentially prevent it from being widely adopted or incorporated into apps/websites the way that Chromecast is. I think it would have to be created by a large tech or media company and/or be compatible with Chromecast.

Apps are still really frustrating though. If an app exists (big if), I found the apps to either miss key features compared to the corresponding apps on other platforms or the UI/UX was terrible for a TV app.

I could get by if just one of casting or the apps were comparable to more popular alternatives. Having neither makes it very difficult to moved away from those alternatives.

I am unsure if I understood everything correctly, but I believe I am already doing everything that you mentioned. I followed the Kube-VIP's ARP daemonset's documentation. The leader election works. I am not using Kube-VIP for load balancing though. Instead, I am using Traefik, which is using the same IP address that was assigned to the control plane during both k3s's and Kube-VIP's setup. However, I am unable to get any additional VIP addresses to properly route to Traefik.

Even if I did get the additional VIP addresses working, I think I still have one last issue to overcome. I can control the local network's DNS so that service#1 is assigned VIP#1 and service#2 assigned VIP#2. However, how would this be handled for traffic received externally? If the external/public DNS has service#1 and service#2 assigned to the network's public IP address, both service's traffic would be received by the router/firewall on port 22. The router/firewall could forward traffic on port 22 to (presumably) a single IP address, which would only allow service#1 or service#2 (but not both) to receive traffic publicly, correct?

Are you able to provide some details on how you are doing this? I don't think you can do much with reverse proxies and SSH beyond routing all traffic on port 22 (or the configured SSH port) to whichever port SSH is listening on. In other words, the reverse proxy cannot route SSH traffic for the host on port 22 to the host, route SSH traffic for Forgejo on port 22 to Forgejo's SSH process, and SFTP traffic on port 22 to the SFTP process - at least not via domain name like a HTTP/HTTPS reverse proxy would work.

Instead, this would need to be done via IP address where the host SSH process listens on 192.168.1.2, the Forgejo SSH process listens on 192.168.1.3, and the SFTP process listens on 192.168.4. Otherwise, each of those services would need to use different ports.

I believe the reverse proxy settings in your post is just configured to handle the http/https connection, not the SSH connection. Instead, SSH connections are likely being routed to the machine running Foegejo via DNS and your reverse proxy is not involved with anything related to SSH.

I assume you either have SSH disabled on your host or SSH on your host uses a port other than 22?

Thanks - I appreciate the response. However, for the services I'm asking about (eg: SSH, SFTP, etc.) do not support SSL/TLS and forcing SSL/TLS with a wrapper would likely cause incompatibility with clients.

I agree though - for other the services connecting via HTTP/HTTPS, there would likely either be some incompatibility or at least warning messages about not using SSL/TLS.

There are plenty of valid reasons to want to use a reverse proxy for SSH:

• Maybe there is a Forgejo instance and Gitea instance running on the same server.
• Maybe there is a Prod Forgejo instance and Dev Forgejo instance running on the same server.
• Maybe both Forgejo and an SFTP are running on the same server.
• Maybe Forgejo is running in a cluster like Docker Swarm or Kubernetes
• Maybe there is a desire to have Caddy act as a bastion host due to an inability to run a true bastion host for SSH or reduce maintenance of managing yet another service/server in addition to Caddy

Regardless of the reason, your last point is valid and the real issue here. I do not think it is possible for Caddy to reverse proxy SSH traffic - at least not without additional software (either on the client, server, or both) or some overly complicated (and likely less secure) setup. This may be possible if TCP traffic included SNI information, but unfortunately it does not.

Open source projects still need a maintainer/owner. For example, Facebook "controls" react, Microsoft "controls" Visual Studio Code, and AdguardTeam "controls" AdGuardHome. There are several reasons to not trust a maintainer (eg, license changes, prioritizing or implementing undesired functionality and anti-features, converting to "open core", abandoning the project, selling out to less trust worthy entities, etc.).

Per Adguard's website, the legal entity behind the various AdGuard products is ADGUARD SOFTWARE LIMITED. A quick search on that company shows that there are 3 founders and they seem to have some ties to Russia. There is more information online about this, but whether this means they can be trusted or not is up to each potential user of one of the AdGuard products.

It likely is a config issue, but likely not a config issue relating to SSH not using SSL/TLS (and therefor not using SNI). There is a lot of information out there about this exact issue, but very little about how to properly handle this issue.

• https://crypto.stackexchange.com/questions/60255/why-doesnt-ssh-use-tls
• https://community.traefik.io/t/routing-ssh-traffic-with-traefik-v2/717
• https://docs.gitea.com/installation/install-with-docker#ssh-container-passthrough
• https://security.stackexchange.com/questions/1599/what-is-the-difference-between-ssl-vs-ssh-which-is-more-secure
• https://stackoverflow.com/questions/67251200/does-ssh-use-sni-headers-and-how-sni-inject-to-ssh-tunnel-connection
• https://www.haproxy.com/blog/route-ssh-connections-with-haproxy
• https://serverfault.com/questions/34552/is-there-a-name-based-virtual-host-ssh-reverse-proxy/610971#610971

That all makes sense and tried setting it up that way but could not get it to work. I am not sure if it was an issue with my network, k3s, Kube-VIP, or Traefik (or some combination of them). I will try getting it to work again.

Even if I do though, I would run into an issue if I publicly exposed these services (I understand there are security implications of doing so). How would I route traffic received externally/publicly on port 22 to more than one IP address? I think I would only be able to do this for local/internal traffic by managing the local DNS.