Garuda Linux server configurations
General information
- Our current infrastructure is hosted in one of these.
- The only other server not being contained in this dedicated server is our mail server.
- Both servers are being backed up to Hetzner storage boxes via Borg.
- After multiple different setups, we settled on NixOS as our main OS as it provides reproducible and atomically updated system states
- Most (sub)domains are protected by Cloudflare while also making use of its caching feature. Exemptions are services such as our mail server and parts violating Cloudflares rules such as proxying Piped content.
Quick links
Devshell and how to enter it
This NixOS flake provides a devshell which contains all deployment tools as well as handy aliases for common tasks. The only requirement for using it is having the Nix package manager available. It can be installed on various distributions via the package manager or the following script (click me for more information):
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix -o nix-install.sh # Check its content afterwards
sh ./nix-install.sh install --diagnostic-endpoint=""
This installs the Nix packages with flakes already pre-enabled. After that, the shell can be invoked as follows:
nix develop # The intended way to use the devshell
nix-shell # Legacy, non-flakes way if flakes are not available for some reason
This also sets up pre-commit-hooks and shows the currently implemented tasks, which can be executed by running the command.
[infra-nix]
ansible-core - Radically simple IT automation
apply - Applies the infra-nix configuration previously deployed to the servers
buildiso-local - Spawns a local buildiso shell to build to ./buildiso (needs Docker)
buildiso-remote - Spawns a buildiso shell on the iso-runner builder
clean - Runs the garbage collection on the servers
deploy - Deploys the local NixOS configuration to the servers
update - Performs a full system update on the servers by bumping flake lock
update-forum - Updates the Discourse container of our forum
update-toolbox - Updates the locked Chaotic toolbox commit and deploys the changes
update-website - Updates the locked website commit and deploys the changes
General structure
A general overview of the folder structure can be found below:
├── assets
├── docker-compose
│ ├── all-in-one
│ ├── github-runner
│ └── proxied
├── docs
│ ├── hosts
│ └── theme
├── home-manager
├── host_vars
│ ├── garuda-build
│ ├── garuda-mail
│ └── immortalis
├── nixos
│ ├── hosts
│ │ ├── garuda-build
│ │ ├── garuda-mail
│ │ └── immortalis
│ ├── modules
│ │ └── static
│ └── services
│ ├── chaotic
│ ├── docker-compose-runner
│ └── monitoring
├── playbooks
├── scripts
└── secrets
Secrets in this repository
Secrets are managed via a custom Git submodule that contains ansible-vault encrypted files as well as a custom NixOS module garuda-lib which makes them available to our services.
The submodule is available in the secrets directory once it has been set up for the first time. It can be initialized by running:
git submodule init
git submodule update
To view or edit any of these files, one can use the following commands:
ansible-vault decrypt secrets/pathtofile
ansible-vault edit secrets/pathtofile
ansible-vault encrypt secrets/pathtofile
Further information on ansible-vault can be found in its documentation.
It is important to keep the secrets directory in the latest state before deploying a new configuration as misconfigurations might happen otherwise.
Passwords in general
Our mission-critical passwords that maintainers and team members need to have access to are stored in our Bitwarden instance. After creating an account, maintainers need to be invited to the Garuda Linux organisation in order to access the stored credentials.
Linting and formatting
We utilize pre-commit-hooks to automatically set up the pre-commit-hook with all the tools once nix-shell or nix develop is run for the first time.
Checks can then be executed by running one of the following configs:
nix flake check # checks flake outputs and runs pre-commit at the end
pre-commit run --all-files # only runs the pre-commit tools on all files
Its configuration can be found in the flake.nix file. (click me). At the time of writing, the following tools are being run:
It is recommended to run pre-commit run --all-files before trying to commit changes. Then use cz commit to generate a commitizen complying commit message.
CI/CD
We have used pull-/push-based mirroring for this git repository, which allows easy access to Renovate without having to run a custom instance of it. The following tasks have been implemented as of now:
nix flake checkruns for every labeled PR and commit on main.- Renovate periodically checks
docker-compose.ymland other supported files for version updates. It has a dependency dashboard as well as the developer interface to check logs of individual runs. Minor updates appear as grouped PRs while major updates are separated from those. Note that this only applies to the GitHub side. - Deployment of our mdBook-based documentation to Cloudflare pages.
- Deployment of our Website to Cloudflare pages.
Workflows will generally only be executed if a relevant file has been changed, eg. nix flake check won't run if only the README was changed.
Monitoring
Our current monitoring stack mostly relies on Netdata to provide insight into current system loads and trends.
The major reason for using it was that it provides the most vital metrics and alerts out of the box without having to create in-depth configurations.
Might switch to the Prometheus/Grafana/Loki stack in the future. We used to set up children -> parent streaming in the past, though after transitioning to one big host this didn't make sense anymore.
Instead, up to 10GB of data gets stored on individual hosts.
While Netdata agents do have their dashboard, the Dashboard provided by Netdata is far superior and allows a better insight, eg. by offering the functions feature.
Additional services like Squid or Nginx have been configured to be monitored by Netdata plugins as well. Further information can be found in its documentation.
To access the previously linked dashboard, use [email protected] as login, the login will be completed after opening the link sent here.
Common maintenance tasks
Rebuilding / updating the forum container
Sometimes Discourse needs its container to build rebuild via cli rather than the webinterface. This can be done with:
ssh -p 224 [email protected]
cd /var/discourse
sudo ./launcher rebuild app
Building ISO files
To build Garuda ISO, one needs to connect to the iso-runner container and execute the buildiso command, which opens a shell containing the needed environment:
ssh -p 227 [email protected] # if one ran nix develop before, this can be skipped
buildiso
buildiso -i # updates the iso-profiles repo
buildiso -p dr460nized
Further information on available commands can be found in the garuda-tools repository. After the build process is finished, builds can be found on iso.builds.garudalinux.org. No automatic pushing to Sourceforge and Cloudflare R2 happens by default, see below for more information on how to achieve this.
Deploying a new ISO release
We are assuming all ISOs have been tested for functionality before executing any of those commands.
ssh -p 227 [email protected]
buildall # builds all ISO provided in the buildall command
deployiso -FS # sync to Cloudflare R2 and Sourceforge
deployiso -FSR # sync to Cloudflare R2 and Sourceforge while also updating the latest (stable, non-nightly) release
deployiso -Sd # to delete the old ISOs on Sourceforge once they aren't needed anymore
deployiso -FSRd # oneliner for the above-given commands
Updating the system
One needs to have the infra-nix repo cloned locally. Then proceed by updating the flake.lock file, pushing it to the server & building the configurations:
nix flake update
ansible-playbook garuda.yml -l $servername # Eg. immortalis for the Hetzner host
deploy # Skip using the above command and use this one in case nix develop was used
Then you can either apply it via Ansible or connect to the host to view more details about the process while it runs:
ansible-playbook apply.yml -l $servername # Ansible
apply # Nix develop shell
ssh -p 666 [email protected] # Manually, exemplary on immortalis
sudo nixos-rebuild switch
Keep in mind that this will restart every service whose files changed since the last system update. On our Hetzner server, this includes a restart of every declarative nixos-container if needed, causing a small downtime.
Changing system configurations
Most system configurations are contained in individual Nix files in the nix directory of this repo. This means changing anything must not be done manually but by editing the corresponding file and pushing/applying the configuration afterward.
ansible-playbook garuda.yml -l $servername # Eg. immortalis for the Hetzner host
deploy # In case nix develop is used
As with the system update, one can either apply via Ansible or manually:
ansible-playbook apply.yml -l $servername # Ansible
apply # Nix develop shell
ssh -p 666 [email protected] # Manually, exemplary on immortalis
sudo nixos-rebuild switch
Adding a user
Adding users needs to be done in users.nix:
- Add a new user here
- Add the SSH public key to flake inputs
- Add the specialArgs
keys.useras seen here - Deploy & apply the configuration
Changing Docker configurations
If configurations of services running in Docker containers need to be altered, one needs to edit the corresponding docker-compose.yml (./nix/docker-compose/$name) file or .env file in the secrets directory (see the secrets section for details on that topic).
The deployment is done the same way as with normal system configuration.
Updating Docker containers
Docker containers sometimes use the latest tag in case no current tag is available or in the case of services like Piped and Searx, where it is often crucial to have the latest build to bypass Google's restrictions.
Containers using the latest tag are automatically updated via watchtower daily.
The remaining ones can be updated by changing their version in the corresponding docker-compose.yml and then running deploy & apply.
If containers are to be updated manually, this can be achieved by connecting to the host, running nixos-container root-login $containername, and executing:
cd /var/garuda/docker-compose-runner/$name/ # replace $name with the actual docker-compose.yml or autocomplete via tab
sudo docker compose pull
sudo docker compose up -d
The updated containers will be pulled and automatically recreated using the new images.
Rotating IPv6
Sometimes it is needed to rotate the available IPv6 addresses to solve the current ones being rate-limited for outgoing requests of Piped, Searx, etc.
This can be achieved by editing the hosts Nix file immortalis.nix, replacing the existing values of the networking.interfaces."eth0".ipv6.addresses keys seen here.
Then, proceed doing the same with the squid configuration.
IPv6 addresses need to be generated from our available /64 subnet space and can't be chosen completely random.
To ease the process, a command called ipv6-generator is available in this git repos' devshell.
Checking whether backups were successful
To check whether backups to Hetzner are still working as expected, connect to the server and execute the following:
systemctl status borgbackup-job-backupToHetzner
This should yield a successful unit state. The only exception is having an exit code != 0 due to files having changed during the run.
Updating Chaotic-AUR toolbox
This needs to be done by updating the flake input (git repo URL of the website) src-chaotic-toolbox:
cd nix
nix flake lock --update-input src-chaotic-toolbox # toolbox
After that deploy as usual by running deploy and apply. The commit and corresponding hash will be updated and NixOS will use it to build the toolbox using the new revision automatically.
Updating the Garuda startpage content
Our startpage consists of a simple homer deployment. Its configuration is stored in the startpage repo, which gets cloned to the docker-compose.yml's directory to serve the files. In order, updating is currently done manually after pushing the changes to the repo (might automate this soon via systemd timer!):
ssh -p 225 [email protected]
cd /var/garuda/docker-compose-runner/all-in-one/startpage
git pull
sudo docker restart homer
Important links
This is a collection of important links when working with the infrastructure:
Most important
Nix-related
- Devshell documentation
- Flake-parts documentation
- Home Manager options search
- NixOS mailserver documentation
- The Nix documentation
- The Nix package and option search
Tools documentation
Web interfaces
- Chaotic-AUR Syncthing
- Cloudflare Dashboard
- Freshping
- Freshstatus
- Hetzner Robot
- Matrix Admin
- Netdata
- Renovate Dashboard
- Tailscale
Services to be administrated
- Bitwarden
- Discourse
- Chaotic-AUR
- Element
- Firefox syncserver
- Invidious
- Lemmy
- Lingva
- Mastodon
- Matrix
- Matrix Discord bridge (internal only)
- Matrix IRC bridge (internal only)
- Matrix Telegram bridge (internal only)
- Nextcloud
- Piped
- PrivateBin
- Searx
- TheLounge
- Whoogle
- WikiJs
Additional pages
Users
Multiple kinds of users can make use of our infrastructure. A current list of users is available here.
Adding new users
New users can be added by supplying a fitting configuration in the users.nix module.
In case of a password being required, its hash needs to be generated as follows:
nix-shell -p mkpasswd --run 'mkpasswd -sm bcrypt' > /path/to/hashedPasswordFile
The file then needs to be ansible-vault encrypted and added to our secrets repository.
This one is only available to members of our GitLab org and usually is cloned as git submodule to ./secrets.
Onboarding a new admin
After confirming the trustworthiness of a new admin, the following actions need to be executed:
- Add them to the admin users
- Add their ssh public key to the flake inputs and specialArgs
- Make them an owner of the GitLab organization
- Add them to our Bitwarden organization to allow access to passwords and email accounts
- Add them to the Cloudflare Account
- Make them an admin of Discourse
- Make them an admin of Matrix
Users
These are the people who are currently allowed to use our servers.
Admins
Admins have root access to all servers and may therefore change everything. They are responsible for the well-being of the infrastructure and its development.
*/
users.nico = {
extraGroups = [ "wheel" "docker" "chaotic_op" ];
home = "/home/nico";
isNormalUser = true;
openssh.authorizedKeys.keyFiles = [ keys.nico ];
hashedPasswordFile = "/var/garuda/secrets/pass/nico";
uid = lib.mkIf garuda-lib.unifiedUID 1001;
};
users.sgs = {
extraGroups = [ "wheel" ];
home = "/home/sgs";
isNormalUser = true;
openssh.authorizedKeys.keys = [
"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxBY8TX0iEQkf3Bym+3XVlrk8OLOwHOrj7Uy+WxjncOkkutyZ1WsY9liF4j9yjptyQG7Lx8OM8q44NE6+Rk1OXJXMF7CZ4Jq/WvMVnh2zKyNnF8wHBcspsAdG90wCxo6OmNpnY/rRRlNwwnore7raF2PrERtSlsEvLsUgvspYQ8cnLwerJP43QeETlpE1oR0FrbXWQet0I63Ky6UDEp07x0yee21VHnAG74rjGeFGwJBmCPSxnfGVNhCaR0zyu9+hh222liBrlilYm8nqLlsYGZCXiVdOxXJbBy89EVpHds7Lutf+TAYwsPGZf7U4k+g2Jx8N0JHXyzVZa0zS+I48+tqBBflEOqU9oEfGuz4cU/qWys5soLcRX2p9td+RF3OEdBKlTW4UYsINJUri6QSEUrsGaXqQZy8Ds2FBdUpb4pmFVlo9+4qRouiI80a5xVa7a1E5eS5xK5BzWH4fNg5SqtT5L9i2i1ocZp7FA0oa+ixnXNiC1umPZaY/9s+5fh1s= [email protected]"
];
hashedPasswordFile = "/var/garuda/secrets/pass/sgs";
uid = lib.mkIf garuda-lib.unifiedUID 1002;
};
users.tne = {
extraGroups = [ "wheel" "docker" "chaotic_op" ];
home = "/home/tne";
isNormalUser = true;
openssh.authorizedKeys.keyFiles = [ keys.tne ];
hashedPasswordFile = "/var/garuda/secrets/pass/tne";
uid = lib.mkIf garuda-lib.unifiedUID 1003;
};
/*
Maintainers
Maintainers have restricted access, which allows them to use buildiso to build new ISO files via the iso-runner container.
*/
users.frank = {
home = "/home/frank";
isNormalUser = true;
openssh.authorizedKeys.keyFiles = lib.mkIf config.services.garuda-iso.enable [ keys.frank ];
shell = lib.mkIf (!config.services.garuda-iso.enable) "${pkgs.util-linux}/bin/nologin";
uid = lib.mkIf garuda-lib.unifiedUID 1007;
};
/*
Chaotic-AUR maintainers
Chaotic-AUR maintainers have access to the builder containers of our infrastructure. They may operate the repository by doing all kinds of packaging-related tasks such as adding or removing those.
*/
users.technetium = {
extraGroups = lib.mkIf garuda-lib.chaoticUsers [ "chaotic_op" ];
home = "/home/technetium";
isNormalUser = true;
openssh.authorizedKeys.keyFiles = lib.mkIf garuda-lib.chaoticUsers [ keys.technetium1 ];
shell = lib.mkIf (!garuda-lib.chaoticUsers) "${pkgs.util-linux}/bin/nologin";
uid = lib.mkIf garuda-lib.unifiedUID 1004;
};
users.alexjp = {
extraGroups = lib.mkIf garuda-lib.chaoticUsers [ "chaotic_op" ];
home = "/home/alexjp";
isNormalUser = true;
openssh.authorizedKeys.keyFiles = lib.mkIf garuda-lib.chaoticUsers [ keys.alexjp ];
shell = lib.mkIf (!garuda-lib.chaoticUsers) "${pkgs.util-linux}/bin/nologin";
uid = lib.mkIf garuda-lib.unifiedUID 1005;
};
users.xiota = {
extraGroups = lib.mkIf garuda-lib.chaoticUsers [ "chaotic_op" ];
home = "/home/xiota";
isNormalUser = true;
openssh.authorizedKeys.keyFiles = lib.mkIf garuda-lib.chaoticUsers [ keys.xiota ];
shell = lib.mkIf (!garuda-lib.chaoticUsers) "${pkgs.util-linux}/bin/nologin";
uid = lib.mkIf garuda-lib.unifiedUID 1006;
};
/*
immortalis (Hetzner dedicated)
General
This system utilizes a NixOS host which uses nixos-containers to build declarative systemd-nspawn machines for different purposes. To make the best use of the available resources, common directories are shared between containers. This includes /home (home-manager / NixOS configurations writing to home are generated by the host and disabled for the containers), Pacman and Chaotic cache, the /nix directory, and a few others. Further details can be found in the Nix expression of the host.
All directories containing important data were mapped to /data_1 and /data_2 to have them all in one place. The first mostly contains web services' files, the latter only builds related directories such as the Pacman cache.
The current line-up looks as follows:
nico@immortalis ~> machinectl
MACHINE CLASS SERVICE OS VERSION ADDRESSES
chaotic-kde container systemd-nspawn nixos 24.05 10.0.5.90
chaotic-v4 container systemd-nspawn nixos 24.05 10.0.5.140
docker container systemd-nspawn nixos 24.05 10.0.5.100
docker-proxied container systemd-nspawn nixos 24.05 10.0.5.110
forum container systemd-nspawn nixos 24.05 10.0.5.70
github-runner container systemd-nspawn nixos 24.05 10.0.5.130
iso-runner container systemd-nspawn nixos 24.05 10.0.5.40
lemmy container systemd-nspawn nixos 24.05 10.0.5.120
mastodon container systemd-nspawn nixos 24.05 10.0.5.80
postgres container systemd-nspawn nixos 24.05 10.0.5.50
temeraire container systemd-nspawn nixos 24.05 10.0.5.20
web-front container systemd-nspawn nixos 24.05 10.0.5.10
We are seeing:
- 1 ISO builder (
iso-runner) - 1 reverse proxy serving all the websites and services (
web-front) - 2 Docker dedicated nspawn containers (
docker&docker-proxied) - 4 Chaotic-AUR builders (
chaotic-kde,chaotic-v4,github-runner&temeraire) - 5 app dedicated containers (
forum,lemmy,mastodon&postgres)
Connecting to the server
After connecting to the host via ssh -p 666 [email protected], containers can generally be entered by running nixos-container login $containername, eg. nixos-container login web-front. Some containers may also be connected via SSH using the following ports:
- 22:
temeraire(needs to be 22 to allow pushing packages to the main Chaotic-AUR node via rsync) - 224:
forum - 225:
docker - 226:
chaotic-kde - 227:
iso-runner - 228:
web-front - 229:
postgres(access the database in127.0.0.1viassh -p 229 [email protected] -L 5432:127.0.0.1:5432) - 400:
chaotic-v4
Docker containers
Some services not packaged in NixOS or are easier to deploy this way are serviced via the Docker engine. This contains services like Piped, Whoogle, and Matrix. We use a custom NixOS module to deploy those with the rest of the system. Secrets are handled via our secret management which consists of a git submodule secret (private repo with ansible-vault encrypted files) and garuda-lib (see secrets section). Those contain a docker-compose directory in which the .env files for the docker-compose.yml are stored.
Chaotic-AUR / repository
Our repository leverages Chaotic-AUR's toolbox to provide the main node for the [chaotic-aur] repository as well as two more instances building the [garuda] and [chaotic-kde] repositories. Users of the chaotic_op group may build packages on the corresponding nixos-container via the chaotic command:
chaotic get $package # pull PKGBUILD
chaotic mkd $package # build package in the previously cloned directory
chaotic bump $package # increment pkgver of $package by 0.1 to allow a rebuild
chaotic rm $package # remove the package from the repository
Further information may be obtained by clicking chaotic seen above. The corresponding builders are:
[chaotic-aur]:temeraire[garuda]:repo[chaotic-kde]:chaotic-kde
Squid proxy
Squid is being installed on the host machine to proxy outgoing requests via random IPv6 addresses of the /64 subnet Hetzner provides for services that need it, eg. Piped, the Chaotic-AUR builders, and other services that are getting rate limited quickly. The process is not entirely automated, which means that we currently have a pool of IPv6 addresses active and need to switch them whenever those are getting rate-limited again.
Since we supplied an invalid IPv4 to force outgoing IPv6, the log files were somewhat cluttered by (expected) errors. Systemd-unit logging has been set to LogLevelMax=1 to un-clutter the journal and needs to be increased again if debugging needs to be done.
Backups
Backups are provided by daily Borg runs. Only the /data_1 directory is backed up (minus /data_1/{dockercache,dockerdata}) as the rest are either Nix-generated or build-related files that can easily recovered from another repository mirror. The corresponding systemd-unit is named borgbackup-job-backupToHetzner.
Tailscale / mesh network
While Tailscale was commonly used to connect multiple VMs before, this server only has it active on the host. However, we are leveraging Tailscale's subnet router feature to serve the 10.0.5.0/24 subnet via Tailscale, which means that other Tailscale clients may access the nixos-containers via their IP if tailscale up --accept-routes was used to set up the service.
Nix expression
{ garuda-lib
, pkgs
, config
, ...
}: {
imports = [
../modules
./immortalis/containers.nix
./immortalis/hardware-configuration.nix
];
# Increase /tmp & /run size to make better use of RAM
boot = {
kernelPackages = pkgs.linuxPackages_6_6;
loader.systemd-boot.enable = true;
runSize = "50%";
tmp = {
tmpfsSize = "95%";
useTmpfs = true;
};
};
# Network configuration with a bridge interface
networking = {
defaultGateway = "116.202.208.65";
defaultGateway6 = {
address = "fe80::1";
interface = "eth0";
};
hostName = "immortalis";
interfaces = {
"eth0" = {
ipv4.addresses = [
{
address = "116.202.208.112";
prefixLength = 26;
}
];
ipv6.addresses = [
# Random outgoing
{
address = "2a01:4f8:2200:30ac:cf2d:7d73:eddf:8871";
prefixLength = 64;
}
{
address = "2a01:4f8:2200:30ac:5b38:dbde:e5a7:91b2";
prefixLength = 64;
}
{
address = "2a01:4f8:2200:30ac:fa33:0d97:0755:6833";
prefixLength = 64;
}
{
address = "2a01:4f8:2200:30ac:8f15:81f6:355c:d9d6";
prefixLength = 64;
}
{
address = "2a01:4f8:2200:30ac:4436:e5e7:2236:0d77";
prefixLength = 64;
}
{
address = "2a01:4f8:2200:30ac:1ea4:1794:1963:b8da";
prefixLength = 64;
}
{
address = "2a01:4f8:2200:30ac:5628:7e9f:d8ec:544d";
prefixLength = 64;
}
{
address = "2a01:4f8:2200:30ac:d830:ce99:e2b7:3e43";
prefixLength = 64;
}
{
address = "2a01:4f8:2200:30ac:edc9:2d08:2b32:e532";
prefixLength = 64;
}
{
address = "2a01:4f8:2200:30ac:a833:0fd7:29d4:5309";
prefixLength = 64;
}
];
};
};
# Specify these here to allow containers to access
# our services from the internal network via NAT reflection
nat.forwardPorts = [
{
# web-front (HTTP)
destination = "10.0.5.10:80";
loopbackIPs = [ "116.202.208.112" ];
proto = "tcp";
sourcePort = 80;
}
{
# web-front (HTTPS)
destination = "10.0.5.10:443";
loopbackIPs = [ "116.202.208.112" ];
proto = "tcp";
sourcePort = 443;
}
{
# web-front (HTTPS)
destination = "10.0.5.10:443";
loopbackIPs = [ "116.202.208.112" ];
proto = "udp";
sourcePort = 443;
}
{
# web-front (Matrix)
destination = "10.0.5.10:8448";
loopbackIPs = [ "116.202.208.112" ];
proto = "tcp";
sourcePort = 8448;
}
{
# iso-runner (SSH)
destination = "10.0.5.40:22";
loopbackIPs = [ "116.202.208.112" ];
proto = "tcp";
sourcePort = 227;
}
{
# chaotic-v4 (SSH)
destination = "10.0.5.140:22";
loopbackIPs = [ "116.202.208.112" ];
proto = "tcp";
sourcePort = 400;
}
];
firewall.trustedInterfaces = [ "br0" ];
};
# OpenSSH on another port to keep Chaotic's main node working
services.openssh.ports = [ 666 ];
# Make use of all threads!
security.allowSimultaneousMultithreading = true;
# Raise limits to support many containers
# (from LXC's recommendedSysctlSettings)
boot.kernel.sysctl = {
"fs.inotify.max_user_instances" = 1048576;
"fs.inotify.max_user_watches" = 1048576;
"kernel.dmesg_restrict" = 1;
"kernel.keys.maxkeys" = 2000;
"kernel.pid_max" = 4194303;
"net.ipv4.neigh.default.gc_thresh3" = 8192;
"net.ipv6.neigh.default.gc_thresh3" = 8192;
};
# Improve nspawn container performance since we grant all capabilities anyway
# https://github.com/systemd/systemd/issues/18370#issuecomment-768645418
environment.variables.SYSTEMD_SECCOMP = "0";
# Custom tailscale configuration to advertise our bridge's subnet route
systemd.services.tailscale-autoconnect.script = with pkgs; ''
sleep 2
status="$(${tailscale}/bin/tailscale status -json | ${jq}/bin/jq -r .BackendState)"
if [ $status = "Running" ]; then
exit 0
fi
${tailscale}/bin/tailscale up --authkey ${garuda-lib.secrets.tailscale.authkey} \
--advertise-routes=10.0.5.0/24
'';
# We want to have same UID's in all containers to allow sharing home directories
garuda-lib.unifiedUID = true;
# Monitor a few services of the containers
services = {
netdata.configDir = {
"go.d/postgres.conf" = pkgs.writeText "postgres.conf" ''
jobs:
- name: postgres
dsn: 'postgres://netdata:[email protected]:5432/'
'';
"go.d/squidlog.conf" = pkgs.writeText "squidlog.conf" ''
jobs:
- name: squid
path: /var/log/squid/access.log
log_type: csv
csv_config:
format: '- resp_time client_address result_code resp_size req_method - - hierarchy mime_type'
'';
"go.d/web_log.conf" = pkgs.writeText "web_log.conf" ''
jobs:
- name: nginx
path: /var/log/nginx/access.log
'';
};
smartd = {
enable = true;
extraOptions = [ "-A /var/log/smartd/" "--interval=600" ];
};
};
# Fix permissions of nginx log files to allow Netdata to read it (gets reset frequently)
system.activationScripts.netdata = ''chown 60:netdata -R /var/log/nginx'';
# Backup configurations to Hetzner storage box
programs.ssh.macs = [ "hmac-sha2-512" ];
services.borgbackup.jobs = {
backupToHetzner = {
compression = "auto,zstd";
doInit = true;
encryption = {
mode = "repokey-blake2";
passCommand = "cat /var/garuda/secrets/backup/repo_key";
};
environment = {
BORG_RSH = "ssh -i /var/garuda/secrets/backup/ssh_immortalis -p 23";
};
exclude = [ "/data_1/dockercache" "/data_1/dockerdata" ];
paths = [ "/data_1" ];
prune.keep = {
within = "1d";
daily = 3;
weekly = 2;
monthly = 2;
};
repo = "[email protected]:./immortalis";
startAt = "daily";
};
};
# A proxy server making use of our IPv6 IP addresses
# traffic sent through the proxy is only allowing IPv6 connections
services.squid = {
enable = true;
extraConfig = ''
forwarded_for delete
dns_nameservers 2606:4700:4700::1111
acl tenth random 1/10
acl ninth random 1/9
acl eighth random 1/8
acl seventh random 1/7
acl sixth random 1/6
acl fifth random 1/5
acl fourth random 1/4
acl third random 1/3
acl half random 1/2
# Invalid IP
tcp_outgoing_address 10.254.254.254
tcp_outgoing_address 2a01:4f8:2200:30ac:cf2d:7d73:eddf:8871 tenth
tcp_outgoing_address 2a01:4f8:2200:30ac:5b38:dbde:e5a7:91b2 ninth
tcp_outgoing_address 2a01:4f8:2200:30ac:fa33:0d97:0755:6833 eighth
tcp_outgoing_address 2a01:4f8:2200:30ac:8f15:81f6:355c:d9d6 seventh
tcp_outgoing_address 2a01:4f8:2200:30ac:4436:e5e7:2236:0d77 sixth
tcp_outgoing_address 2a01:4f8:2200:30ac:1ea4:1794:1963:b8da fifth
tcp_outgoing_address 2a01:4f8:2200:30ac:5628:7e9f:d8ec:544d fourth
tcp_outgoing_address 2a01:4f8:2200:30ac:d830:ce99:e2b7:3e43 third
tcp_outgoing_address 2a01:4f8:2200:30ac:edc9:2d08:2b32:e532 half
tcp_outgoing_address 2a01:4f8:2200:30ac:a833:0fd7:29d4:5309
# Invalid IP
udp_outgoing_address 10.254.254.254
udp_outgoing_address 2a01:4f8:2200:30ac:cf2d:7d73:eddf:8871 tenth
udp_outgoing_address 2a01:4f8:2200:30ac:5b38:dbde:e5a7:91b2 ninth
udp_outgoing_address 2a01:4f8:2200:30ac:fa33:0d97:0755:6833 eighth
udp_outgoing_address 2a01:4f8:2200:30ac:8f15:81f6:355c:d9d6 seventh
udp_outgoing_address 2a01:4f8:2200:30ac:4436:e5e7:2236:0d77 sixth
udp_outgoing_address 2a01:4f8:2200:30ac:1ea4:1794:1963:b8da fifth
udp_outgoing_address 2a01:4f8:2200:30ac:5628:7e9f:d8ec:544d fourth
udp_outgoing_address 2a01:4f8:2200:30ac:d830:ce99:e2b7:3e43 third
udp_outgoing_address 2a01:4f8:2200:30ac:edc9:2d08:2b32:e532 half
udp_outgoing_address 2a01:4f8:2200:30ac:a833:0fd7:29d4:5309
# This does not rotate the logs, but asks squid to reopen the log file so that logrotate can rotate it
logfile_rotate 0
'';
proxyAddress = "10.0.5.1";
};
systemd.services.squid = {
serviceConfig = {
Restart = "always";
RestartSec = 10;
# Shut off all logging but level 1 errors as we get spamming a lot due to
# not being able to use our invalid address 10.254.254.254
LogLevelMax = 1;
};
startLimitIntervalSec = 80;
startLimitBurst = 6;
};
services.logrotate.settings.squid = {
files = "/var/log/squid/*.log";
frequency = "daily";
su = "squid squid";
rotate = 5;
compress = true;
delaycompress = true;
postrotate = "${config.systemd.package}/bin/systemctl kill --signal=SIGUSR1 squid";
};
# Can't really instantly remove this, need to find an alternative first
nixpkgs.config.permittedInsecurePackages = [ "squid-6.8" ];
# Adapt Nix to our core-count
nix.settings.max-jobs = 8;
system.stateVersion = "23.05";
}
chaotic-kde
General
This is a package builder, that is supposed to build a KDE stack from master branch. It is still unused while packages are waiting to be fixed.
Nix expression
{ pkgs
, sources
, ...
}: {
imports = sources.defaultModules ++ [ ../modules ];
garuda-lib.chaoticUsers = true;
# Enable Chaotic-AUR building
services.chaotic.enable = true;
services.chaotic.cluster-name = "kde-git";
services.chaotic.host = "kde-git.chaotic.cx";
services.chaotic.extraConfig = ''
export CAUR_DEPLOY_LABEL="KDE Dragon 🐉"
export CAUR_LOWER_PKGS+=(chaotic-mirrorlist chaotic-keyring git qt6-declarative qt6-tools qt6-doc clang doxygen qt6-declarative)
export CAUR_PACKAGER="Garuda Builder <[email protected]>"
export CAUR_ROUTINES=/tmp/chaotic/routines
export CAUR_SIGN_KEY=D6C9442437365605
export CAUR_SIGN_USER=root
export CAUR_TELEGRAM_TAG="@dr460nf1r3"
export GIT_SSH_COMMAND="ssh -i /var/garuda/secrets/chaotic/interfere_ed25519"
export HTTP_PROXY=http://10.0.5.1:3128/
export HTTPS_PROXY=http://10.0.5.1:3128/
export NO_PROXY=mirror.rackspace.com,cloudflaremirrors.com,github.com,downloads.sentry-cdn.com
'';
services.chaotic.db-name = "chaotic-aur-kde";
services.chaotic.routines = [ "hourly" ];
services.chaotic.patches = [ ../services/chaotic/add-chaotic-repo.diff ../services/chaotic/prepend-repo.diff ];
services.chaotic.useACMEHost = "garudalinux.org";
# Allow systemd-nspawn to create subcgroups (for Chaotic-AUR builders)
systemd.services.remount-sysfscgroup = {
description = "Remount cgroup2 to allow systemd-nspawn to create subcgroups";
wantedBy = [ "multi-user.target" ];
serviceConfig.Type = "oneshot";
script = ''
${pkgs.mount}/bin/mount -t cgroup2 -o rw,nosuid,nodev,noexec,relatime none /sys/fs/cgroup
'';
};
system.stateVersion = "23.05";
}
docker-proxied
General
Here, all of the Docker containers that need to have proxied outgoing requests are being deployed.
Nix expression
{ garuda-lib
, sources
, ...
}: {
imports = sources.defaultModules ++ [ ../modules ];
# This container runs proxied docker containers
services.docker-compose-runner.proxied = {
envfile = garuda-lib.secrets.docker-compose.proxied;
source = ../../docker-compose/proxied;
};
# Let Docker use squid as outgoig proxy
# Fails to pull images if *.docker.io is not excluded from proxy
systemd.services.docker = {
environment = {
HTTPS_PROXY = "http://10.0.5.1:3128";
HTTP_PROXY = "http://10.0.5.1:3128";
NO_PROXY = "localhost,127.0.0.1,*.docker.io,ghcr.io";
};
};
# This one is set manually as service because it needs restart: always
# (which the docker-compose-runner overwrites)
virtualisation.oci-containers = {
backend = "docker";
containers.whoogle = {
environment = {
WHOOGLE_AUTOCOMPLETE = "1";
WHOOGLE_CONFIG_LANGUAGE = "lang_en";
WHOOGLE_CONFIG_NEW_TAB = "1";
WHOOGLE_CONFIG_SEARCH_LANGUAGE = "lang_en";
WHOOGLE_CONFIG_STYLE = ":root {--whoogle-logo: #4c4f69;--whoogle-page-bg: #eff1f5;--whoogle-element-bg: #bcc0cc;--whoogle-text: #4c4f69;--whoogle-contrast-text: #5c5f77;--whoogle-secondary-text: #6c6f85;
--whoogle-result-bg: #ccd0da;--whoogle-result-title: #7287fd;--whoogle-result-url: #dc8a78;--whoogle-result-visited: #e64553;--whoogle-dark-logo: #cdd6f4;
--whoogle-dark-page-bg: #1e1e2e;--whoogle-dark-element-bg: #45475a;--whoogle-dark-text: #cdd6f4;--whoogle-dark-contrast-text: #bac2de;--whoogle-dark-secondary-text: #a6adc8;
--whoogle-dark-result-bg: #313244;--whoogle-dark-result-title: #b4befe;--whoogle-dark-result-url: #f5e0dc;--whoogle-dark-result-visited: #eba0ac;}
#whoogle-w {fill: #89b4fa;} #whoogle-h {fill: #f38ba8;}#whoogle-o-1 {fill: #f9e2af;}#whoogle-o-2 {fill: #89b4fa;}#whoogle-g {fill: #a6e3a1;}#whoogle-l {fill: #f38ba8;}
#whoogle-e {fill: #f9e2af;}";
WHOOGLE_CONFIG_THEME = "dark";
WHOOGLE_CONFIG_URL = "https://search.garudalinux.org";
WHOOGLE_CONFIG_VIEW_IMAGE = "1";
WHOOGLE_RESULTS_PER_PAGE = "15";
};
extraOptions = [
"--cap-drop=all"
"--pids-limit=50"
"--security-opt=no-new-privileges:true"
"--tmpfs=/run/tor/:size=1M,uid=927,gid=927,mode=1700"
"--tmpfs=/var/lib/tor/:size=10M,uid=927,gid=927,mode=1700"
];
hostname = "whoogle";
image = "benbusby/whoogle-search:latest";
ports = [ "5000:5000" ];
user = "whoogle";
volumes = [ "/var/garuda/docker-compose-runner/proxied/whoogle:/config" ];
};
};
system.stateVersion = "23.05";
}
Docker compose
---
services:
# Searxng search engine
searx:
image: searxng/searxng:latest # It tends do be important to stay current
container_name: searx
volumes: [./searxng:/etc/searxng]
ports: [8080:8080]
environment:
BASE_URL: https://searx.garudalinux.org/
BIND_ADDRESS: 0.0.0.0:8080
HTTPS_PROXY: http://10.0.5.1:3128
HTTP_PROXY: http://10.0.5.1:3128
INSTANCE_NAME: Garuda's SearxNG
NO_PROXY: "*.garudalinux.org"
cap_drop: [ALL]
cap_add: [CHOWN, SETGID, SETUID, DAC_OVERRIDE]
restart: always
# Librey search engine
librey:
image: ghcr.io/ahwxorg/librey:latest # It tends do be important to stay current
container_name: librey
ports:
- 8081:8080
environment:
- CONFIG_CACHE_TIME=20
- CONFIG_DISABLE_BITTORRENT_SEARCH=false
- CONFIG_GOOGLE_DOMAIN=com
- CONFIG_HIDDEN_SERVICE_SEARCH=true
- CONFIG_INSTANCE_FALLBACK=true
- CONFIG_INVIDIOUS_INSTANCE=https://invidious.snopyta.org
- CONFIG_LANGUAGE=en
- CONFIG_NUMBER_OF_RESULTS=10
- CONFIG_RATE_LIMIT_COOLDOWN=25
- CONFIG_TEXT_SEARCH_ENGINE=google
restart: unless-stopped
# Lingva
lingva:
image: thedaviddelta/lingva-translate:latest # Only latest tag is available
container_name: lingva
environment:
DARK_THEME: "true"
DEFAULT_SOURCE_LANG: auto
DEFAULT_TARGET_LANG: en
HTTP_PROXY: http://10.0.5.1:3128
HTTPS_PROXY: http://10.0.5.1:3128
SITE_DOMAIN: lingva.garudalinux.org
ports: [3002:3000]
restart: always
redlib:
image: quay.io/redlib/redlib:latest
container_name: redlib
environment:
REDLIB_BANNER_: Garuda's Redlib
REDLIB_DEFAULT_AUTOPLAY_VIDEOS: true
REDLIB_DEFAULT_BLUR_NSFW: true
REDLIB_DEFAULT_COMMENT_SORT: confidence
REDLIB_DEFAULT_DISABLE_VISIT_REDDIT_CONFIRMATION: false
REDLIB_DEFAULT_FIXED_NAVBAR: true
REDLIB_DEFAULT_FRONT_PAGE: popular
REDLIB_DEFAULT_HIDE_AWARDS: true
REDLIB_DEFAULT_HIDE_HLS_NOTIFICATION=: true
REDLIB_DEFAULT_HIDE_SCORE: false
REDLIB_DEFAULT_LAYOUT: card
REDLIB_DEFAULT_POST_SORT: hot
REDLIB_DEFAULT_SHOW_NSFW: false
REDLIB_DEFAULT_THEME: dracula
REDLIB_DEFAULT_USE_HLS: true
REDLIB_DEFAULT_WIDE: false
REDLIB_PUSHSHIFT_FRONTEND: undelete.pullpush.io
REDLIB_ROBOTS_DISABLE_INDEXING: true
REDLIB_SFW_ONLY: false
ports:
- 8082:8080
user: nobody
read_only: true
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
healthcheck:
test: ["CMD", "wget", "--spider", "-q", "--tries=1", "http://localhost:8082/settings"]
interval: 5m
timeout: 3s
restart: always
# Automated container updates
watchtower:
image: containrrr/watchtower:1.7.1
container_name: watchtower
command:
--cleanup searx lingva whoogle librey
volumes: [/var/run/docker.sock:/var/run/docker.sock]
restart: always
volumes:
piped_proxy:
docker
General
This container consists of our docker-compose-runner module, which deploys all Docker-based services that don't need to proxied outgoing requests. For the other ones, have a look here.
Nix expression
{ garuda-lib
, sources
, ...
}: {
imports = sources.defaultModules ++ [ ../modules ];
# This container is just for docker-compose stuff
services.docker-compose-runner.all-in-one = {
envfile = garuda-lib.secrets.docker-compose.all-in-one;
source = ../../docker-compose/all-in-one;
};
# MongoDB port is being forwarded to this container
networking.firewall = { allowedTCPPorts = [ 27017 ]; };
system.stateVersion = "23.05";
}
Docker compose
---
services:
# Nextcloud AIO (self-managed containers)
nextcloud-aio-mastercontainer:
image: nextcloud/all-in-one:latest
restart: always
container_name: nextcloud-aio-mastercontainer # Don't change this!
volumes:
- nextcloud_aio_mastercontainer:/mnt/docker-aio-config # Don't change this!
- /var/run/docker.sock:/var/run/docker.sock:ro
ports:
- 8080:8080
environment:
- APACHE_PORT=11000
- APACHE_IP_BINDING=10.0.5.100
# Firefox syncserver
syncserver:
container_name: syncserver
image: crazymax/firefox-syncserver:edge # newest, versioned one 3 years old
volumes: [./syncserver:/data]
ports: [5001:5000]
environment:
FF_SYNCSERVER_ACCESSLOG: true
FF_SYNCSERVER_FORCE_WSGI_ENVIRON: true
FF_SYNCSERVER_FORWARDED_ALLOW_IPS: "*"
FF_SYNCSERVER_PUBLIC_URL: https://ffsync.garudalinux.org
FF_SYNCSERVER_SECRET: ${FF_SYNCSERVER_SECRET:-?err}
FF_SYNCSERVER_SQLURI: sqlite:////data/syncserver.db
TZ: Europe/Berlin
restart: always
# Web IRC access
thelounge:
image: thelounge/thelounge:4.4.3
container_name: thelounge
volumes: [./thelounge:/var/opt/thelounge]
ports: [9000:9000]
restart: always
# Password vault
bitwarden:
image: vaultwarden/server:1.30.5
container_name: bitwarden
volumes: [./bitwarden:/data]
ports: [8081:80]
environment:
ADMIN_TOKEN: ${BW_ADMIN_TOKEN:-?err}
DOMAIN: https://bitwarden.garudalinux.org
SIGNUPS_ALLOWED: true
SMTP_FROM: [email protected]
SMTP_HOST: mail.garudalinux.org
SMTP_PASSWORD: ${BW_SMTP_PASSWORD:-?err}
SMTP_PORT: 587
SMTP_SSL: false
SMTP_USERNAME: [email protected]
WEBSOCKET_ENABLED: true
YUBICO_CLIENT_ID: ${BW_YUBICO_CLIENT_ID:-?err}
YUBICO_SECRET_KEY: ${BW_YUBICO_ADMIN_SECRET:-?err}
restart: always
# Secure pastebin
privatebin:
image: privatebin/nginx-fpm-alpine:1.7.3
container_name: privatebin
volumes:
- ./privatebin:/srv/data
- ./configs/privatebin.cfg.php:/srv/cfg/conf.php
ports: [8082:8080]
restart: always
# Garuda startpage
homer:
image: b4bz/homer:v24.04.1
container_name: homer
volumes: [./startpage:/www/assets]
ports: [8083:8080]
restart: always
# MongoDB instance (Chaotic-AUR / repo metrics)
mongodb:
image: mongo:7.0.9
container_name: mongodb
volumes: [./mongo:/data/db]
ports: [27017:27017]
environment:
MONGO_INITDB_ROOT_USERNAME: ${MONGO_ROOT_USERNAME:-?err}
MONGO_INITDB_ROOT_PASSWORD: ${MONGO_ROOT_PASSWORD:-?err}
restart: always
# WikiJs
wikijs:
image: requarks/wiki:2.5
container_name: wikijs
volumes: [./wikijs/assets:/wiki/assets/favicons]
ports: [3001:3000]
environment:
DB_TYPE: postgres
DB_HOST: 10.0.5.50
DB_PORT: 5432
DB_USER: wikijs
DB_PASS: ${WIKIJS_DB_PASS:-?err}
DB_NAME: wikijs
restart: always
# Matrix homeserver
matrix:
image: matrixdotorg/synapse:v1.107.0
container_name: matrix
volumes: [./matrix/matrix:/data]
ports: [8008:8008]
restart: always
mautrix-telegram:
image: dock.mau.dev/mautrix/telegram
container_name: mautrix-telegram
volumes: [./matrix/mautrix-telegram:/data]
restart: always
healthcheck:
test:
- CMD-SHELL
- "! (grep -q 'System clock is wrong, set time offset to' /tmp/debug.log &&\
\ rm /tmp/debug.log && kill -SIGINT 1)"
interval: 1m
timeout: 10s
matrix-appservice-discord:
image: ghcr.io/matrix-org/matrix-appservice-discord:develop
container_name: matrix-appservice-discord
volumes: [./matrix/matrix-appservice-discord:/data]
restart: always
matrix_web:
image: vectorim/element-web:v1.11.67
container_name: element_web
depends_on: [matrix]
volumes: [./matrix/element/config.json:/app/config.json]
ports: [8084:80]
restart: always
# Admin interface for Matrix
matrix_admin:
image: awesometechnologies/synapse-admin:latest # Versioned lags behind 7 months
container_name: matrix_admin
depends_on: [matrix]
ports: [8085:80]
restart: always
# Matrix to IRC/Discord/Telegram relay
matterbridge:
image: 42wim/matterbridge:1.26
container_name: matterbridge
depends_on: [matrix]
volumes:
- ./matterbridge/matterbridge.toml:/etc/matterbridge/matterbridge.toml:ro
restart: always
# Makes world content available for our Lemmy instance
lemmy_seeder:
image: nowsci/lcs:20231201035206
container_name: lemmy_lcs
environment:
COMMUNITY_COUNT: 50
COMMUNITY_SORT_METHODS: '[ "TopAll", "TopDay" ]'
COMMUNITY_TYPE: All
LOCAL_URL: https://lemmy.garudalinux.org
LOCAL_USERNAME: ${LOCAL_USERNAME:-?err}
LOCAL_PASSWORD: ${LOCAL_PASSWORD:-?err}
MINUTES_BETWEEN_RUNS: 240
NSFW: false
POST_COUNT: 50
REMOTE_INSTANCES:
'[ "beehaw.org", "lemmy.world", "lemmy.ml", "sh.itjust.works",
"lemmy.one" ]'
SECONDS_AFTER_COMMUNITY_ADD: 17
restart: unless-stopped
# Automated container updates
watchtower:
image: containrrr/watchtower:1.7.1
container_name: watchtower
command:
--cleanup matrix_web matrix_admin wikijs mongodb homer privatebin bitwarden
thelounge syncserver nextcloud_app lemmy_seeder
volumes: [/var/run/docker.sock:/var/run/docker.sock]
restart: always
volumes:
nextcloud_aio_mastercontainer:
name: nextcloud_aio_mastercontainer # Don't change this!
forum
General
In here, we only have Docker set up and use the traditional way of installing Discourse to /var/discourse. Since own scripts are provided to handle the container, not much is to be seen here.
Links
Nix expression
{ sources, ... }: {
imports = sources.defaultModules ++ [ ../modules ];
# Enable Docker since we use the official Docker image in /var/discourse
virtualisation.docker.enable = true;
# Open required port
networking.firewall.allowedTCPPorts = [ 80 ];
system.stateVersion = "23.05";
}
github-runner
General
With this container, we provide a GitHub runner as well as (more recently), a GitLab runner. This container does not have the regular Garuda configurations because it is considered untrusted.
Access needs to happen by running nixos-container root-login on immortalis (click me).
Nix expression
{ keys
, ...
}: {
# No default modules, untrusted container!
# imports = sources.defaultModules ++ [
# ./garuda/garuda.nix
# ];
imports = [
../modules/hardening.nix
../modules/motd.nix
../services/docker-compose-runner/docker-compose-runner.nix
];
# Common Docker configurations
virtualisation.docker = {
autoPrune.enable = true;
autoPrune.flags = [ "-a" ];
};
# This container is just for docker-compose stuff
services.docker-compose-runner.github-runner = {
envfile = "/var/garuda/secrets/github-runner.env";
source = ../../docker-compose/github-runner;
};
services.docker-compose-runner.gitlab-runner = {
source = ../../docker-compose/gitlab-runner;
};
# Enable SSH
services.openssh.enable = true;
# No custom users - only Pedro and root via nixos-container root-login
users = {
allowNoPasswordLogin = true;
mutableUsers = false;
users.pedrohlc = {
home = "/home/pedrohlc";
isNormalUser = true;
openssh.authorizedKeys.keyFiles = [ keys.pedrohlc ];
};
};
# Make Pedro god here
nix.settings.trusted-users = [ "pedrohlc" ];
security.sudo.extraRules = [
{
users = [ "pedrohlc" ];
commands = [
{
command = "ALL";
options = [ "NOPASSWD" ];
}
];
}
];
# OOM prevention
systemd.oomd = {
enable = true; # This is actually the default, anyways...
enableSystemSlice = true;
enableUserSlices = true;
};
networking.firewall = {
extraCommands = ''
iptables -t nat -A PREROUTING -p tcp -d 172.17.0.1 --dport 3128 -j DNAT --to-destination 10.0.5.1:3128
iptables -t nat -A POSTROUTING -p tcp -d 172.17.0.1 --dport 3128 -j SNAT --to-source 10.0.5.130
'';
extraStopCommands = ''
iptables -t nat -D PREROUTING -p tcp -d 10.130.0.1 --dport 3128 -j DNAT --to-destination 10.0.5.1:3128
iptables -t nat -D POSTROUTING -p tcp -d 10.0.5.1 --dport 3128 -j SNAT --to-source 10.0.5.130
'';
};
system.stateVersion = "23.05";
}
lemmy
General
This container provides our Lemmy instance
Nix expression
{ garuda-lib
, sources
, ...
}: {
imports = sources.defaultModules ++ [ ../modules ];
# Our Lemmy instance
services.lemmy = {
database.uri = "postgresql://lemmy:${garuda-lib.secrets.lemmy.database}@10.0.5.50/lemmy";
enable = true;
settings = {
hostname = "lemmy.garudalinux.org";
email = {
smtp_server = "mail.garudalinux.net:587";
smtp_login = "[email protected]";
inherit (garuda-lib.secrets.lemmy) smtp_password;
smtp_from_address = "[email protected]";
tls_type = "starttls";
};
};
};
services.nginx = {
enable = true;
httpConfig = ''
map "$request_method:$http_accept" $proxpass {
# If no explicit matches exists below, send traffic to lemmy-ui
default "http://lemmy-ui";
# GET/HEAD requests that accepts ActivityPub or Linked Data JSON should go to lemmy
# "~^(?:GET|HEAD):.*?application\/(?:activity|ld)\+json" "http://lemmy";
# All non-GET/HEAD requests should go to lemmy
"~^(?!(GET|HEAD)).*:" "http://lemmy";
}
upstream lemmy {
server "127.0.0.1:8536";
}
upstream lemmy-ui {
server "127.0.0.1:1234";
}
server {
listen 80;
server_name lemmy.garudalinux.org;
server_tokens off;
gzip on;
gzip_types text/css application/javascript image/svg+xml;
gzip_vary on;
client_max_body_size 25M;
add_header X-Frame-Options SAMEORIGIN;
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
real_ip_header X-Real-IP;
set_real_ip_from 10.0.5.10;
# frontend general requests
location / {
proxy_pass $proxpass;
rewrite ^(.+)/+$ $1 permanent;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
}
# backend
location ~ ^/(api|pictrs|feeds|nodeinfo|.well-known) {
proxy_pass "http://lemmy";
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
}
}
'';
};
system.stateVersion = "23.05";
}
mastodon
General
This container provides our Mastodon instance.
Nix expression
{ lib
, pkgs
, sources
, garuda-lib
, ...
}: {
imports = sources.defaultModules ++ [ ../modules ];
# Our Mastodon
services.mastodon = {
configureNginx = true;
database = {
createLocally = false;
host = "10.0.5.50";
name = "mastodon";
passwordFile = "/var/lib/mastodon/secrets/db-password";
user = "mastodon";
};
enable = true;
extraConfig = {
"LOCAL_DOMAIN" = "garudalinux.org";
"SMTP_DOMAIN" = "social.garudalinux.org";
"WEB_DOMAIN" = "social.garudalinux.org";
};
extraEnvFiles = [ "/var/lib/mastodon/secrets/env" ];
localDomain = "social.garudalinux.org";
mediaAutoRemove.enable = false;
smtp = {
authenticate = true;
fromAddress = "[email protected]";
host = "mail.garudalinux.net";
passwordFile = "/var/lib/mastodon/secrets/smtp-password";
port = 587;
user = "[email protected]";
};
streamingProcesses = 4;
};
# Run daily cleanup of statuses and media of Mastodon
systemd.services.mastodon-media-cleanup = {
description = "Run daily cleanup of statuses and media of Mastodon";
serviceConfig = {
ExecStart = pkgs.writeShellScript "execstart" ''
set -e
/run/current-system/sw/bin/mastodon-tootctl media remove --days=30
/run/current-system/sw/bin/mastodon-tootctl statuses remove --days=30
'';
Path = [ pkgs.mastodon ];
Restart = "on-failure";
RestartSec = "30";
};
wantedBy = [ "multi-user.target" ];
};
systemd.timers.mastodon-media-cleanup = {
description = "Monthly cleanup of statuses and media of Mastodon";
timerConfig.OnCalendar = [ "monthly" ];
wantedBy = [ "timers.target" ];
};
# Scan for orphaned media mo
systemd.services.mastodon-orphan-cleanup = {
description = "Run weekly cleanup of orphaned media of Mastodon";
serviceConfig = {
ExecStart = pkgs.writeShellScript "execstart" ''
set -e
/run/current-system/sw/bin/mastodon-tootctl media remove --days=7
/run/current-system/sw/bin/mastodon-tootctl statuses remove --days=7
'';
Path = [ pkgs.mastodon ];
Restart = "on-failure";
RestartSec = "30";
};
wantedBy = [ "multi-user.target" ];
};
systemd.timers.mastodon-orphan-cleanup = {
description = "Run weekly cleanup of orphaned media of Mastodon";
timerConfig.OnCalendar = [ "weekly" ];
wantedBy = [ "timers.target" ];
};
services.nginx = {
virtualHosts."social.garudalinux.org" = {
enableACME = lib.mkForce false;
useACMEHost = "garudalinux.org";
extraConfig = ''
${garuda-lib.nginxReverseProxySettings}
real_ip_header X-Real-IP;
set_real_ip_from 10.0.5.10;
'';
locations = {
"@proxy" = {
proxyWebsockets = lib.mkForce false;
};
"/api/v1/streaming/" = {
proxyWebsockets = lib.mkForce false;
};
};
};
};
system.stateVersion = "23.05";
}
postgres
General
This container houses our Postgres database. Multiple servces access it:
- Lemmy
- Mastodon
- Matrix
- Matrix bridges
- WikiJs
Nix expression
{ garuda-lib
, pkgs
, sources
, config
, lib
, ...
}:
let
server_config = pkgs.writeText "server-config" ''
{
"Servers": {
"1": {
"Name": "Main",
"Group": "Garuda",
"Username": "pgadmin",
"Host": "/var/run/postgresql",
"Port": 5432,
"SSLMode": "prefer",
"MaintenanceDB": "postgres",
"PassFile": "/dev/null",
"Shared": true,
"SharedUsername": "pgadmin"
}
}
}
'';
in
{
imports = sources.defaultModules ++ [ ../modules ];
# Our Postgres database
services.postgresql = {
enable = true;
ensureDatabases = [
"lemmy"
"matrix-discord"
"matrix-irc"
"matrix-telegram"
"mastodon"
"synapse"
"wikijs"
];
ensureUsers = [
{
name = "lemmy";
ensureDBOwnership = true;
}
{
name = "mastodon";
ensureDBOwnership = true;
}
{
name = "matrix-bridges";
}
{
name = "synapse";
ensureDBOwnership = true;
}
{
name = "wikijs";
ensureDBOwnership = true;
}
{
name = "pgadmin";
ensureClauses.superuser = true;
}
];
initialScript = pkgs.writeText "backend-initScript" ''
CREATE USER netdata;
GRANT pg_monitor TO netdata;
# After 23.11, ensurePermissions got deprecated
GRANT ALL PRIVILEGES ON DATABASE matrix-bridges TO matrix-discord;
GRANT ALL PRIVILEGES ON DATABASE matrix-bridges TO matrix-irc;
GRANT ALL PRIVILEGES ON DATABASE matrix-bridges TO matrix-telegram;
'';
authentication = "host all all 10.0.5.0/24 md5";
# We don't need to worry about different interfaces, because the only interface
# available is eth0, which is fully isolated
enableTCPIP = true;
};
# Regular backups for our database (every 6h)
services.postgresqlBackup = {
compression = "zstd";
enable = true;
location = "/var/garuda/backups/postgres";
};
# Run daily synapse state compressor on Matrix database
systemd.services.synapse_auto_compressor = {
description = "Run synapse state compressor on Matrix db";
serviceConfig = {
ExecStart = pkgs.writeShellScript "execstart" ''
set -e
${pkgs.matrix-synapse-tools.rust-synapse-compress-state}/bin/synapse_auto_compressor \
-p postgresql://${garuda-lib.secrets.matrix.db_string}@10.0.5.50/synapse -c 500 -n 100
'';
Restart = "on-failure";
RestartSec = "30";
};
wantedBy = [ "multi-user.target" ];
};
systemd.timers.synapse_auto_compressor = {
description = "Run synapse state compressor on Matrix db";
timerConfig.OnCalendar = [ "daily" ];
wantedBy = [ "timers.target" ];
};
services.pgadmin = {
enable = true;
initialEmail = "[email protected]";
initialPasswordFile = garuda-lib.secrets.pgadmin_password;
openFirewall = true;
settings = {
FIXED_BINARY_PATHS = {
"pg" = "${config.services.postgresql.package}/bin";
};
SUPPORT_SSH_TUNNEL = false;
AUTHENTICATION_SOURCES = [ "webserver" ];
WEBSERVER_REMOTE_USER = "X-Forwarded-User";
MASTER_PASSWORD_REQUIRED = false;
};
};
systemd.services.pgadmin = {
preStart = lib.mkAfter ''
EMAIL=${lib.escapeShellArg config.services.pgadmin.initialEmail}
FILE=${lib.escapeShellArg server_config}
${config.services.pgadmin.package}/bin/pgadmin4-cli load-servers "$FILE" --user "$EMAIL"
'';
};
# Open up ports for Postgres
networking.firewall.allowedTCPPorts = [ 5432 ];
system.stateVersion = "23.05";
}
repo
temeraire
General
This is our package builder, which also serves as main node for Chaotic-AUR.
Nix expression
{ config
, garuda-lib
, pkgs
, sources
, ...
}: {
imports = sources.defaultModules ++ [ ../modules ];
# This disables HTTPS certificates and forced redirects
garuda-lib.behind_proxy = true;
# Enable Chaotic-AUR building
services.chaotic.enable = true;
services.chaotic.cluster-name = "garuda-cluster";
# Let nginx set itself up for this local domain
services.chaotic.host = "local.chaotic.invalid";
services.chaotic.extraConfig = ''
export CAUR_DEPLOY_LABEL="Temeraire 🐉"
export CAUR_PACKAGER="Garuda Builder <[email protected]>"
export CAUR_ROUTINES=/tmp/chaotic/routines
export CAUR_SIGN_KEY=D6C9442437365605
export CAUR_SIGN_USER=root
export CAUR_TELEGRAM_TAG="@dr460nf1r3"
export CAUR_TYPE=primary
export CAUR_URL=https://builds.garudalinux.org/repos/chaotic-aur/x86_64
export REPOCTL_CONFIG=/usr/local/etc/chaotic-repoctl.toml
export GIT_SSH_COMMAND="ssh -i /var/garuda/secrets/chaotic/interfere_ed25519"
export HTTPS_PROXY=http://10.0.5.1:3128/
export HTTP_PROXY=http://10.0.5.1:3128/
export NO_PROXY=mirror.rackspace.com,cloudflaremirrors.com,github.com,downloads.sentry-cdn.com
'';
services.chaotic.db-name = "chaotic-aur";
services.chaotic.routines = [ "afternoon" "hourly.1" "hourly.2" "morning" "nightly" "tkg-wine" ];
# Special Syncthing configuration allowing to push to main node
services.syncthing = {
enable = true;
openDefaultPorts = true;
configDir = config.services.syncthing.dataDir;
inherit (garuda-lib.secrets.syncthing.esxi-build) cert key;
overrideFolders = false;
overrideDevices = false;
user = "root";
group = "chaotic_op";
settings = {
gui = {
apikey = "garudalinux";
insecureSkipHostcheck = true;
inherit (garuda-lib.secrets.syncthing.esxi-build.credentials) user password;
};
};
guiAddress = "10.0.5.20:8384";
};
# Allow systemd-nspawn to create subcgroups (for Chaotic-AUR builders)
systemd.services.remount-sysfscgroup = {
description = "Remount cgroup2 to allow systemd-nspawn to create subcgroups";
wantedBy = [ "multi-user.target" ];
serviceConfig.Type = "oneshot";
script = ''
${pkgs.mount}/bin/mount -t cgroup2 -o rw,nosuid,nodev,noexec,relatime none /sys/fs/cgroup
'';
};
# Auto reset syncthing stuff
systemd.services.syncthing-reset = {
serviceConfig.Type = "oneshot";
script = ''
"${pkgs.curl}/bin/curl" -X POST -H "X-API-Key: garudalinux" http://localhost:8384/rest/db/override?folder=${garuda-lib.secrets.syncthing.folders.chaotic-aur}
'';
};
systemd.timers.syncthing-reset = {
wantedBy = [ "timers.target" ];
timerConfig.OnCalendar = [ "hourly" ];
};
garuda-lib.chaoticUsers = true;
# Chaotic-AUR builders need to upload their packages
users.users.ufscar_hpc = {
extraGroups = [ "chaotic_op" ];
isNormalUser = true;
openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFslN7a613H3hztK/yzHE4ZBOJ4448+EN867Y/IDpAfc [email protected]"
];
};
users.users.catbuilder = {
extraGroups = [ "chaotic_op" ];
isNormalUser = true;
openssh.authorizedKeys.keys = [
"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHELhrMFNvxgAYMdzwerszypuvQc3uCFjkR6xCbcQnrcCrueJqTQ4y8WzddwxhRzKbSTQVhPdB5l95IYk7eOtmBmaMp4LAV2osMWDI/x3NyoY5s7YgpW815qNX9Io7VnrFUr0LK7hJ+Uw87nyxGp3zGddPVMUK7PIdJf2GxTxKPryycdLa9QWijfm3YBdN10yBMp6KrfPEnhtmNPMrc3wuBG4+xBoJxNOy0DJdIf2PRwU2CddP0zdDWwlMbGeHGcaJmlAx0u9e1jL8KWB/oyGT1D9q4l+fU8E9nZG+kAFMO1yG25je9bJnYNPMV1gdRT47G3J/B982XYO4G4AiOER0v0M0MN0qWTvIVBG6Vnly81ME91Qao34Lw2QOhZMVFwWz01u8KLLQy/Z2rX7jKyqeUyGXgs5NPmkeJ1vzpSRLXY+5GX5yva8A041Nft7sfKYPFjMsDaxAKVPz7LkKX1dYdiC4c3a/RcCzLKY+Uabjr0QAK4MKwmMW+SNF0QHr9mk= root@Chaotic"
];
};
users.users.chaotic-dragon = {
extraGroups = [ "chaotic_op" ];
isNormalUser = true;
openssh.authorizedKeys.keys = [
"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0zLuPM4IE4xsxen2XBqWKSQz5CpHONgguOhVuR5rTxRqijiwGro0VR4gPhpmuZjLkms4CJ2YGyjTbjDkh48+wAoiPjdvVqF6kJ9TLkHZabMJfx5chKMCVFcHM+0/F768fF/nRsfusbRO7H2nLGMXJ1eObemiCGg0e8Ccs0XA4PF9bGaDm+4bblNasVyT6PsnaYziyBtwU3fzBVbdQmErw37sjXV9jNsEq3XF9wSaFf/Dfzh9xY1CR1KC7Af84lL1vOj7QL06tEmDO6W4JJCpRS4OonpuahwaaR4gn6wW09eDgrpXUI5DhxGizwGPLdwENRONpcXP0xnWetC9IaUADHb9yZwQKZhN9RCoO5ytqrt/NkGfn7Si+mWSfMQRGvfgJocC89peIhbchXalT+JS1XWD+Isvj2I+sqmAcoKgji09MTF0lMW+m83/+YA7Jdhn5CLVs9RxZ5cwz1TqveuUaq4i9P867iKCltrqZxxgXD4emZXhHGvGrw8cNQZOVAhc= root@chaotic-dragon"
];
};
users.users.dragons-ryzen = {
extraGroups = [ "chaotic_op" ];
isNormalUser = true;
openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAd8nLLjysVefmk3I6BI/IkooUvnGSy7966T54gWNvgW nico@slim-lair"
];
};
# Ufscar-HPC needs diffie-hellman-group-exchange-sha1
services.openssh.settings = {
KexAlgorithms = [
"curve25519-sha256"
"[email protected]"
"diffie-hellman-group-exchange-sha1"
"diffie-hellman-group16-sha512"
"diffie-hellman-group18-sha512"
"[email protected]"
];
};
# Our main webserver on this machine
services.nginx = {
enable = true;
virtualHosts = {
"builds.garudalinux.org" = {
extraConfig = ''
# Disable index.html
index fully_disabled.html;
# Our beautiful autoindex theme
autoindex on;
autoindex_exact_size off;
autoindex_format xml;
xslt_string_param path $uri;
xslt_string_param hostname "Chaotic-AUR main node - Temeraire";
# Security
add_header X-XSS-Protection "1; mode=block" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "no-referrer-when-downgrade" always;
add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'; frame-ancestors 'self';" always;
add_header Permissions-Policy "interest-cohort=()" always;
# Locations
location ~* ^.+\.log {
default_type text/plain;
}
location ~* /repos/(chaotic-aur|garuda)/x86_64/(?!.*(chaotic-aur|garuda)\.(db|files)).+\.tar.* {
return 301 https://cf-builds.garudalinux.org$request_uri;
expires 2d;
}
location / {
xslt_string_param path $uri;
xslt_string_param hostname "Chaotic-AUR main node - Temeraire 🐉";
xslt_stylesheet "${garuda-lib.xslt_style}";
location /iso {
expires 2d;
return 301 https://iso.builds.garudalinux.org$request_uri;
}
}
'';
http3 = true;
root = "/srv/http/";
};
"cf-builds.garudalinux.org" = {
extraConfig = ''
location ~* /repos/(chaotic-aur|garuda)/x86_64/(?!.*(chaotic-aur|garuda)\.(db|files)).+\.tar.* {
add_header Cache-Control "max-age=150, stale-while-revalidate=150, stale-if-error=86400";
}
location ~* /repos/(chaotic-aur|garuda)/x86_64/(chaotic-aur|garuda)\.db.* {
add_header Cache-Control 'no-cache';
}
location /repos/chaotic-aur {
expires 5m;
error_page 403 =301 https://builds.garudalinux.org$request_uri;
error_page 404 =301 https://builds.garudalinux.org$request_uri;
}
location /repos/garuda {
expires 5m;
error_page 403 =301 https://builds.garudalinux.org$request_uri;
error_page 404 =301 https://builds.garudalinux.org$request_uri;
}
location / {
expires 2d;
return 301 https://builds.garudalinux.org$request_uri;
}
'';
http3 = true;
root = "/srv/http/";
};
"iso.builds.garudalinux.org" = {
extraConfig = ''
autoindex on;
autoindex_format xml;
xslt_string_param path $uri;
xslt_string_param hostname "Garuda Linux ISO Builds";
'';
locations."/".return = "307 https://builds.garudalinux.org";
locations."/iso" = {
root = "/var/garuda/buildiso";
extraConfig = ''
xslt_stylesheet "${garuda-lib.xslt_style}";
if ($symlink_target_rel != "") {
rewrite ^ https://$server_name/iso/$symlink_target_rel redirect;
}
if ($arg_sourceforge) {
rewrite ^/iso/(.*)$ https://sourceforge.net/projects/garuda-linux/files/$1? permanent;
}
if ($arg_r2) {
set $args "";
rewrite ^/iso/(.*)$ https://r2.garudalinux.org/iso/$1?r2request permanent;
}
break;
'';
};
};
};
};
# Explicitly open our firewall ports - HTTPS & rsyncd
networking.firewall.allowedTCPPorts = [ config.services.rsyncd.port 8384 ];
# Our rsyncd server
services.rsyncd = {
enable = true;
settings = {
chaotic = {
"read only" = "yes";
comment = "Chaotic-AUR repository";
exclude = "/chaotic-aur/archive/*** /chaotic-aur/logs/***";
path = "/srv/http/repos/";
};
chaotic-minimal = {
"read only" = "yes";
comment = "Chaotic-AUR repository minus largest packages";
exclude = "/chaotic-aur/archive/*** /chaotic-aur/logs/*** /chaotic-aur/x86_64/quartus* /chaotic-aur/x86_64/unrealtournament4* /chaotic-aur/x86_64/urbanterror*";
path = "/srv/http/repos/";
};
iso = {
path = "/var/garuda/buildiso/iso/";
comment = "ISO downloads";
"read only" = "yes";
};
global = {
"max connections" = 80;
"max verbosity" = 3;
"transfer logging" = true;
"use chroot" = false;
gid = "nobody";
uid = "nobody";
};
};
};
# Push chaotic to r2 hourly automatically
services.garuda-rclone.chaotic = {
src = "/srv/http/repos/";
dest = "r2:/mirror/repos";
config = garuda-lib.secrets.cloudflare.r2.rclone;
args = "--s3-upload-cutoff 5G --s3-chunk-size 4G --fast-list --s3-no-head --s3-no-check-bucket --ignore-checksum --s3-disable-checksum -u --use-server-modtime --delete-during --delete-excluded --include /*/x86_64/*.pkg.tar.zst --include /*/lastupdate --order-by modtime,ascending --stats-log-level NOTICE";
startAt = "hourly";
};
systemd.services.chaotic-rclone-inotify = {
wantedBy = [ "multi-user.target" ];
after = [ "network-online.target" ];
wants = [ "network-online.target" ];
# Get all file changes, upload pkg.tar.zst. Not more than 5 per 5 seconds queued and only one uploaded at the same time. Queue dropped if uploading takes longer than 15 seconds.
# This prevents the queue from getting overloaded with nonsense requests if that ever were to happen. The hourly sync should take care of this.
script = ''
upload() {
operation="''${1%%|*}"
path="''${1#*|}"
relative="$(realpath --relative-to="." "$path")"
relative="''${relative#./}"
destpath="r2:/mirror/$relative"
if [ "$operation" != "MOVED_FROM" ]; then
${pkgs.flock}/bin/flock -w 30 /tmp/chaotic-rclone-inotify.lock \
${pkgs.rclone}/bin/rclone copyto "$path" "$destpath" --s3-upload-cutoff 5G --s3-chunk-size 4G --s3-no-head --no-check-dest --s3-no-check-bucket --ignore-checksum --s3-disable-checksum --config "${garuda-lib.secrets.cloudflare.r2.rclone}" --stats-one-line -v
else
${pkgs.flock}/bin/flock -w 30 /tmp/chaotic-rclone-inotify.lock ${pkgs.rclone}/bin/rclone deletefile "$destpath" --s3-no-head --no-check-dest --s3-no-check-bucket --config "${garuda-lib.secrets.cloudflare.r2.rclone}" --stats-one-line -v
(
${pkgs.flock}/bin/flock -w 200 -s 200
${pkgs.curl}/bin/curl -s -X POST "https://api.cloudflare.com/client/v4/zones/$CF_ZONE_GARUDALINUX_ORG/purge_cache" -H "Authorization: Bearer $CF_CACHE_API_TOKEN" -H "Content-Type:application/json" --data "{\"files\":[\"https://r2.garudalinux.org/''${relative}\"]}"
sleep 0.5
) 200>/tmp/chaotic-rclone-inotify-invalidate.lock
fi
}
export -f upload
${pkgs.inotify-tools}/bin/inotifywait -m ./repos/*/x86_64 -e CLOSE_WRITE,MOVED_TO,MOVED_FROM --format "%e|%w%f" | \
${pkgs.gawk}/bin/awk '/\.pkg\.tar\.zst$/ { print $0; fflush(); }' | \
xargs -rP 0 -I % ${pkgs.bash}/bin/bash -c 'upload "%"'
'';
serviceConfig = {
EnvironmentFile = garuda-lib.secrets.cloudflare.apikeys;
Restart = "always";
WorkingDirectory = "/srv/http";
};
};
system.stateVersion = "23.05";
}
web-front
General
This container is used as reverse proxy for all of our public facing services.
Nix expression
{ garuda-lib
, sources
, lib
, ...
}:
let
allowOnlyCloudflared = config: (
config // {
listen = [
{
addr = "127.0.0.1";
port = 80;
}
];
extraConfig = (config.extraConfig or "") + ''
real_ip_header CF-Connecting-IP;
set_real_ip_from 127.0.0.1;
'';
}
);
# This is technically unecessary, but safety!
# This refers to the Cloudflare service "Cloudflare Access" to allow only specified users to access the service
allowOnlyCloudflareZerotrust = base_config:
let
config = allowOnlyCloudflared base_config;
in
config // {
extraConfig = config.extraConfig + ''
ssl_verify_client on;
underscores_in_headers off;
ssl_client_certificate ${sources.cloudflare-authenticated_origin_pull_ca};
'';
locations = lib.mapAttrs
(_: location: location // {
extraConfig = ''
if ($http_cf_access_authenticated_user_email = "") {
return 403;
}
'' + (location.extraConfig or "");
})
config.locations;
};
generateCloudflaredIngress = virtualHosts:
let
destination = "http://127.0.0.1:80";
toIngress = array: map (host: { name = host; value = destination; }) array;
isCloudflared = values: values ? listen && values.listen == (allowOnlyCloudflared { }).listen;
in
builtins.listToAttrs (lib.flatten (lib.mapAttrsToList (host: values: lib.optionals (isCloudflared values) (toIngress ([ host ] ++ (values.serverAliases or [ ])))) virtualHosts));
in
rec {
imports = sources.defaultModules ++ [ ../modules ];
# Reverse proxy for our docker-compose stack
services.nginx = {
enable = true;
virtualHosts = {
"cloud.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = {
"/" = {
extraConfig = ''
# Increase our buffer size to allow bigger up- & downloads
client_max_body_size 2048M;
proxy_max_temp_file_size 2048M;
proxy_request_buffering off;
# HSTS headers
add_header Strict-Transport-Security "max-age=31536000; includeSubdomains; preload" always;
# Allow accessing through trusted domain
set_real_ip_from 172.0.0.0/16;
'';
proxyPass = "https://10.0.5.100:443";
};
"/.well-known/carddav" = {
extraConfig = "expires 12h;";
return = "301 $scheme://$host/remote.php/dav";
};
"/.well-known/caldav" = {
extraConfig = "expires 12h;";
return = "301 $scheme://$host/remote.php/dav";
};
"/.well-known/webfinger" = {
return = "301 $scheme://$host/index.php/.well-known/webfinger";
extraConfig = ''
access_log off;
log_not_found off;
'';
};
"/.well-known/nodeinfo" = {
extraConfig = ''
access_log off;
log_not_found off;
'';
return = "301 $scheme://$host/index.php/.well-known/nodeinfo";
};
};
quic = true;
useACMEHost = "garudalinux.org";
};
"cloud-aio.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = {
"/" = {
extraConfig = ''
client_body_buffer_size 512k;
proxy_read_timeout 86400s;
client_max_body_size 0;
# Allow accessing through trusted domain
set_real_ip_from 172.0.0.0/16;
'';
proxyPass = "http://10.0.5.100:11000";
};
};
quic = true;
useACMEHost = "garudalinux.org";
};
"cloud-temp.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = {
"/" = {
extraConfig = ''
client_body_buffer_size 512k;
proxy_read_timeout 86400s;
client_max_body_size 0;
# Allow accessing through trusted domain
set_real_ip_from 172.0.0.0/16;
'';
proxyPass = "https://10.0.5.100:8080";
};
};
quic = true;
useACMEHost = "garudalinux.org";
};
"search.garudalinux.org" = allowOnlyCloudflared {
addSSL = true;
http3 = true;
locations = { "/" = { proxyPass = "http://10.0.5.110:5000"; }; };
quic = true;
useACMEHost = "garudalinux.org";
extraConfig = ''
${garuda-lib.nginxReverseProxySettings}
'';
};
"searx.garudalinux.org" = allowOnlyCloudflared {
addSSL = true;
http3 = true;
locations = { "/" = { proxyPass = "http://10.0.5.110:8080"; }; };
quic = true;
useACMEHost = "garudalinux.org";
extraConfig = ''
${garuda-lib.nginxReverseProxySettings}
'';
};
"librey.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = { "/" = { proxyPass = "http://10.0.5.110:8081"; }; };
quic = true;
useACMEHost = "garudalinux.org";
};
"ffsync.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = { "/" = { proxyPass = "http://10.0.5.100:5001"; }; };
quic = true;
useACMEHost = "garudalinux.org";
};
"start.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = { "/" = { proxyPass = "http://10.0.5.100:8083"; }; };
quic = true;
useACMEHost = "garudalinux.org";
};
"irc.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = { "/" = { proxyPass = "http://10.0.5.100:9000"; }; };
quic = true;
useACMEHost = "garudalinux.org";
};
"bin.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = { "/" = { proxyPass = "http://10.0.5.100:8082"; }; };
quic = true;
useACMEHost = "garudalinux.org";
};
"bitwarden.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = {
"/" = {
proxyPass = "http://10.0.5.100:8081";
};
};
quic = true;
serverAliases = [ "vault.garudalinux.org" ];
useACMEHost = "garudalinux.org";
};
"status.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = {
"/" = { tryFiles = "/status.html /status.html"; };
"=/status.html" = {
extraConfig = "expires 30d;";
root = "${sources.garuda-website}/internal";
};
};
quic = true;
useACMEHost = "garudalinux.org";
};
"stats.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = {
"/" = { tryFiles = "/stats.html /stats.html"; };
"=/stats.html" = {
extraConfig = "expires 30d;";
root = "${sources.garuda-website}/internal";
};
};
quic = true;
useACMEHost = "garudalinux.org";
};
"forum.garudalinux.org" = {
addSSL = true;
extraConfig = ''
client_max_body_size 100M;
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = {
"/" = { proxyPass = "http://10.0.5.70:80"; };
"/c/announcements/announcements-maintenance/45.json" = {
extraConfig = "expires 2m;";
proxyPass = "http://10.0.5.70:80";
};
};
quic = true;
useACMEHost = "garudalinux.org";
};
"social.garudalinux.org" = {
addSSL = true;
extraConfig = ''
client_max_body_size 100M;
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = {
"/" = {
proxyPass = "https://10.0.5.80:443";
};
"/.well-known/webfinger" = {
proxyPass = "https://10.0.5.80:443";
extraConfig = ''
if ($args ~* "resource=acct:(.*)@(chaotic.cx|social.garudalinux.org)$") {
set $w1 $1;
rewrite .* /.well-known/webfinger?resource=acct:[email protected]? break;
}
'';
};
};
quic = true;
useACMEHost = "garudalinux.org";
};
"social-video.garudalinux.org" = {
addSSL = true;
extraConfig = ''
client_max_body_size 100M;
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
location ~* .(mp4|webm)$ {
proxy_pass https://10.0.5.80:443;
}
'';
locations = {
"/" = { return = "301 https://social.garudalinux.org$request_uri"; };
};
http3 = true;
quic = true;
useACMEHost = "garudalinux.org";
};
"builds.garudalinux.org" = {
addSSL = true;
extraConfig = ''
proxy_buffering off;
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = {
"/" = {
proxyPass = "http://10.0.5.20:80";
};
"/logs/" = {
proxyPass = "http://10.0.5.140:8080/";
extraConfig = ''
proxy_buffering off;
proxy_read_timeout 330s;
'';
};
};
quic = true;
serverAliases = [ "cf-builds.garudalinux.org" "iso.builds.garudalinux.org" ];
useACMEHost = "garudalinux.org";
};
"element.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = {
"/" = { proxyPass = "http://10.0.5.100:8084"; };
};
quic = true;
useACMEHost = "garudalinux.org";
};
"wiki.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = { "/" = { proxyPass = "http://10.0.5.100:3001"; }; };
quic = true;
useACMEHost = "garudalinux.org";
};
"matrix.garudalinux.org" = {
addSSL = true;
http3 = true;
listen = [
{
addr = "0.0.0.0";
port = 443;
ssl = true;
}
{
addr = "0.0.0.0";
port = 8448;
ssl = true;
}
];
locations = {
"/" = {
extraConfig = "client_max_body_size 50M;";
proxyPass = "http://10.0.5.100:8008";
};
};
quic = true;
useACMEHost = "garudalinux.org";
};
"lemmy.garudalinux.org" = {
addSSL = true;
extraConfig = ''
${garuda-lib.setRealIpFromConfig}
${garuda-lib.nginxReverseProxySettings}
'';
http3 = true;
locations = {
"/" = {
proxyPass = "http://10.0.5.120:80";
};
};
quic = true;
useACMEHost = "garudalinux.org";
};
"lingva.garudalinux.org" = allowOnlyCloudflared {
addSSL = true;
http3 = true;
locations = {
"/" = {
proxyPass = "http://10.0.5.110:3002";
};
};
quic = true;
useACMEHost = "garudalinux.org";
extraConfig = ''
${garuda-lib.nginxReverseProxySettings}
'';
};
"reddit.garudalinux.org" = allowOnlyCloudflared {
addSSL = true;
http3 = true;
locations = {
"/" = {
proxyPass = "http://10.0.5.110:8082";
};
};
quic = true;
useACMEHost = "garudalinux.org";
extraConfig = ''
${garuda-lib.nginxReverseProxySettings}
'';
};
"pgadmin.garudalinux.net" = allowOnlyCloudflareZerotrust {
locations = {
"/" = {
extraConfig = ''
${garuda-lib.nginxReverseProxySettings}
proxy_pass http://10.0.5.50:5050;
proxy_set_header X-Forwarded-User $http_cf_access_authenticated_user_email;
proxy_hide_header Cache-Control;
proxy_hide_header Expires;
add_header Cache-Control 'no-store';
'';
};
};
};
"syncthing-build.garudalinux.net" = allowOnlyCloudflareZerotrust {
extraConfig = ''
${garuda-lib.nginxReverseProxySettings}
'';
locations = {
"/" = {
extraConfig = ''
proxy_pass http://10.0.5.20:8384;
proxy_set_header Authorization "Basic ${garuda-lib.secrets.syncthing.esxi-build.credentials.base64}";
'';
};
};
};
"matrixadmin.garudalinux.net" = allowOnlyCloudflareZerotrust {
extraConfig = ''
${garuda-lib.nginxReverseProxySettings}
'';
locations = {
"/" = {
proxyPass = "http://10.0.5.100:8085";
};
};
};
};
};
services.garuda-cloudflared = {
enable = true;
ingress = {
# "example.garudalinux.net" = "http://10.0.5.100:8085";
} // (generateCloudflaredIngress services.nginx.virtualHosts);
tunnel-credentials =
garuda-lib.secrets.cloudflare.cloudflared.esxi-web.cred;
};
system.stateVersion = "23.05";
}
garuda-mail (Netcup VPS)
General
This system mainly consists of the simple-nixos-mailserver. Its only purpose is providing a mail service to team members. The current config looks like this. In case of issues, the documentation can be consulted.
Mail server setup
The mail server details are as follows:
- host:
mail.garudalinux.net - username: full email address
- password: given password
- incoming: IMAP via
993(SSL) - outgoing: SMTP via
587/465(STARTTLS/SSL)
Additionally, it is possible to make use of the Roundcube-powered web interface.
Roundcube
Roundcube is used to provide a web interface for our mail accounts. It features a few plugins to enhance the general user experience.
Plugins
- attachment_reminder - reminds about forgotten attachments
- authres_status - checks for whether SPF/DKIM/DMARC match the sending domain
- carddav - allows adding a CardDAV contact book as source (eg. Nextcloud)
- contextmenu - adds a right click context menu to the most pages
- custom_from - allows customizing from address
- managesieve - allows managing Sieve rules, which automatically sort incoming mails based on rules
- newmail_notifier - new mail notifier for desktops
- persistent_login - alows storing a persistent login cookie for no more login prompts
- thunderbird_labels - shows Thunderbird labels
- zipdownload - allows downloading all attachments at once
Backups
Backups are happening daily via Borg. A Hetzner storage box is used to store multiple generations of backups.
Creating a new user
A new user can be created be adding a new loginAccounts value and supplying the password via secrets.
We make use of hashedPasswordFile, therefore new hashes can be generated by running nix-shell -p mkpasswd --run 'mkpasswd -sm bcrypt'. Add it to the secrets, then execute deploy and apply.
Don't forget to commit both changes.
Issues and their solution
Local DNS resolver failing to start
Simple NixOS mailserver runs a local DNS server to prevent the log filling up with junk (source). There can be cases of the persisted files need to be deleted in order for the service to recover from dumping core. See this issue for reference.
Nix expression
{ config
, lib
, pkgs
, ...
}:
let
authres_status = pkgs.roundcubePlugins.roundcubePlugin rec {
pname = "authres_status";
version = "0.6.3";
src = pkgs.fetchzip {
url = "https://github.com/pimlie/authres_status/archive/refs/tags/${version}.zip";
hash = "sha256-WebJiN0vRkvc0AKvMm+inK3FY37R04q3y/0rFoiUW6A=";
};
};
in
{
imports = [
../modules
./garuda-mail/hardware-configuration.nix
];
# Base configuration
networking.interfaces.ens3.ipv4.addresses = [{
address = "94.16.112.218";
prefixLength = 22;
}];
networking.hostName = "garuda-mail";
networking.defaultGateway = "94.16.112.3";
# GRUB
boot.loader.grub.devices = [ "/dev/vda" ];
# Backup configurations to Hetzner storage box
programs.ssh.macs = [ "hmac-sha2-512" ];
services.borgbackup.jobs = {
backupToHetzner = {
compression = "auto,zstd";
doInit = true;
encryption = {
mode = "repokey-blake2";
passCommand = "cat /var/garuda/secrets/backup/repo_key";
};
environment = {
BORG_RSH = "ssh -i /var/garuda/secrets/backup/ssh_garuda-mail -p 23";
};
paths = [ config.mailserver.mailDirectory "/var/dkim" ];
prune.keep = {
within = "1d";
daily = 5;
weekly = 2;
monthly = 1;
};
repo = "[email protected]:./garuda-mail";
startAt = "daily";
};
};
# NixOS Mailserver
mailserver = {
certificateScheme = "acme-nginx";
dmarcReporting = {
domain = "garudalinux.org";
enable = true;
organizationName = "Garuda Linux";
};
domains = [ "garudalinux.org" "chaotic.cx" "dr460nf1r3.org" ];
enable = true;
enableManageSieve = true;
# Forwards (mostly chaotic.cx only)
forwards =
{
"[email protected]" = "[email protected]";
"[email protected]" = "[email protected]";
"[email protected]" = "[email protected]";
"[email protected]" = "[email protected]";
};
fqdn = "mail.garudalinux.net";
fullTextSearch = {
enable = true;
enforced = "body";
indexAttachments = true;
memoryLimit = 512;
};
# To create the password hashes, use nix-shell -p mkpasswd --run 'mkpasswd -sm bcrypt'
loginAccounts = {
# garudalinux.org
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/cloudatgl";
sendOnly = true;
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/complaintsatgl";
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/dr460nf1r3atgl";
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/filoatgl";
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/gitlabatgl";
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/mastodonatgl";
sendOnly = true;
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/namanatgl";
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/noreplyatgl";
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/rohitatgl";
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/securityatgl";
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/sgsatgl";
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/spam-reportsatgl";
};
"[email protected]" = {
aliases = [
"[email protected]"
"[email protected]"
"[email protected]"
"[email protected]"
];
hashedPasswordFile = "/var/garuda/secrets/mail/teamatgl";
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/tneatgl";
};
"[email protected]" = {
hashedPasswordFile = "/var/garuda/secrets/mail/yorperatgl";
};
# chaotic.cx
"[email protected]" = {
aliases = [ "[email protected]" ];
hashedPasswordFile = "/var/garuda/secrets/mail/actionsatcx";
};
"[email protected]" = {
aliases = [
"[email protected]"
"[email protected]"
"[email protected]"
];
hashedPasswordFile = "/var/garuda/secrets/mail/nicoatcx";
};
# dr460nf1r3.org
"[email protected]" = {
aliases = [ "@dr460nf1r3.org" ];
catchAll = [ "dr460nf1r3.org" ];
hashedPasswordFile = "/var/garuda/secrets/mail/nicoatdf";
};
};
indexDir = "/var/lib/dovecot/indices";
monitoring = {
alertAddress = "[email protected]";
enable = true;
};
rebootAfterKernelUpgrade.enable = true;
};
# Fix dovecot errors caused by failed scudo allocations
environment.memoryAllocator.provider = lib.mkForce "libc";
# Postmaster alias
services.postfix.postmasterAlias = "[email protected]";
# Web UI
services.roundcube = {
enable = true;
# this is the url of the vhost, not necessarily the same as the fqdn of
# the mailserver
hostName = "mail.garudalinux.net";
extraConfig = ''
# starttls needed for authentication, so the fqdn required to match
# the certificate
$config['smtp_server'] = "tls://${config.mailserver.fqdn}";
$config['smtp_user'] = "%u";
$config['smtp_pass'] = "%p";
'';
package = pkgs.roundcube.withPlugins (
plugins: [
authres_status
plugins.carddav
plugins.contextmenu
plugins.custom_from
plugins.persistent_login
plugins.thunderbird_labels
]
);
plugins = [
"attachment_reminder" # Roundcube internal plugin
"authres_status"
"carddav"
"contextmenu"
"custom_from"
"managesieve" # Roundcube internal plugin
"newmail_notifier" # Roundcube internal plugin
"persistent_login"
"thunderbird_labels"
"zipdownload" # Roundcube internal plugin
];
};
# At least try to prevent the insane spam of login attempts
services.openssh.ports = [ 1022 ];
# This mostly sends annoying notifications because SSH port is non-default
services.monit.enable = false;
system.stateVersion = "22.05";
}
Repositories
Notifications for new events at GitLab
Since GitLab has an inbuilt Telegram integration, we can leverage this feature to send notifications to our a dedicated Telegram development updates channel. Posts are send for all kinds of relevent, but non-confidential events like commits, comments or new merge requests. Failed pipelines would also be reported here.
Backing up current repositories
Current repositories may be backed up using ghorg. In order to use ghorg, one needs a GitLab access token and the application itself. To generate a fitting token, follow these instructions.
ghorg clone --scm gitlab --token "glpat-1234567890" garuda-linux # regular system
nix run nixpkgs#ghorg -- clone --scm gitlab --token "glpat-1234567890" garuda-linux # oneliner on Nix
Archive
We have an archive repository for all files, which are no longer needed for our current operations. It contains old PKGBUILDs and settings packages, eg. the state of the ones before we moved to a unified PKGBUILD repository.
PKGBUILDs
Types of PKGBUILDs
There are 2 types of repos packaging-wise:
- The ones that have all required files in the new pkgbuilds repo and don't reference any external repo in PKGBUILDs
source() - The ones requiring external repositories as source. These are listed in the SOURCES files below, packages not listed here are automatically packages of the first category:
This file provides the needed information to check for the new version with the scheme $repourl $pkgbuildPathInPkgbuildsRepo $GitlabProjectId
Releasing a new version
This means executing the following for doing changes and releasing a new version:
- Would be modified directly in the new pkgbuilds repo, along with their source.
Versions are bumped in the PKGBUILD itself and deployments need to happen by increasing
pkgver+ supplying a fitting commit message (append [deploy pkgname ] to it) - In case of modifying these, one would make the changes to the source files repo (not the new PKGBUILDs one).
Then, if a new version should be built, one would push the corresponding tag to that repo (omitting "v", adding v breaks the PKGBUILD!).
That's everything needed in case no packaging changes (adding new dependencies for example) that require changing the PKGBUILD occur.
The half-hourly pipeline of the PKGBUILD repo then checks for the existence of a new tag.
Once a new one gets detected, the PKGBUILD gets updated and deployment occurs via
[deploy *]in the commit message. If PKGBUILD changes need to be implemented, this would of course indicate doing it as described in 1. This would increase pkgrel only and not the actual version.
There are currently three bash scripts responsible for CI/CD:
- Checking PKGBUILDs/code style
- Updating the package versions automatically
- Triggering automated deployments via commit message
Past pipeline runs may be reviewed by visiting the pipelines page.
Discourse
Discourse is the application we use to host our forum.
Documentation
Building it
The documentation is created by using mdBook, which generates Markdown files and generates HTML pages for them. The documentation can be build by running:
nix build .#docs # plain simple
The files can then be found at ./result/, which is a symlink to the corresponding path in /nix/store.
mdBook is also able automatically serve the current content and update it automatically whenever a change is detected.
This makes testing and previewing content easy.
mdbook serve --open # the latter additionally opens the website in a browser
Useful information
mdBook syntax
While the general syntax for writing Markdown applies to mdBook, it has several extensions beyond the standard CommonMark specification.
Especially importing code blocks as Markdown is really handy to keep content always up-to-date and helps providing a full text searchable code documentation.
Updating mdBook plugins contents
Some of the mdBook parts are plugins that need their content to be updated from time to time. Namely, thats:
- mdbook-admonish: run
mdbook-admonishinside thedocsfolder - mdbook-emojicodes: works without CSS, so no updates needed
- mdbook-catppuccin: run
mdbook-catppuccininside thedocsfolder (might need to grab binary from its website, no Nix package available yet)
Deployment
Deployment to Cloudflare pages automated and happens whenever a commit to main occurs.
A GitHub actions workflow builds and pushes it to the cf-pages branch, which will then be used by the Cloudflare pages app to deploy the new version from.
---
name: Cloudflare pages
on:
push:
branches: [main]
paths: [docs/**, README.md]
permissions:
contents: write
jobs:
build-and-deploy:
concurrency: ci-${{ github.ref }}
runs-on: ubuntu-latest
steps:
- name: Checkout 🛎️
uses: actions/checkout@v4
- name: Setup mdBook 📜
uses: peaceiris/actions-mdbook@v2
with:
mdbook-version: latest
- name: Install further deps 📦
run: |
sudo apt-get install -y --no-install-recommends cargo
cargo install mdbook-admonish
cargo install mdbook-emojicodes
PATH=$HOME/.cargo/bin:$PATH
- name: Install and Build 🔧
run: cd docs && mdbook build
- name: Deploy 🚀
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_branch: cf-pages
publish_dir: docs/book
Issues and their solution
Sidebar or something else on the documentation doesn't work as expected
Chances are that the custom CSS parts need to be rebased to a newer version.
They can be found in ./docs/theme/css and the only addition we made here is to use the Fira Sans font instead of the default one.
To rebase against a newer version comment out dditional-css in ./docs/book.toml and move the css folder somewhere else temporarily.
After that, run mdbook build inside the docs folder. The new CSS files can now be found inside the ./docs/book/css folder.
Copy those to the ./docs/theme/css folder and alter the occurrences of font settings to include Fira Sans (or run a diff to find out where).
After uncommenting additional-css in book.toml, run mdbook build again to verify nothing got broken along the way.
Tailscale
Our currently access policies look as follows:
// This tailnet's ACLs are maintained in https://gitlab.com/garuda-linux/infra-nix
{
// Define access control lists for users, groups, autogroups, tags,
// Tailscale IP addresses, and subnet ranges
"acls": [
// All servers can connect to each other, use exit nodes and oracle-dragon as DNS
{
"action": "accept",
"src": ["tag:infra"],
"dst": ["tag:infra:*", "autogroup:internet:*", "100.86.102.115:*"],
},
// Tailscale admins can access every device
{
"action": "accept",
"src": ["autogroup:admin"],
"dst": ["*:*"],
},
// Shared out nodes can be accessed on SSH / Mosh ports
{
"action": "accept",
"src": ["autogroup:shared"],
"dst": ["*:22,222-230,666,60000-61000"],
},
// Let the chaotic nodes connect to chaotic-v4's Redis (build distribution)
{
"action": "accept",
"src": ["tag:chaotic-node"],
"dst": ["100.75.227.149:22,6379"],
},
],
// Current infra maintainers
"groups": {
"group:admins": ["dr460nf1r3@github", "JustTNE@github"],
},
// Define a tag to use as destinations
"tagOwners": {
// Admins may apply the "infra" tag
"tag:infra": ["group:admins"],
"tag:chaotic-node": ["group:admins"],
},
}
Garuda Linux Code of Conduct
Thank you for being a part of the Garuda Linux community. We value your participation and want everyone to have an enjoyable and fulfilling experience. Accordingly, all participants are expected to follow this Code of Conduct, and to show respect, understanding, and consideration to one another. Thank you for helping make this a welcoming, friendly community for everyone.
Scope
This Code of Conduct applies to all Garuda Linux community spaces, including, but not limited to:
- Code repositories -
gitlab.com/garuda-linuxandgithub.com/garuda-linux - Garuda Linux's Telegram channels and groups (including bridges to Matrix)
- Mailing
*@garudalinux.org - Community spaces hosted on
garudalinux.orginfrastructure
Communication channels and private conversations that are normally out of scope may be considered in scope if a Garuda Linux participant is being stalked or harassed. Social media conversations may be considered in-scope if the incident occurred under a Garuda Linux related hashtag, or when an official Garuda Linux account on social media is tagged, or within any other discussion about Garuda Linux. The Garuda Linux's staff reserves the right to take actions against behaviors that happen in any context, if they are deemed to be relevant to the Garuda Linux project and its participants.
All participants in Garuda Linux community spaces are subject to the Code of Conduct. This includes founding members, staff members, corporate sponsors, and paid employees. This also includes volunteers, maintainers, leaders, contributors, contribution reviewers, issue reporters, Garuda Linux users, and anyone participating in discussion in Garuda Linux community spaces.
Reporting an Incident
If you believe that someone is violating the Code of Conduct, or have any other concerns, please contact [email protected].
Our Standards
The Garuda Linux community is dedicated to providing a positive experience for everyone, regardless of:
- age
- body size
- caste
- citizenship
- disability
- education
- ethnicity
- familial status
- gender expression
- gender identity
- genetic information
- immigration status
- level of experience
- nationality
- personal appearance
- pregnancy
- race
- religion
- sex characteristics
- sexual orientation
- sexual identity
- socio-economic status
- tribe
- veteran status
Community Guidelines
Behaviors that contribute to creating a positive environment include:
- Be friendly. Use welcoming and inclusive language.
- Be empathetic. Be respectful of others' viewpoints and experiences.
- Be respectful. Express disagreements in a polite and constructive manner.
- Be considerate. Focus on what is best for the community. Keep discussions around technology choices constructive and respectful. Remember that decisions are often a difficult choice between competing priorities.
- Be patient and generous. If someone asks for help, it is because they need it. When documentation is available that answers the question, politely point them to it. If the question is off-topic, suggest a more appropriate online space to seek help.
- Try to be concise. Read the discussion before commenting in order to not repeat a point that has been made.
Inappropriate Behavior
We want all participants in the Garuda Linux community have the best possible experience they can. Community members asked to stop any inappropriate behavior are expected to comply immediately.
Inappropriate behaviors include, but are not limited to:
- Deliberate intimidation, stalking, or following.
- Sustained disruption of online discussion, talks, or other events. Sustained disruption of events, online discussions, or meetings, including talks and presentations, will not be tolerated. This includes 'Talking over' or 'heckling' event speakers or influencing crowd actions that cause hostility in event sessions. Sustained disruption also includes drinking alcohol to excess or using recreational drugs to excess, or pushing others to do so.
- Harassment of people who don't drink alcohol or other legal substances. We do not tolerate derogatory comments about those who abstain from alcohol or other legal substances. We do not tolerate pushing people to drink, talking about their abstinence or preferences to others, or pressuring them to drink - physically or through jeering.
- Sexist, racist, homophobic, transphobic, ableist language or otherwise exclusionary language. This includes deliberately referring to someone by a gender that they do not identify with, and/or questioning the legitimacy of an individual's gender identity. If you're unsure if a word is derogatory, don't use it. This also includes repeated subtle and/or indirect discrimination.
- Unwelcome sexual attention or behavior that contributes to a sexualized environment. This includes sexualized comments, jokes or imagery in interactions, communications or presentation materials, as well as inappropriate touching, groping, or sexual advances. Sponsors should not use sexualized images, activities, or other material. Meetup organizing staff and other volunteer organizers should not use sexualized clothing/uniforms/costumes, or otherwise create a sexualized environment.
- Unwelcome physical contact. This includes touching a person without permission, including sensitive areas such as their hair, pregnant stomach, mobility device (wheelchair, scooter, etc) or tattoos. This also includes physically blocking or intimidating another person. Physical contact without affirmative consent is not acceptable. This includes sharing or distribution of sexualized images or text.
- Violence or threats of violence. Violence and threats of violence are not acceptable - online or offline. This includes incitement of violence toward any individual, including encouraging a person to commit self-harm. This also includes posting or threatening to post other people's personally identifying information ("doxxing") online.
- Influencing or encouraging inappropriate behavior. If you influence or encourage another person to violate the Code of Conduct, you may face the same consequences as if you had violated the Code of Conduct.
Safety versus Comfort
The Garuda Linux community prioritizes marginalized people's safety over privileged people's comfort. The following are not against the Code of Conduct.
- "Reverse"-isms, including "reverse racism," "reverse sexism," and "cisphobia"
- Reasonable communication of boundaries, such as "leave me alone," "go away," or "I'm not discussing this with you."
- Criticizing racist, sexist, cissexist, or otherwise oppressive behavior or assumptions
- Communicating boundaries or criticizing oppressive behavior in a "tone" you don't find congenial
If you have questions about the above statements, please read GNOME Foundation's document on Supporting Diversity.
Outreach and diversity efforts directed at under-represented groups are permitted under the code of conduct. For example, a social event for women would not be classified as being outside the Code of Conduct under this provision.
Basic expectations for conduct are not covered by the "reverse-ism clause" and would be enforced irrespective of the demographics of those involved. For example, racial discrimination will not be tolerated, irrespective of the race of those involved. Nor would unwanted sexual attention be tolerated, whatever someone's gender or sexual orientation. Members of our community have the right to expect that participants in the project will uphold these standards.
If a participant engages in behavior that violates this code of conduct, the Garuda Linux's staff may take any action they deem appropriate. In cases involving the staff or founding members the immediate action is expelishment.
Procedure for Handling Incidents
You can make a report by emailing [email protected].
If you make a report via email, we hope you can provide us with some information that will help us identify the reported person. If you don’t remember all the details, we still encourage you to make a report.
We encourage you to include the following information in your report:
- Your contact info (so we can get in touch with you if we need to follow up)
- Date and time of the incident
- Whether the incident is ongoing
- Which online community and which part of the online community space it occurred in
- Description of the incident
- Identifying information of the reported person such as name, online username, handle, email address, or IP address
- A link to the conversation
- Any logs or screenshots of the conversation
- Additional circumstances surrounding the incident
- Other people involved in or witnesses to the incident and their contact information or # Garuda Linux Code of Conduct
Thank you for being a part of the Garuda Linux community. We value your participation and want everyone to have an enjoyable and fulfilling experience. Accordingly, all participants are expected to follow this Code of Conduct, and to show respect, understanding, and consideration to one another. Thank you for helping make this a welcoming, friendly community for everyone.
Scope
This Code of Conduct applies to all Garuda Linux community spaces, including, but not limited to:
- Code repositories -
gitlab.com/garuda-linuxandgithub.com/garuda-linux - Garuda Linux's Telegram channels and groups (including bridges to Matrix)
- Mailing
*@garudalinux.org - Community spaces hosted on
garudalinux.orginfrastructure
Communication channels and private conversations that are normally out of scope may be considered in scope if a Garuda Linux participant is being stalked or harassed. Social media conversations may be considered in-scope if the incident occurred under a Garuda Linux related hashtag, or when an official Garuda Linux account on social media is tagged, or within any other discussion about Garuda Linux. The Garuda Linux's staff reserves the right to take actions against behaviors that happen in any context, if they are deemed to be relevant to the Garuda Linux project and its participants.
All participants in Garuda Linux community spaces are subject to the Code of Conduct. This includes founding members, staff members, corporate sponsors, and paid employees. This also includes volunteers, maintainers, leaders, contributors, contribution reviewers, issue reporters, Garuda Linux users, and anyone participating in discussion in Garuda Linux community spaces.
Reporting an Incident
If you believe that someone is violating the Code of Conduct, or have any other concerns, please contact [email protected].
Our Standards
The Garuda Linux community is dedicated to providing a positive experience for everyone, regardless of:
- age
- body size
- caste
- citizenship
- disability
- education
- ethnicity
- familial status
- gender expression
- gender identity
- genetic information
- immigration status
- level of experience
- nationality
- personal appearance
- pregnancy
- race
- religion
- sex characteristics
- sexual orientation
- sexual identity
- socio-economic status
- tribe
- veteran status
Community Guidelines
Behaviors that contribute to creating a positive environment include:
- Be friendly. Use welcoming and inclusive language.
- Be empathetic. Be respectful of others' viewpoints and experiences.
- Be respectful. Express disagreements in a polite and constructive manner.
- Be considerate. Focus on what is best for the community. Keep discussions around technology choices constructive and respectful.
Remember that decisions are often a difficult choice between competing priorities. - Be patient and generous. If someone asks for help, it is because they need it. When documentation is available that answers the question, politely point them to it. If the question is off-topic, suggest a more appropriate online space to seek help.
- Try to be concise. Read the discussion before commenting in order to not repeat a point that has been made.
Inappropriate Behavior
We want all participants in the Garuda Linux community have the best possible experience they can. Community members asked to stop any inappropriate behavior are expected to comply immediately.
Inappropriate behaviors include, but are not limited to:
- Deliberate intimidation, stalking, or following.
- Sustained disruption of online discussion, talks, or other events. Sustained disruption of events, online discussions, or meetings, including talks and presentations, will not be tolerated. This includes 'Talking over' or 'heckling' event speakers or influencing crowd actions that cause hostility in event sessions. Sustained disruption also includes drinking alcohol to excess or using recreational drugs to excess, or pushing others to do so.
- Harassment of people who don't drink alcohol or other legal substances. We do not tolerate derogatory comments about those who abstain from alcohol or other legal substances. We do not tolerate pushing people to drink, talking about their abstinence or preferences to others, or pressuring them to drink - physically or through jeering.
- Sexist, racist, homophobic, transphobic, ableist language or otherwise exclusionary language. This includes deliberately referring to someone by a gender that they do not identify with, and/or questioning the legitimacy of an individual's gender identity. If you're unsure if a word is derogatory, don't use it. This also includes repeated subtle and/or indirect discrimination.
- Unwelcome sexual attention or behavior that contributes to a sexualized environment. This includes sexualized comments, jokes or imagery in interactions, communications or presentation materials, as well as inappropriate touching, groping, or sexual advances. Sponsors should not use sexualized images, activities, or other material. Meetup organizing staff and other volunteer organizers should not use sexualized clothing/uniforms/costumes, or otherwise create a sexualized environment.
- Unwelcome physical contact. This includes touching a person without permission, including sensitive areas such as their hair, pregnant stomach, mobility device (wheelchair, scooter, etc) or tattoos. This also includes physically blocking or intimidating another person. Physical contact without affirmative consent is not acceptable. This includes sharing or distribution of sexualized images or text.
- Violence or threats of violence. Violence and threats of violence are not acceptable - online or offline. This includes incitement of violence toward any individual, including encouraging a person to commit self-harm. This also includes posting or threatening to post other people's personally identifying information ("doxxing") online.
- Influencing or encouraging inappropriate behavior. If you influence or encourage another person to violate the Code of Conduct, you may face the same consequences as if you had violated the Code of Conduct.
Safety versus Comfort
The Garuda Linux community prioritizes marginalized people's safety over privileged people's comfort. The following are not against the Code of Conduct.
- "Reverse"-isms, including "reverse racism," "reverse sexism," and "cisphobia"
- Reasonable communication of boundaries, such as "leave me alone," "go away," or "I'm not discussing this with you."
- Criticizing racist, sexist, cissexist, or otherwise oppressive behavior or assumptions
- Communicating boundaries or criticizing oppressive behavior in a "tone" you don't find congenial
If you have questions about the above statements, please read GNOME Foundation's document on Supporting Diversity.
Outreach and diversity efforts directed at under-represented groups are permitted under the code of conduct. For example, a social event for women would not be classified as being outside the Code of Conduct under this provision.
Basic expectations for conduct are not covered by the "reverse-ism clause" and would be enforced irrespective of the demographics of those involved. For example, racial discrimination will not be tolerated, irrespective of the race of those involved. Nor would unwanted sexual attention be tolerated, whatever someone's gender or sexual orientation. Members of our community have the right to expect that participants in the project will uphold these standards.
If a participant engages in behavior that violates this code of conduct, the Garuda Linux's staff may take any action they deem appropriate. In cases involving the staff or founding members the immediate action is expelishment.
Procedure for Handling Incidents
You can make a report by emailing [email protected].
If you make a report via email, we hope you can provide us with some information that will help us identify the reported person. If you don’t remember all the details, we still encourage you to make a report.
We encourage you to include the following information in your report:
- Your contact info (so we can get in touch with you if we need to follow up)
- Date and time of the incident
- Whether the incident is ongoing
- Which online community and which part of the online community space it occurred in
- Description of the incident
- Identifying information of the reported person such as name, online username, handle, email address, or IP address
- A link to the conversation
- Any logs or screenshots of the conversation
- Additional circumstances surrounding the incident
- Other people involved in or witnesses to the incident and their contact information or description
License
The Garuda Linux Code of Conduct is licensed under a Creative Commons Attribution Share-Alike 3.0 Unported License.
Attribution
The Garuda Linux Code of Conduct was forked from GNOME Foundation's Code of Conduct (last modified 2020-10-01), which is under a Creative Commons license. See the original page for the original attributions. description
Privacy policy for Garuda Linux
About this document
This Privacy Policy governs the manner in which Garuda Linux collects, uses, maintains and discloses information collected from users (each, a “User”) of our website and web services..
What information do we collect?
We collect information from you when you register on our site and gather data when you participate in the forum by reading, writing, and evaluating the content shared here.
When registering on our site, you may be asked to enter your name and e-mail address. You may, however, visit our site without registering. Your e-mail address will be verified by an email containing a unique link. If that link is visited, we know that you control the e-mail address. Your IP address will be checked against a database of known spammers to prevent such actions.
If you contact us directly, we may receive additional information about you such as your name, email address, the contents of the message and/or attachments you may send us, and any other information you may choose to provide.
When registered and posting, we record the IP address that the post originated from. We also may retain server logs which include the IP address of every request to our server, which will be purged after 30 days.
What do we use your information for?
Any of the information we collect from you may be used in one of the following ways:
- To provide, operate, and maintain our infrastructure
- To allow using our services that require a login, as well as to provide convenience features such as staying logged in or keeping personally chosen settings.
- To send periodic emails that are generated by our services such as the forum, which may however be turned off if desired.
We have no interest in your data and only store the minimum needed to operate the services we provide to our users.
How do we protect your information?
We implement a variety of security measures to maintain the safety of your personal information when you enter, submit, or access your personal information.
What is your data retention policy?
We will make a good faith effort to:
- Retain server logs containing the IP address of all requests to this server no more than 90 days.
- Retain the IP addresses associated with registered users and their posts no more than 5 years.
Third Party Privacy Policies
Garuda Linux's Privacy Policy does not apply to some of the services we utilize in our infrastructure. Thus, we are advising you to consult the respective Privacy Policies of these third-party services for more detailed information.
This includes, but may not be limited to:
- Cloudflare to protect against common threats and enhance our infrastructure
- Hetzner as server and backup storage provider (located in Germany)
- Google Translate to offer translations on our website
- OpenCollective, Liberapay and Paypal to allow the collection of donations that sustain our infrastructure
Cookies
Our Site may use “cookies” to enhance User experience. User’s web browser places cookies on their hard drive for record-keeping purposes and sometimes to track information about them. The user may choose to set their web browser to refuse cookies or to alert you when cookies are being sent. If they do so, note that some parts of the Site may not function properly.
Sharing your personal information
We do not sell, trade, or rent User’s personal identification information to others.
How long do we retain your data
If you leave a comment, the comment and its metadata are retained indefinitely. This is so we can recognize and approve any follow-up comments automatically instead of holding them in a moderation queue.
For users that register on our website (if any), we also store the personal information they provide in their user profile. All users can see, edit, or delete their personal information at any time (except they cannot change their username). Website administrators can also see and edit that information.
Embedded content from other websites
Articles on this site may include embedded content (e.g. videos, images, articles, etc.). Embedded content from other websites behaves in the exact same way as if the visitor has visited the other website.
These websites may collect data about you, use cookies, embed additional third-party tracking, and monitor your interaction with that embedded content, including tracing your interaction with the embedded content if you have an account and are logged in to that website.
Free software
Garuda Linux develops free software. All our tools are and will always be free software. Garuda Linux is part of OIN since November 2020. The current license can be viewed here. Additional information about packages covered by this license can be viewed here.
If you want to check the license of a package, you can do so with Pacman.
What rights you have over your data
If you have an account on this site or have left comments, you can request to receive an exported file of the personal data we hold about you, including any data you have provided to us. You can also request that we erase any personal data we hold about you. This does not include any data we are obliged to keep for administrative, legal, or security purposes.
Children's Information
Another part of our priority is adding protection for children while using the internet. We encourage parents and guardians to observe, participate in, and/or monitor and guide their online activity.
Garuda Linux does not knowingly collect any personal identifiable information from children under the age of 13. If you think that your child provided this kind of information on our website, we strongly encourage you to contact us immediately and we will do our best efforts to promptly remove such information from our records.
Changes to This Privacy Policy
We may update our Privacy Policy from time to time. Thus, we advise you to review this page periodically for any changes. We will notify you of any changes by posting the new Privacy Policy on this page. These changes are effective immediately, after they are posted on this page.
Your acceptance of these terms
By using this Site, you signify your acceptance of this policy. If you do not agree to this policy, please do not use our services. Your continued use of them following the posting of changes to this policy will be deemed your acceptance of those changes.
Contact Us
If you have any questions about this Privacy Policy, the practices of this site, or your dealings with this site, please contact us via email.
This privacy policy has been updated in September 2023.
Security Policy
If any vulnerability or security flaw is discovered please contact us directly via [email protected].
We will try to respond within 24-48 hours on a best-effort basis.