hosting/selfhostblocks
1
0
Fork 0
Code Issues Activity

move all monitoring documentation to the docs/ folder

Browse source
This commit is contained in:
ibizaman 2023-11-26 09:49:40 -08:00
parent a4a8a2adad
commit 35ae4481f2
2 changed files with 66 additions and 37 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

38
README.md
View file

@ -326,43 +326,7 @@ See the [`ldap.nix`](./modules/ldap.nix) and [`authelia.nix`](./modules/authelia
### Deploy the full Grafana, Prometheus and Loki suite
This is not a prerequisite for anything and could be enabled just for debugging.
```nix
shb.monitoring = {
enable = true;
subdomain = "grafana";
inherit domain;
};
```
With that, Grafana, Prometheus, Loki and Promtail are setup! You can access `Grafana` at
`grafana.example.com`.
A few Prometheus metrics scrapers are setup automatically:
- node - cpu, memory, disk I/O, network I/O and a few others of the computer
- smartctl - hard drive health
- prometheus_internal - scraping jobs health
- nginx
- dnsmasq (if the service is enabled)
The following Loki logs scraper is setup automatically:
- systemd journal
I intend to provide more options so that you could for example tweak data retention.
Also, since all logs are now stored in Loki, you can probably reduce the systemd journal retention
time with:
```nix
# See https://www.freedesktop.org/software/systemd/man/journald.conf.html#SystemMaxUse=
services.journald.extraConfig = ''
SystemMaxUse=2G
SystemKeepFree=4G
SystemMaxFileSize=100M
MaxFileSec=day
'';
```
See [docs/blocks/monitoring.md](docs/blocks/monitoring.md).
### Set up network tunnel with VPN and Proxy

65
docs/blocks/monitoring.md
View file

@ -6,6 +6,71 @@ This block sets up the monitoring stack for Self Host Blocks. It is composed of:
- Prometheus as the database for metrics.
- Loki as the database for logs.
## Configuration
```nix
shb.monitoring = {
enable = true;
subdomain = "grafana";
inherit domain;
contactPoints = [ "me@example.com" ];
adminPasswordFile = config.sops.secrets."monitoring/admin_password".path;
secretKeyFile = config.sops.secrets."monitoring/secret_key".path;
};
sops.secrets."monitoring/admin_password" = {
sopsFile = ./secrets.yaml;
mode = "0400";
owner = "grafana";
group = "grafana";
restartUnits = [ "grafana.service" ];
};
sops.secrets."monitoring/secret_key" = {
sopsFile = ./secrets.yaml;
mode = "0400";
owner = "grafana";
group = "grafana";
restartUnits = [ "grafana.service" ];
};
```
With that, Grafana, Prometheus, Loki and Promtail are setup! You can access `Grafana` at
`grafana.example.com` with user `admin` and password ``.
I recommend adding a STMP server configuration so you receive alerts by email:
```nix
shb.monitoring.smtp = {
from_address = "grafana@$example.com";
from_name = "Grafana";
host = "smtp.mailgun.org";
port = 587;
username = "postmaster@mg.example.com";
passwordFile = config.sops.secrets."monitoring/smtp".path;
};
sops.secrets."monitoring/secret_key" = {
sopsFile = ./secrets.yaml;
mode = "0400";
owner = "grafana";
group = "grafana";
restartUnits = [ "grafana.service" ];
};
```
Since all logs are now stored in Loki, you can probably reduce the systemd journal retention
time with:
```nix
# See https://www.freedesktop.org/software/systemd/man/journald.conf.html#SystemMaxUse=
services.journald.extraConfig = ''
SystemMaxUse=2G
SystemKeepFree=4G
SystemMaxFileSize=100M
MaxFileSec=day
'';
```
## Provisioning
Self Host Blocks will create automatically the following resources: