use chapters instead of parts to allow sub-headings
This commit is contained in:
ibizaman
Pierre Penninckx
parent
13b756b09c
commit
5a3e072f34
9 changed files with 118 additions and 127 deletions
|
|
@ -1,7 +0,0 @@
|
|||
# Requests Error Budget Alert {#blocks-monitoring-budget-alerts}
|
||||
|
||||
This alert will fire when the ratio between number of requests getting a 5XX response from a service
|
||||
and the total requests to that service exceeds 1%.
|
||||
|
||||
|
||||
|
||||
|
|
@ -1,64 +0,0 @@
|
|||
# Configuration {#blocks-monitoring-configuration}
|
||||
|
||||
```nix
|
||||
shb.monitoring = {
|
||||
enable = true;
|
||||
subdomain = "grafana";
|
||||
inherit domain;
|
||||
contactPoints = [ "me@example.com" ];
|
||||
adminPasswordFile = config.sops.secrets."monitoring/admin_password".path;
|
||||
secretKeyFile = config.sops.secrets."monitoring/secret_key".path;
|
||||
};
|
||||
|
||||
sops.secrets."monitoring/admin_password" = {
|
||||
sopsFile = ./secrets.yaml;
|
||||
mode = "0400";
|
||||
owner = "grafana";
|
||||
group = "grafana";
|
||||
restartUnits = [ "grafana.service" ];
|
||||
};
|
||||
sops.secrets."monitoring/secret_key" = {
|
||||
sopsFile = ./secrets.yaml;
|
||||
mode = "0400";
|
||||
owner = "grafana";
|
||||
group = "grafana";
|
||||
restartUnits = [ "grafana.service" ];
|
||||
};
|
||||
```
|
||||
|
||||
With that, Grafana, Prometheus, Loki and Promtail are setup! You can access `Grafana` at
|
||||
`grafana.example.com` with user `admin` and password ``.
|
||||
|
||||
I recommend adding a STMP server configuration so you receive alerts by email:
|
||||
|
||||
```nix
|
||||
shb.monitoring.smtp = {
|
||||
from_address = "grafana@$example.com";
|
||||
from_name = "Grafana";
|
||||
host = "smtp.mailgun.org";
|
||||
port = 587;
|
||||
username = "postmaster@mg.example.com";
|
||||
passwordFile = config.sops.secrets."monitoring/smtp".path;
|
||||
};
|
||||
|
||||
sops.secrets."monitoring/secret_key" = {
|
||||
sopsFile = ./secrets.yaml;
|
||||
mode = "0400";
|
||||
owner = "grafana";
|
||||
group = "grafana";
|
||||
restartUnits = [ "grafana.service" ];
|
||||
};
|
||||
```
|
||||
|
||||
Since all logs are now stored in Loki, you can probably reduce the systemd journal retention
|
||||
time with:
|
||||
|
||||
```nix
|
||||
# See https://www.freedesktop.org/software/systemd/man/journald.conf.html#SystemMaxUse=
|
||||
services.journald.extraConfig = ''
|
||||
SystemMaxUse=2G
|
||||
SystemKeepFree=4G
|
||||
SystemMaxFileSize=100M
|
||||
MaxFileSec=day
|
||||
'';
|
||||
```
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
# Errors Dashboard {#blocks-monitoring-error-dashboard}
|
||||
|
||||
This dashboard is meant to be the first stop to understand why a service is misbehaving.
|
||||
|
||||
|
||||
|
||||
|
||||
The yellow and red dashed vertical bars correspond to the [Requests Error Budget
|
||||
Alert](#blocks-monitoring-budget-alerts) firing.
|
||||
|
|
@ -1,6 +0,0 @@
|
|||
# Performance Dashboard {#blocks-monitoring-performance-dashboard}
|
||||
|
||||
This dashboard is meant to be the first stop to understand why a service is performing poorly.
|
||||
|
||||
|
||||
|
||||
|
|
@ -8,10 +8,113 @@ This block sets up the monitoring stack for Self Host Blocks. It is composed of:
|
|||
- Prometheus as the database for metrics.
|
||||
- Loki as the database for logs.
|
||||
|
||||
```{=include=} parts
|
||||
configuration.md
|
||||
provisioning.md
|
||||
dashboard-errors.md
|
||||
dashboard-performance.md
|
||||
alerts-requests-error-budger.md
|
||||
## Configuration {#blocks-monitoring-configuration}
|
||||
|
||||
```nix
|
||||
shb.monitoring = {
|
||||
enable = true;
|
||||
subdomain = "grafana";
|
||||
inherit domain;
|
||||
contactPoints = [ "me@example.com" ];
|
||||
adminPasswordFile = config.sops.secrets."monitoring/admin_password".path;
|
||||
secretKeyFile = config.sops.secrets."monitoring/secret_key".path;
|
||||
};
|
||||
|
||||
sops.secrets."monitoring/admin_password" = {
|
||||
sopsFile = ./secrets.yaml;
|
||||
mode = "0400";
|
||||
owner = "grafana";
|
||||
group = "grafana";
|
||||
restartUnits = [ "grafana.service" ];
|
||||
};
|
||||
sops.secrets."monitoring/secret_key" = {
|
||||
sopsFile = ./secrets.yaml;
|
||||
mode = "0400";
|
||||
owner = "grafana";
|
||||
group = "grafana";
|
||||
restartUnits = [ "grafana.service" ];
|
||||
};
|
||||
```
|
||||
|
||||
With that, Grafana, Prometheus, Loki and Promtail are setup! You can access `Grafana` at
|
||||
`grafana.example.com` with user `admin` and password ``.
|
||||
|
||||
I recommend adding a STMP server configuration so you receive alerts by email:
|
||||
|
||||
```nix
|
||||
shb.monitoring.smtp = {
|
||||
from_address = "grafana@$example.com";
|
||||
from_name = "Grafana";
|
||||
host = "smtp.mailgun.org";
|
||||
port = 587;
|
||||
username = "postmaster@mg.example.com";
|
||||
passwordFile = config.sops.secrets."monitoring/smtp".path;
|
||||
};
|
||||
|
||||
sops.secrets."monitoring/secret_key" = {
|
||||
sopsFile = ./secrets.yaml;
|
||||
mode = "0400";
|
||||
owner = "grafana";
|
||||
group = "grafana";
|
||||
restartUnits = [ "grafana.service" ];
|
||||
};
|
||||
```
|
||||
|
||||
Since all logs are now stored in Loki, you can probably reduce the systemd journal retention
|
||||
time with:
|
||||
|
||||
```nix
|
||||
# See https://www.freedesktop.org/software/systemd/man/journald.conf.html#SystemMaxUse=
|
||||
services.journald.extraConfig = ''
|
||||
SystemMaxUse=2G
|
||||
SystemKeepFree=4G
|
||||
SystemMaxFileSize=100M
|
||||
MaxFileSec=day
|
||||
'';
|
||||
```
|
||||
|
||||
## Provisioning {#blocks-monitoring-provisioning}
|
||||
|
||||
Self Host Blocks will create automatically the following resources:
|
||||
|
||||
- For Grafana:
|
||||
- datasources
|
||||
- dashboards
|
||||
- contact points
|
||||
- notification policies
|
||||
- alerts
|
||||
- For Prometheus, the following exporters and related scrapers:
|
||||
- node
|
||||
- smartctl
|
||||
- nginx
|
||||
- For Loki, the following exporters and related scrapers:
|
||||
- systemd
|
||||
|
||||
Those resources are namespaced as appropriate under the Self Host Blocks namespace:
|
||||
|
||||
[](./assets/folder.png)
|
||||
|
||||
## Errors Dashboard {#blocks-monitoring-error-dashboard}
|
||||
|
||||
This dashboard is meant to be the first stop to understand why a service is misbehaving.
|
||||
|
||||
|
||||
|
||||
|
||||
The yellow and red dashed vertical bars correspond to the [Requests Error Budget
|
||||
Alert](#blocks-monitoring-budget-alerts) firing.
|
||||
|
||||
## Performance Dashboard {#blocks-monitoring-performance-dashboard}
|
||||
|
||||
This dashboard is meant to be the first stop to understand why a service is performing poorly.
|
||||
|
||||
|
||||
|
||||
|
||||
## Requests Error Budget Alert {#blocks-monitoring-budget-alerts}
|
||||
|
||||
This alert will fire when the ratio between number of requests getting a 5XX response from a service
|
||||
and the total requests to that service exceeds 1%.
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,20 +0,0 @@
|
|||
# Provisioning {#blocks-monitoring-provisioning}
|
||||
|
||||
Self Host Blocks will create automatically the following resources:
|
||||
|
||||
- For Grafana:
|
||||
- datasources
|
||||
- dashboards
|
||||
- contact points
|
||||
- notification policies
|
||||
- alerts
|
||||
- For Prometheus, the following exporters and related scrapers:
|
||||
- node
|
||||
- smartctl
|
||||
- nginx
|
||||
- For Loki, the following exporters and related scrapers:
|
||||
- systemd
|
||||
|
||||
Those resources are namespaced as appropriate under the Self Host Blocks namespace:
|
||||
|
||||
[](./assets/folder.png)
|
||||
|
|
@ -96,12 +96,12 @@ in stdenv.mkDerivation {
|
|||
${optionsDocs.optionsJSON}/share/doc/nixos/options.json
|
||||
|
||||
find . -name "*.md" -print0 | \
|
||||
while IFS= read -r -d ''' f; do
|
||||
substituteInPlace "''${f}" \
|
||||
--replace \
|
||||
'@REPO@' \
|
||||
"${lib.debug.traceVal ghRoot}"
|
||||
done
|
||||
while IFS= read -r -d ''' f; do
|
||||
substituteInPlace "''${f}" \
|
||||
--replace \
|
||||
'@REPO@' \
|
||||
"${ghRoot}"
|
||||
done
|
||||
|
||||
nixos-render-docs manual html \
|
||||
--manpage-urls ${manpage-urls} \
|
||||
|
|
|
|||
|
|
@ -7,18 +7,10 @@
|
|||
preface.md
|
||||
```
|
||||
|
||||
```{=include=} parts html:into-file=//blocks-monitoring.html
|
||||
```{=include=} chapters html:into-file=//blocks-monitoring.html
|
||||
blocks/monitoring/default.md
|
||||
```
|
||||
|
||||
```{=include=} appendix html:into-file=//options.html
|
||||
options.md
|
||||
```
|
||||
|
||||
<!-- ```{=include=} appendix html:into-file=//nixos-options.html -->
|
||||
<!-- nixos-options.md -->
|
||||
<!-- ``` -->
|
||||
|
||||
<!-- ```{=include=} appendix html:into-file=//nix-darwin-options.html -->
|
||||
<!-- nix-darwin-options.md -->
|
||||
<!-- ``` -->
|
||||
|
|
|
|||
|
|
@ -1,6 +1,8 @@
|
|||
# Preface {#preface}
|
||||
|
||||
::: {.note}
|
||||
This document is the complete manual for Self Host Blocks, the building blocks for self-hosting with battery included.
|
||||
:::
|
||||
|
||||
Self Host Blocks is hosted on [GitHub](https://github.com/ibizaman/selfhostblocks). If you encounter
|
||||
problems or bugs then please report them on the [issue
|
||||
|
|
|
|||
Loading…
Reference in a new issue