From 898c9bdc6572c6d7c7404cf5948d7989e316aa48 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev <slavi@devture.com> Date: Wed, 18 Oct 2023 08:19:28 +0300 Subject: [PATCH] Upgrade Postgres (v16.0-5 -> v16.0-6) - adds auto-tuning --- CHANGELOG.md | 13 +++++++- docs/services/postgres.md | 64 +++++---------------------------------- requirements.yml | 2 +- 3 files changed, 21 insertions(+), 58 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d37ebd1..05b29c9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,9 +1,20 @@ +# 2023-10-18 + +## Postgres parameters are automatically tuned now + +The playbook has provided some hints about [Tuning PostgreSQL](docs/maintenance-postgres.md#tuning-postgresql) for quite a while now. + +From now on, the [Postgres Ansible role](https://github.com/devture/com.devture.ansible.role.postgres) automatically tunes your Postgres configuration with the same [calculation logic](https://github.com/le0pard/pgtune/blob/master/src/features/configuration/configurationSlice.js) that powers https://pgtune.leopard.in.ua/. + +Our [Tuning PostgreSQL](docs/maintenance-postgres.md#tuning-postgresql) documentation page has details about how you can turn auto-tuning off or adjust the automatically-determined Postgres configuration parameters manually. + + # 2023-04-23 ## (Backward Compatibility Break) Authentik container variables renamed For the authentik role there wehre initially two containers: `authentic_worker_container` and `authentic_server_container`. To simnplifiy the setup this was reduced to one container. -As the role is pretty young and to avoid confusion because of legacy and reverted design decisions all variables containing `authentik_server_container` will now start with authentik_container. This means you will have to renemae these variables in your `vars.yml` if you already use authentik. If you use a standard setup this only includes +As the role is pretty young and to avoid confusion because of legacy and reverted design decisions all variables containing `authentik_server_container` will now start with authentik_container. This means you will have to renemae these variables in your `vars.yml` if you already use authentik. If you use a standard setup this only includes * `authentic_server_container_additional_networks_custom` -> `authentik_container_additional_networks_custom` diff --git a/docs/services/postgres.md b/docs/services/postgres.md index d0eb787..d36a5fd 100644 --- a/docs/services/postgres.md +++ b/docs/services/postgres.md @@ -160,68 +160,20 @@ Example: `--extra-vars="postgres_dump_name=mash-postgres-dump.sql"` **All databases, roles, etc. on the Postgres server are migrated**. -### Tuning PostgreSQL +## Tuning PostgreSQL -PostgreSQL can be tuned to make it run faster. This is done by passing extra arguments to Postgres with the `devture_postgres_process_extra_arguments` variable. You should use a website like https://pgtune.leopard.in.ua/ or information from https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server to determine what Postgres settings you should change. +PostgreSQL can be [tuned](https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server) to make it run faster. This is done by passing extra arguments to the Postgres process. -**Note**: the configuration generator at https://pgtune.leopard.in.ua/ adds spaces around the `=` sign, which is invalid. You'll need to remove it manually (`max_connections = 300` -> `max_connections=300`) +The [Postgres Ansible role](https://github.com/devture/com.devture.ansible.role.postgres) **already does some tuning by default**, which matches the [tuning logic](https://github.com/le0pard/pgtune/blob/master/src/features/configuration/configurationSlice.js) done by websites like https://pgtune.leopard.in.ua/. +You can manually influence some of the tuning variables . These parameters (variables) are injected via the `devture_postgres_postgres_process_extra_arguments_auto` variable. -#### Here are some examples: +Most users should be fine with the automatically-done tuning. However, you may wish to: -These are not recommended values and they may not work well for you. This is just to give you an idea of some of the options that can be set. If you are an experienced PostgreSQL admin feel free to update this documentation with better examples. +- **adjust the automatically-deterimned tuning parameters manually**: change the values for the tuning variables defined in the Postgres role's [default configuration file](https://github.com/devture/com.devture.ansible.role.postgres/blob/main/defaults/main.yml) (see `devture_postgres_max_connections`, `devture_postgres_data_storage` etc). These variables are ultimately passed to Postgres via a `devture_postgres_postgres_process_extra_arguments_auto` variable -Here is an example config for a small 2 core server with 4GB of RAM and SSD storage: -``` -devture_postgres_process_extra_arguments: [ - "-c shared_buffers=128MB", - "-c effective_cache_size=2304MB", - "-c effective_io_concurrency=100", - "-c random_page_cost=2.0", - "-c min_wal_size=500MB", -] -``` +- **turn automatically-performed tuning off**: override it like this: `devture_postgres_postgres_process_extra_arguments_auto: []` -Here is an example config for a 4 core server with 8GB of RAM on a Virtual Private Server (VPS); the paramters have been configured using https://pgtune.leopard.in.ua with the following setup: PostgreSQL version 12, OS Type: Linux, DB Type: Mixed type of application, Data Storage: SSD storage: -``` -devture_postgres_process_extra_arguments: [ - "-c max_connections=100", - "-c shared_buffers=2GB", - "-c effective_cache_size=6GB", - "-c maintenance_work_mem=512MB", - "-c checkpoint_completion_target=0.9", - "-c wal_buffers=16MB", - "-c default_statistics_target=100", - "-c random_page_cost=1.1", - "-c effective_io_concurrency=200", - "-c work_mem=5242kB", - "-c min_wal_size=1GB", - "-c max_wal_size=4GB", - "-c max_worker_processes=4", - "-c max_parallel_workers_per_gather=2", - "-c max_parallel_workers=4", - "-c max_parallel_maintenance_workers=2", -] -``` - -Here is an example config for a large 6 core server with 24GB of RAM: -``` -devture_postgres_process_extra_arguments: [ - "-c max_connections=40", - "-c shared_buffers=1536MB", - "-c checkpoint_completion_target=0.7", - "-c wal_buffers=16MB", - "-c default_statistics_target=100", - "-c random_page_cost=1.1", - "-c effective_io_concurrency=100", - "-c work_mem=2621kB", - "-c min_wal_size=1GB", - "-c max_wal_size=4GB", - "-c max_worker_processes=6", - "-c max_parallel_workers_per_gather=3", - "-c max_parallel_workers=6", - "-c max_parallel_maintenance_workers=3", -] -``` +- **add additional tuning parameters**: define your additional Postgres configuration parameters in `devture_postgres_postgres_process_extra_arguments_custom`. See `devture_postgres_postgres_process_extra_arguments_auto` defined in the Postgres role's [default configuration file](https://github.com/devture/com.devture.ansible.role.postgres/blob/main/defaults/main.yml) for inspiration ## Recommended other services diff --git a/requirements.yml b/requirements.yml index 5fd5ffb..96def9a 100644 --- a/requirements.yml +++ b/requirements.yml @@ -37,7 +37,7 @@ - src: git+https://github.com/devture/com.devture.ansible.role.playbook_state_preserver.git version: ff2fd42e1c1a9e28e3312bbd725395f9c2fc7f16 - src: git+https://github.com/devture/com.devture.ansible.role.postgres.git - version: v16.0-5 + version: v16.0-6 - src: git+https://github.com/devture/com.devture.ansible.role.postgres_backup.git version: 8e9ec48a09284c84704d7a2dce17da35f181574d - src: git+https://github.com/devture/com.devture.ansible.role.systemd_docker_base.git