From 18b922fae5b144c71bec86fa49a24909c7bd1c19 Mon Sep 17 00:00:00 2001 From: Kushal Shukla Date: Tue, 19 Nov 2024 15:23:08 +0530 Subject: [PATCH] Added --storage.tsdb.retention.size to retain data for certain size. Signed-off-by: Kushal Shukla --- prombench/docs/eks.md | 30 ++++++++++++++++++- prombench/docs/gke.md | 30 ++++++++++++++++++- prombench/docs/kind.md | 4 +++ .../3b_prometheus-test_deployment.yaml | 6 ++-- 4 files changed, 66 insertions(+), 4 deletions(-) diff --git a/prombench/docs/eks.md b/prombench/docs/eks.md index f529f57ca..fb40d0b8a 100644 --- a/prombench/docs/eks.md +++ b/prombench/docs/eks.md @@ -107,8 +107,36 @@ Run Prombench tests in [Elastic Kubernetes Service (EKS)](https://aws.amazon.com ```bash make node_create ``` +3. When setting up a benchmarking environment, it’s often useful to have pre-generated data available.This data can help speed up testing and make benchmarks more realistic by simulating actual workloads. -3. **Deploy the Kubernetes Objects**: +In this setup, you have two choices: + +Here’s how each option works: +- **Option 1: Download data from object storage** + + To download data from object storage, create a Kubernetes secret with exact named `bucket-secret` and file name ```object-config.yml``` with the necessary credentials as per your object storage. This secret enables access to the stored data. +> Note: Make sure this secret applied before ```3b_prometheus-test_deployment.yaml``` + +- **Option 2: Skip downloading data** + +If you don’t Want to Download data create an empty secret like this - + +```yaml +# Empty Secret to Skip Downloading Data +apiVersion: v1 +kind: Secret +metadata: + name: bucket-secret + namespace: prombench-{{ .PR_NUMBER }} +type: Opaque +stringData: + object-config.yml: + +Regardless of the option chosen, data stored in Prometheus will only be retained based on the configured retention settings (`--storage.tsdb.retention.size`). + +> **⚠️ Warning:** The benchmark will change its basis when the retention size limit is reached and older downloaded blocks are deleted. Ensure that you have sufficient retention settings configured to avoid data loss that could affect benchmarking results. + +4. **Deploy the Kubernetes Objects**: ```bash make resource_apply diff --git a/prombench/docs/gke.md b/prombench/docs/gke.md index dbcb4b292..d410972be 100644 --- a/prombench/docs/gke.md +++ b/prombench/docs/gke.md @@ -96,8 +96,36 @@ Run Prombench tests in [Google Kubernetes Engine (GKE)](https://cloud.google.com ```bash make node_create ``` +3. When setting up a benchmarking environment, it’s often useful to have pre-generated data available.This data can help speed up testing and make benchmarks more realistic by simulating actual workloads. -3. **Deploy the Kubernetes Objects**: +In this setup, you have two choices: + +Here’s how each option works: +- **Option 1: Download data from object storage** + + To download data from object storage, create a Kubernetes secret with exact named `bucket-secret` and file name ```object-config.yml``` with the necessary credentials as per your object storage. This secret enables access to the stored data. +> Note: Make sure this secret applied before ```3b_prometheus-test_deployment.yaml``` + +- **Option 2: Skip downloading data** + +If you don’t Want to Download data create an empty secret like this - + +```yaml +# Empty Secret to Skip Downloading Data +apiVersion: v1 +kind: Secret +metadata: + name: bucket-secret + namespace: prombench-{{ .PR_NUMBER }} +type: Opaque +stringData: + object-config.yml: + +Regardless of the option chosen, data stored in Prometheus will only be retained based on the configured retention settings (`--storage.tsdb.retention.size`). + +> **⚠️ Warning:** The benchmark will change its basis when the retention size limit is reached and older downloaded blocks are deleted. Ensure that you have sufficient retention settings configured to avoid data loss that could affect benchmarking results. + +4. **Deploy the Kubernetes Objects**: ```bash make resource_apply diff --git a/prombench/docs/kind.md b/prombench/docs/kind.md index 7a77bad79..42d267b8b 100644 --- a/prombench/docs/kind.md +++ b/prombench/docs/kind.md @@ -140,6 +140,10 @@ metadata: type: Opaque stringData: object-config.yml: + +Regardless of the option chosen, data stored in Prometheus will only be retained based on the configured retention settings (`--storage.tsdb.retention.size`). + +> **⚠️ Warning:** The benchmark will change its basis when the retention size limit is reached and older downloaded blocks are deleted. Ensure that you have sufficient retention settings configured to avoid data loss that could affect benchmarking results. 3. Deploy the Kubernetes objects: > **_Note:_** If you encounter a `too many files open` error caused by promtail, increase the default value of `/proc/sys/fs/inotify/max_user_instances` from 128 to 512: diff --git a/prombench/manifests/prombench/benchmark/3b_prometheus-test_deployment.yaml b/prombench/manifests/prombench/benchmark/3b_prometheus-test_deployment.yaml index 4994b4b31..7355b6f8f 100644 --- a/prombench/manifests/prombench/benchmark/3b_prometheus-test_deployment.yaml +++ b/prombench/manifests/prombench/benchmark/3b_prometheus-test_deployment.yaml @@ -81,7 +81,8 @@ spec: "--web.external-url=http://{{ .DOMAIN_NAME }}/{{ .PR_NUMBER }}/prometheus-pr", "--storage.tsdb.path=/prometheus", "--config.file=/etc/prometheus/prometheus.yml", - "--log.level=debug" + "--log.level=debug", + "--storage.tsdb.retention.size=5GB" ] volumeMounts: - name: config-volume @@ -199,7 +200,8 @@ spec: "--web.external-url=http://{{ .DOMAIN_NAME }}/{{ .PR_NUMBER }}/prometheus-release", "--storage.tsdb.path=/prometheus", "--config.file=/etc/prometheus/prometheus.yml", - "--log.level=debug" + "--log.level=debug", + "--storage.tsdb.retention.size=5GB" ] volumeMounts: - name: config-volume