Skip to content

Commit

Permalink
written an instruct to create bucket-config.yml file on Readme.
Browse files Browse the repository at this point in the history
Signed-off-by: Kushal Shukla <[email protected]>
  • Loading branch information
kushalShukla-web committed Dec 25, 2024
1 parent 6d462bc commit 78397b0
Show file tree
Hide file tree
Showing 4 changed files with 196 additions and 16 deletions.
68 changes: 66 additions & 2 deletions prombench/docs/eks.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,72 @@ Run Prombench tests in [Elastic Kubernetes Service (EKS)](https://aws.amazon.com
```bash
make node_create
```

3. **Deploy the Kubernetes Objects**:
3. **Setting Up Benchmarking Data**

When setting up a benchmarking environment, it’s often useful to have pre-generated data available. This data can help speed up testing and make benchmarks more realistic by simulating actual workloads.

In this setup, you have two choices:

Here’s how each option works:
- **Option 1: Download data from object storage**

To download data from object storage, create a Kubernetes secret with exact named `bucket-secret` and file name `object-config.yml` with the necessary credentials as per your object storage. This secret enables access to the stored data.
> Note: Make sure this secret applied before `3b_prometheus-test_deployment.yaml`

- **Option 2: Skip downloading data**

If you don’t Want to Download data create an empty secret like this -

```yaml
# Empty Secret to Skip Downloading Data
apiVersion: v1
kind: Secret
metadata:
name: bucket-secret
namespace: prombench-{{ .PR_NUMBER }}
type: Opaque
stringData:
object-config.yml:
```

Regardless of the option chosen, data stored in Prometheus will only be retained based on the configured retention settings (```--storage.tsdb.retention.size```).

> **⚠️ Warning:** The benchmark will change its basis when the retention size limit is reached and older downloaded blocks are deleted. Ensure that you have sufficient retention settings configured to avoid data loss that could affect benchmarking results.

4. **Downloading Directory configuration**

PromBench can download data from a specific directory in object storage based on a configuration file. This configuration file specifies the directory name along with the minimum and maximum timestamps.
> **Note:** Make sure the file is applied before the ```3b_prometheus-test_deployment.yaml``` .

- **Option 1: To Download Data from a Specific Directory**

Create a ConfigMap with the following structure:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: blocksync-config
namespace: prombench-{{ .PR_NUMBER }}
data:
bucket-config.yml: |
path: your-directory-name
minTime: block-starting-time
maxTime: block-ending-time
```
Replace the values of ```directory```,```minTime```, and ```maxTime``` with your desired configuration. Here ```minTime``` , ```maxTime``` are the starting and ending time of TSDB block.
- **Option 2: To Skip Data Download**

If you do not want to download data, create an empty ConfigMap with the same name:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: blocksync-config
namespace: prombench-{{ .PR_NUMBER }}
data:
```

5. **Deploy the Kubernetes Objects**:

```bash
make resource_apply
Expand Down
67 changes: 65 additions & 2 deletions prombench/docs/gke.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,71 @@ Run Prombench tests in [Google Kubernetes Engine (GKE)](https://cloud.google.com
```bash
make node_create
```

3. **Deploy the Kubernetes Objects**:
3. **Setting Up Benchmarking Data**
When setting up a benchmarking environment, it’s often useful to have pre-generated data available.This data can help speed up testing and make benchmarks more realistic by simulating actual workloads.

In this setup, you have two choices:

Here’s how each option works:
- **Option 1: Download data from object storage**

To download data from object storage, create a Kubernetes secret with exact named `bucket-secret` and file name `object-config.yml` with the necessary credentials as per your object storage. This secret enables access to the stored data.
> Note: Make sure this secret applied before `3b_prometheus-test_deployment.yaml`

- **Option 2: Skip downloading data**

If you don’t Want to Download data create an empty secret like this -

```yaml
# Empty Secret to Skip Downloading Data
apiVersion: v1
kind: Secret
metadata:
name: bucket-secret
namespace: prombench-{{ .PR_NUMBER }}
type: Opaque
stringData:
object-config.yml:
```

Regardless of the option chosen, data stored in Prometheus will only be retained based on the configured retention settings (```--storage.tsdb.retention.size```).

> **⚠️ Warning:** The benchmark will change its basis when the retention size limit is reached and older downloaded blocks are deleted. Ensure that you have sufficient retention settings configured to avoid data loss that could affect benchmarking results.

4. **Downloading Directory configuration**

PromBench can download data from a specific directory in object storage based on a configuration file. This configuration file specifies the directory name along with the minimum and maximum timestamps.
> **Note:** Make sure the file is applied before the ```3b_prometheus-test_deployment.yaml``` .

- **Option 1: To Download Data from a Specific Directory**

Create a ConfigMap with the following structure:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: blocksync-config
namespace: prombench-{{ .PR_NUMBER }}
data:
bucket-config.yml: |
path: your-directory-name
minTime: block-starting-time
maxTime: block-ending-time
```
Replace the values of ```directory```,```minTime```, and ```maxTime``` with your desired configuration. Here ```minTime``` , ```maxTime``` are the starting and ending time of TSDB block.
- **Option 2: To Skip Data Download**

If you do not want to download data, create an empty ConfigMap with the same name:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: blocksync-config
namespace: prombench-{{ .PR_NUMBER }}
data:
```

5. **Deploy the Kubernetes Objects**:

```bash
make resource_apply
Expand Down
66 changes: 65 additions & 1 deletion prombench/docs/kind.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,71 @@ If used with the GitHub integration:
export PR_NUMBER=<PR to benchmark against the selected $RELEASE>
```

2. Deploy the Kubernetes objects:
2. Setting Up Benchmarking Data
When setting up a benchmarking environment, it’s often useful to have pre-generated data available.This data can help speed up testing and make benchmarks more realistic by simulating actual workloads.

In this setup, you have two choices:

Here’s how each option works:
- **Option 1: Download data from object storage**

To download data from object storage, create a Kubernetes secret with exact named `bucket-secret` and file name `object-config.yml` with the necessary credentials as per your object storage. This secret enables access to the stored data.
> Note: Make sure this secret applied before `3b_prometheus-test_deployment.yaml`
- **Option 2: Skip downloading data**

If you don’t Want to Download data create an empty secret like this -

```yaml
# Empty Secret to Skip Downloading Data
apiVersion: v1
kind: Secret
metadata:
name: bucket-secret
namespace: prombench-{{ .PR_NUMBER }}
type: Opaque
stringData:
object-config.yml:
```
Regardless of the option chosen, data stored in Prometheus will only be retained based on the configured retention settings (```--storage.tsdb.retention.size```).

> **⚠️ Warning:** The benchmark will change its basis when the retention size limit is reached and older downloaded blocks are deleted. Ensure that you have sufficient retention settings configured to avoid data loss that could affect benchmarking results.

3. Downloading Directory configuration

PromBench can download data from a specific directory in object storage based on a configuration file. This configuration file specifies the directory name along with the minimum and maximum timestamps.
> **Note:** Make sure the file is applied before the ```3b_prometheus-test_deployment.yaml``` .

- **Option 1: To Download Data from a Specific Directory**

Create a ConfigMap with the following structure:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: blocksync-config
namespace: prombench-{{ .PR_NUMBER }}
data:
bucket-config.yml: |
path: your-directory-name
minTime: block-starting-time
maxTime: block-ending-time
```
Replace the values of ```directory```,```minTime```, and ```maxTime``` with your desired configuration. Here ```minTime``` , ```maxTime``` are the starting and ending time of TSDB block.
- **Option 2: To Skip Data Download**

If you do not want to download data, create an empty ConfigMap with the same name:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: blocksync-config
namespace: prombench-{{ .PR_NUMBER }}
data:
```

4. Deploy the Kubernetes objects:
> **_Note:_** If you encounter a `too many files open` error caused by promtail, increase the default value of `/proc/sys/fs/inotify/max_user_instances` from 128 to 512:
> ```bash
> sudo sysctl fs.inotify.max_user_instances=512
Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,3 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: blocksync-config
namespace: prombench-{{ .PR_NUMBER }}
data:
bucket-config.yml: |
path: gendata
minTime: 1729694072541
maxTime: 1729706400000
---
apiVersion: apps/v1
kind: Deployment
metadata:
Expand Down

0 comments on commit 78397b0

Please sign in to comment.