Deploy in Private Cloud

In the quick installation guide, we introduced how to install all the basic service components of AREX by default with a single Docker-Compose command, including the front-end UI, MongoDB database, Redis cache, Storage Service, Schedule Service, API Service, etc., as shown above. This method does not require manual installation and configuration of each service, significantly simplifying the installation process and improving deployment efficiency. However, when deploying to a private cloud Kubernetes environment or a completely customized runtime environment, the Docker-Compose installation method is not applicable.

This document will detail how to compile and configure AREX in a private cloud environment. (If you have other special deployment needs or suggestions, you can also open a new GitHub Issue).

The compilation and deployment of AREX Agent and others are not reiterated here, see the Registering the Application section for details.

Composition of AREX

ID	Component Name	Description	Required Configurations
1	Schedule Service	Provides the functionality of sending playbacks and retrieving all responses for comparison.	Configure the connections to the MongoDB database, Redis cache, AREX-API service, Schedule service, and Storage service.
2	AREX-API Service	Provides a summary of differences and displays detailed information of difference results after response comparison.	Configure the connections to the MongoDB database, Redis cache, Storage service, and AREX UI (front-end page) (The AREX front-end has a login module. In version 0.2.8, the front-end page calls the AREX-API service. The generated email contains a link to redirect to the front-end page, so this parameter is configured for correct redirection).
3	Storage Service	Provides AREX Java Agent the functionality to save recordings and retrieve responses as Mock data.	Configure the connections to the MongoDB database, Redis cache, and AREX-API service.
4	AREX UI	The front-end display module of AREX.	Configure the connections to the AREX-API service, Schedule service, Storage service, and Sandbox service (AREX Node).
5	MongoDB	Database for data storage and configuration management.	Expose MongoDB's port and link for use by various services.
6	Redis	High-speed playback cache.	Expose Redis's port and link for use by various services.

AREX Installation in Kubernetes Environment

The installation and configuration files for AREX in a Kubernetes environment are currently located in the arex-chart directory of the Deployments repository's main branch at https://github.com/arextest/deployments/tree/main/arex-chart. This directory contains the Helm Chart configuration files needed for installing AREX.

Installation

To install AREX in a Kubernetes environment using Helm, use the following command:

helm install --namespace arex arex-helm-name-beta ./arex-chart

• --namespace arex: The namespace you plan to configure.
• arex-helm-name-beta: The name you assign for this Helm release.

Upgrade

You can directly use the Helm command to update an already deployed application. If problems occur with the updated configuration, Helm also offers a rollback feature to revert AREX to a previous version.

helm upgrade <arex-your-installed-name> -n <your-namespace-name> ./arex-chart

Deletion

To delete the AREX installation, use:

helm delete arex-beta-028 -n arex-beta

Install AREX Step By Step

Different users may have various runtime environments and deployment requirements. The default Helm Chart provided by AREX might not meet the needs of all users. In such cases, users need to manually modify the existing configuration files based on the actual situation or rewrite the configuration files to better adapt to their runtime environments.

tip

The configuration files are in the arex-chart directory of the deployment repository, which are used for deploying and managing the AREX Helm Chart. They include:

1. Chart.yaml: the metadata file of a Chart, which includes the name, version, description and other information of the Chart.
2. Values.yaml: the default values file of a Chart, which includes the default values of all configuration options in the Chart. These values can be overridden during Chart installation by specifying command line parameters or files.
3. templates: used to define Kubernetes manifest files that will be deployed to the Kubernetes cluster during Chart installation. These configuration files define the relationships and configuration information between these AREX components.

MongoDB and Redis

note

It is not recommended to use the default MongoDB and Redis services provided by AREX. It is better to use external MongoDB and Redis services for ensuring data security and stability, as well as for database administrators (DBAs) to perform data backup and recovery tasks.

The AREX Helm Chart provides default storage capacity settings of 5GB for MongoDB and 1GB for Redis. The default storage capacity settings of 5GB for MongoDB and 1GB for Redis are intended for demonstration purposes only and may not be sufficient for use in actual production environments.

AREX utilizes well-known database services, MongoDB 5.0 and Redis 6.2.6. Users can refer to the official documentation to install these services. It’s important to keep in mind that some servers may not support MongoDB 5.0, in such cases, users can opt to use version 4.4.21, which has been tested and confirmed functional on CentOS.

Users have the flexibility to either use their own pre-existing MongoDB and Redis services or to use the built-in MongoDB and Redis services that are included with the AREX installer. These database services are fully compatible with AREX and support all of its components without requiring additional configuration.

In values.yaml, you can configure the connection information for MongoDB and Redis. Here's an example:

# mongodb config demo
mongoInfo: "mongodb://arex:iLoveArex@10.5.153.1:27017/arex_storage_db"
# redis config demo
redisInfo: "redis://10.5.153.1:6390/"

Disable either the MongoDB or Redis services individually, or to disable both services altogether

To disable either MongoDB or Redis services during installation, or to disable both services, use the helm install command and set mongodb.enabled=false and/or redis.enabled=false. When disabling these services, you must provide connection information for any external MongoDB or Redis services using the mongoInfo and redisInfo parameters, respectively.

During an update process, you can disable MongoDB and Redis services by using the helm upgrade command with the same mongodb.enabled=false and/or redis.enabled=false parameters.

Disable MongoDB or Redis services during the initial installation process

helm upgrade
--set mongodb.enabled=false
--set redis.enabled=false
--set mongoInfo="mongodb://arex:iLoveArex@yourMongoDBInfo_10.5.153.1:27017/arex_storage_db"
--set redisInfo="redis://yourIP_10.5.153.1:6390/"
arex-beta-028
-n arex-beta
./arex-chart

Disable during upgrading

helm upgrade
--set mongodb.enabled=false
--set redis.enabled=false
--set mongoInfo="mongodb://arex:iLoveArex@yourMongoDBInfo_10.5.153.1:27017/arex_storage_db"
--set redisInfo="redis://yourIP_10.5.153.1:6390/"
arex-beta-028
-n arex-beta
./arex-chart

Where arex-beta-028 is the name of your release and arex-beta is your namespace.

Another way to disable MongoDB or Redis services is to modify the values.yaml file directly before installing the chart, and then use the helm upgrade command to apply those changes.

helm upgrade arex-beta-028 -n arex-beta ./arex-chart

The _helpers.tpl file in the chart contains template functions and variables used throughout the chart. When MongoDB or Redis is disabled, the chart reads the values of mongoInfo or redisInfo to determine the connection information for any external MongoDB or Redis services.

{{- define "arex.mongoConnection" -}}
{{- if .Values.mongodb.enabled -}}
{{- printf "mongodb://arex:iLoveArex@%s.%s.svc.cluster.local:%v/arex_storage_db" (include "arex.mongodb.fullname" .) .Release.Namespace .Values.mongodb.service.servicePort -}} 
{{- else -}}
{{- .Values.mongoInfo -}}
{{- end -}}
{{- end -}}
 
{{- define "arex.redisConnection" -}}
{{- if .Values.redis.enabled -}}
{{- printf "redis://%s.%s.svc.cluster.local:%v/" (include "arex.redis.fullname" .) .Release.Namespace .Values.redis.service.servicePort -}}
{{- else -}}
{{- .Values.redisInfo -}}  
{{- end -}}
{{- end -}}

Here, the mongodb.enabled value is checked. If it is false, the chart reads the values of mongoInfo.host, mongoInfo.port, mongoInfo.username, and mongoInfo.password to determine the connection information for the external MongoDB service. This information is then used to construct the MongoDB URI.

A similar process is carried out for Redis in the _helpers.tpl file.

AREX API Service

Refer to arex-report-deployment.yaml

Configuration

          env:
            - name: JAVA_OPTS
              value: -Darex.mongo.uri={{ include "arex.mongoConnection" .}}
                -Darex.redis.uri={{ include "arex.redisConnection" .}}
                -Darex.storage.service.url=http://{{ template "arex.storage.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.storage.service.servicePort }} 
                -Darex.ui.url=http://{{ template "arex.front.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.front.service.servicePort }}
          ports:
            - containerPort: {{ .Values.report.service.servicePort }}

The AREX API service is dependent on MongoDB, Redis, and Storage services.

External Dependencies	Configuration Parameters	Configuration Demo	Description
Mongodb	arex.mongo.uri	mongodb://arex:iLoveArex@10.5.153.1:27017/arex_storage_db	[username]:[password]@[server address of mongoDB]:[port]/[Database name of AREX]
Redis	arex.redis.uri	redis://10.5.153.1:6390/	[server address of Redis]:[port]
Storage Service	arex.storage.service.url	http://10.5.153.1:8093	The port for the container is 8080, while the default configuration for AREX in Docker-compose uses port 8093.
AREX UI	arex.ui.url	http://10.5.153.1:8080	This address is the URL of the AREX frontend page configured to handle email redirection, not a dependency.

tip

1. If you want to use the built-in MongoDB and Redis provided by Helm, there is no need to modify any configurations. The system will automatically point to the default addresses.

2. However, if you prefer to use your own MongoDB and Redis services, you can set the mongoInfo and redisInfo addresses in the values.yaml file and set redis.enabled and mongodb.enabled to false.

3. In case you have set up load balancing for the storage service (storage), you will need to set arex.storage.service.url to the address of your storage load balancer server.

4. If you are using the default storage configuration, Kubernetes will provide you with load balancing and service pointing functionality, hence no additional configuration will be required.

AREX Storage Service

Refer to arex-storage-deployment.yaml

Configuration

          env:
            - name: JAVA_OPTS
              value: -Darex.storage.mongo.host={{ include "arex.mongoConnection" .}}
                -Darex.storage.cache.url={{ include "arex.redisConnection" .}}
                -Darex.api.service.api=http://{{ template "arex.report.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.report.service.servicePort }}/
          ports:
            - containerPort: {{ .Values.storage.service.servicePort }} 

The Storage service depends on Mongodb，Redis，AREX-API services.

External Dependencies	Configuration Parameters	Configuration Demo	Description
Mongodb	arex.storage.mongo.host	mongodb://arex:iLoveArex@10.5.153.1:27017/arex_storage_db	[username]:[password]@[server address of mongoDB]:[port]/[Database name of AREX]
Redis	arex.storage.cache.url	redis://10.5.153.1:6390/	[server address of Redis]:[port]
AREX-API Service	arex.api.service.api	http://10.5.153.1:8093	The port for the container is 8080, while the default configuration for AREX in Docker-compose uses port 8093.

tip

1. If you opt to install the AREX API using the built-in MongoDB and Redis provided by Helm, there is no need to modify any configurations. The system will automatically point to the default addresses.

2. Conversely, if you prefer to use your own MongoDB and Redis services, you can set the mongoInfo and redisInfo addresses in the values.yaml file and set redis.enabled and mongodb.enabled to false. However, it’s important to note that in a single environment, you can only choose to use either the built-in services or external services, but not both simultaneously.

3. In case you have set up load balancing for the AREX-API service, you will need to set arex.report.service.url to the address of your report load balancer server.

4. If you are using the default configuration for your AREX API service, Kubernetes will provide you with load balancing and service pointing functionality, and you won’t need to make any additional configurations.

AREX Schedule Service

Refer to arex-schedule-deployment.yaml

Configuration

          env:
            - name: JAVA_OPTS
              value: -Dmongo.uri={{ include "arex.mongoConnection" .}}
                -Darex.schedule.cache.redis.host={{ include "arex.redisConnection" .}}
                -Darex.storage.service.api=http://{{ template "arex.storage.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.storage.service.servicePort }} 
                -Darex.api.service.api=http://{{ template "arex.report.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.report.service.servicePort }}
          ports:
            - containerPort: {{ .Values.schedule.service.servicePort }}

The Schedule service depends on four services: Mongodb, Redis, API, and Storage services.

External Dependencies	Parameter Configuration	Configuration Demo	Description
Mongodb	mongo.uri	mongodb://arex:iLoveArex@10.5.153.1:27017/arex_storage_db	[username]:[password]@[server address of mongoDB]:[port]/[Database name of AREX]
Redis	arex.schedule.cache.redis.host	redis://10.5.153.1:6390/	[server address of Redis]:[port]
Storage Service	arex.storage.service.api	http://10.5.153.1:8093	The port for the container is 8080, while the default configuration for AREX in Docker-compose uses port 8093.
AREX-API Service	arex.api.service.api	http://10.5.153.1:8090	The port for the container is 8080, while the default configuration for AREX in Docker-compose uses port 8090.

tip

1. If you want to use the built-in MongoDB and Redis provided by Helm, there is no need to modify any configurations, as the system will automatically point to the default addresses.

2. However, if you prefer to use your own MongoDB and Redis services, you can set the mongoInfo and redisInfo addresses in the Values.yaml file and set redis.enabled and mongodb.enabled to false. It is important to note that in a single environment, you can only choose to use either the built-in services or external services, but not both simultaneously.

3. If you have set up load balancing for the AREX-API service, you will need to set arex.api.service.url to the address of your report load balancer server.

4. Similarly, if you have set up load balancing for the storage service (storage), you will need to set arex.storage.service.url to the address of your storage load balancer server.

5. If both your API service and storage service (storage) use the default configurations, Kubernetes will automatically provide you with load balancing and service pointing functionality, and you won’t need to make any additional configurations.

AREX UI

Refer to arex-front-deployment.yaml

Configuration

# Configuration in Helm Chart
          env:
            - name: SERVICE_REPORT_URL
              value: http://{{ template "arex.report.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.report.service.servicePort }}  
            - name: SERVICE_SCHEDULE_URL
              value: http://{{ template "arex.schedule.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.schedule.service.servicePort }}  
            - name: SERVICE_STORAGE_URL
              value: http://{{ template "arex.storage.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.storage.service.servicePort }}  
 
# Configuration in Docker-Compose
    environment:
      - SERVICE_REPORT_URL=http://arex-api-service:8080
      - SERVICE_SCHEDULE_URL=http://arex-schedule-service:8080
      - SERVICE_STORAGE_URL=http://arex-storage-service:8080

The AREX UI depends on four services, AREX-API, Schedule, and Storage.

External Dependencies	Configuration Parameters	Configuration Demo	Description
Schedule Service	SERVICE_SCHEDULE_URL	http://10.5.153.1:8092	The port for the container is 8080, while the default configuration for AREX in Docker-compose uses port 8092.
Storage Service	SERVICE_STORAGE_URL	http://10.5.153.1:8093	The port for the container is 8080, while the default configuration for AREX in Docker-compose uses port 8093.
AREX-API Service	SERVICE_REPORT_URL	http://10.5.153.1:8090	The port for the container is 8080, while the default configuration for AREX in Docker-compose uses port 8090.

tip

1. When using the built-in Helm installation method to deploy different AREX services (such as MongoDB, Redis, AREX API service, etc.), there is no need to modify any configurations, as the system will automatically point to the default addresses.

2. However, if you use custom environment and load balancing settings, you will need to configure AREX with your own load balancer service addresses and port information. This can be done by setting the appropriate configuration values in the values.yaml file.

Configuration Demo

Here is an example configuration of the AREX Helm Chart demonstrating how to set the URL for each service:

SERVICE_REPORT_URL:    http://arex-beta-028-arex-api.arex-beta.svc.cluster.local:8080                                                                                                                    
SERVICE_SCHEDULE_URL:  http://arex-beta-028-arex-schedule.arex-beta.svc.cluster.local:8080                                                                                                           
SERVICE_STORAGE_URL:   http://arex-beta-028-arex-storage.arex-beta.svc.cluster.local:8080  
UI:   http://arex-beta-028-arex-ui.arex-beta.svc.cluster.local:8080  

tip

arex-beta-028 is the name of the release and arex-beta is the namespace.

PersistentVolume(PV) Configuration in AREX Helm Chart

The AREX Helm Chart includes examples of PersistentVolume (PV) and PersistentVolumeClaim (PVC) configurations, which are set to use local storage (local-storage) mode.

These configurations are used to specify the storage settings for MongoDB and Redis, with the recommended minimum storage space of 20GB and 5GB, respectively (as recommended by AREX).

However, it is crucial to note that these configurations must be modified to match the actual storage requirements of your environment to ensure that the adequate storage space is allocated based on your usage needs.

Configuration

apiVersion: v1
kind: PersistentVolume
metadata:
  name: arex-pv-5-volume
  labels:
    type: local
spec:
  storageClassName: local-storage
  capacity:
    storage: 20Gi
  accessModes:
    - ReadWriteMany
  hostPath:
    path: "/mnt/data"
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: arex-pv-1-volume
  labels:
    type: local
spec:
  storageClassName: local-storage
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteMany
  hostPath:
    path: "/mnt/data"
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: arex-mongodb-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 20Gi
  storageClassName: local-storage
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: arex-redis-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Gi
  storageClassName: local-storage
 

Other modes of PV and PVC are available and can be configured by the user based on their specific needs.

Configure arex-mongodb-deployment.yaml

To deploy MongoDB with a PersistentVolumeClaim (PVC), you only need to modify the mongo-data configuration section. The other two mount sections, which are related to configuration parameters and initialization settings, do not require any modification.

Demo

          volumeMounts:
            - name: mongodb-config
              mountPath: /etc/mongo.conf
            - name: mongo-data
              mountPath: /data/db
            - name: settings
              mountPath: /docker-entrypoint-initdb.d/mongo-init.js
              subPath: mongo-allone-init.js
          resources:
{{ toYaml .Values.mongodb.resources | indent 12 }}
    {{- if .Values.mongodb.nodeSelector }}
      nodeSelector:
{{ toYaml .Values.mongodb.nodeSelector | indent 8 }}
    {{- end }}
    {{- if .Values.mongodb.tolerations }}
      tolerations:
{{ toYaml .Values.mongodb.tolerations | indent 8 }}
    {{- end }}
    {{- if .Values.mongodb.affinity }}
      affinity:
{{ toYaml .Values.mongodb.affinity | indent 8 }}
    {{- end }}
      volumes:
        - name: mongo-data
          persistentVolumeClaim:            
            claimName: arex-mongodb-pvc

Configure arex-redis-deployment.yaml

You only need to modify the configuration of redis-data.

Demo

          ports:
            - containerPort: 6379
          volumeMounts:
            - name: redis-data
              mountPath: /data
          resources:
{{ toYaml .Values.redis.resources | indent 12 }}
    {{- if .Values.redis.nodeSelector }}
      nodeSelector:
{{ toYaml .Values.redis.nodeSelector | indent 8 }}
    {{- end }}
    {{- if .Values.redis.tolerations }}
      tolerations:
{{ toYaml .Values.redis.tolerations | indent 8 }}
    {{- end }}
    {{- if .Values.redis.affinity }}
      affinity:
{{ toYaml .Values.redis.affinity | indent 8 }}
    {{- end }}
      volumes:
        - name: redis-data
          persistentVolumeClaim:      
            claimName: arex-redis-pvc      
{{- end }}