Data Desensitization
When recording the real traffic in the production environment, in the case of relating to customer security data or some commercially sensitive data, we need to desensitize the data for some sensitive information to protect of sensitive private data. AREX’s data desensitization includes two parts: data desensitization when storing data in the database and data desensitization when displaying production data in the user interface (UI).
Desensitization when storing data in the database
The data desensitization during data storage mainly refers to encrypting the Mocker table, ReplayRunDetails table, and ReplayCompareResult table in the AREX database and desensitizing the data during data storage to ensure the security of sensitive information. The specific implementation is through the SPI mechanism, loading external JAR packages, and dynamically loading encryption methods.
To achieve this, the URL link of the external JAR package needs to be provided. Taking the encryption JAR package provided by AREX as an example, this JAR package uses the AES encryption algorithm to achieve data encryption, and its source code can be viewed in the code repository.
System Setting
In System Setting, fill in the URL of the encryption JAR package, such as the JAR package download link in the Maven repository (e.g., https://s01.oss.sonatype.org/service/local/repositories/releases/content/com/arextest/arex-desensitization-core/0.0.1/arex-desensitization-core-0.0.1-jar-with-dependencies.jar). After filling in, storage and schedule services need to be restarted to make the encryption effective. If using the encryption JAR package provided by the system, an AES key file also needs to be configured. The specific configuration method can refer to the instructions in the next section.
Service Configuration
When using the encryption JAR package provided by the system, it is necessary to configure the key file. arex-storage and arex-schedule services need to read the AES algorithm key to ensure that the services can encrypte data correctly.
Place the
AES keyfile aesKey.bin in thearex-extension/arex-storageandarex-extension/arex-scheduledirectories in thedeploymentsfolder. It should be noted that the location and file name of the key file must strictly follow the requirements, otherwise the service may not be able to correctly read the key file.
When generating a key file, the filename and extension are fixed and cannot be changed. The code for generating the key file is as follows:
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import java.io.File;
import java.io.FileOutputStream;
public class AesKeyWriter {
private static final String AES_KEY_FOLDER_PATH = "./extension";
private static final String AES_KEY_FILE_PATH = AES_KEY_FOLDER_PATH + "/aesKey.bin";
private static void writeAesKey(String keyFilePath) {
try {
File extensionFolder = new File(AES_KEY_FOLDER_PATH);
extensionFolder.mkdirs();
// AES key generation
KeyGenerator keyGenerator = KeyGenerator.getInstance("AES");
// The key length can be adjusted as needed. aes only supports 128-bit, 192-bit or 256-bit keys
keyGenerator.init(256);
SecretKey secretKey = keyGenerator.generateKey();
byte[] keyBytes = secretKey.getEncoded();
// write key to file
FileOutputStream fos = new FileOutputStream(keyFilePath);
fos.write(keyBytes);
fos.close();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
public static void main(String[] args) {
writeAesKey(AES_KEY_FILE_PATH);
}
}Modify the
docker-compose.ymlfile to map thearex-extension/arex-storagedirectory to the extension directory of thearex-storageservice container, and map thearex-extension/arex-scheduledirectory to the extension directory of thearex-scheduleservice. In this way, the services can correctly read the key file and perform encryption operations.
Using custom encrypted JAR packages
If you need to implement your own encryption algorithm, you need to implement the DataDesensitization interface method in the extended interface jar package, and specify the specific implementation method signature in the /META-INF/services folder. You can refer to the JAR implementation provided by AREX here. The coordinates of the extended interface JAR package are as follows:
<dependency>
<groupId>com.arextest</groupId>
<artifactId>arex-extension</artifactId>
<version>0.0.1</version>
</dependency>
- Encryption of JAR files needs to be reversible, which is crucial to ensure that recorded traffic can be used for playback. If encryption is irreversible, the recorded traffic cannot be decrypted correctly during playback, leading to playback failure.
- When using Maven to package, it is important to include the dependent JAR files in the package, so that the classes and resource files can be loaded correctly at runtime. The Maven packaging configuration is as follows:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>package</phase> <!-- bind to the packaging phase -->
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
</build>
Displaying Production Data
If the application has enabled application authentication, it also enables front-end desensitization of recorded data. Only owners are be able to view the user's recorded data.
The recorded details page with permissions is displayed:
The recorded details page without permissions is displayed:
