Many companies (such as fintech companies) process user-sensitive data that cannot be permanently stored due to legal restrictions. According to regulations, the storage time of these data cannot exceed the preset period and it is best to delete it after being used for service purposes. There are many possible solutions to this problem. In this article, I want to show a simplified example of an application that uses Spring and Redis to process sensitive data.
Redis is a high-performance NoSQL database. Usually, it is used as a memory caching solution because it is very fast. However, in this example, we will use it as the primary data store. It perfectly fits the needs of our question and has great integration with Spring Data.
We will create an application that manages user full name and card details (as an example of sensitive data). Card details are passed to the application in the form of an encrypted string via a POST request. The data will be stored in the database for only five minutes. After reading the data through the GET request, the data will be automatically deleted.
The application is designed as an internal microservice and does not provide public access. User data can be passed from user-oriented services. Other internal microservices can then request card details to ensure that sensitive data remains secure and cannot be accessed from external services.
Initialize Spring Boot Project
Let's start creating projects using Spring Initializr. We need Spring Web, Spring Data Redis, and Lombok. I also added Spring Boot Actuator because it will certainly be useful in real microservices.
After initializing the service, we should add other dependencies. To be able to automatically delete data after reading it, we will use AspectJ. I've also added some other dependencies that are helpful to the service to make it look closer to the real one.
The finalThe file looks like this:
plugins { id 'java' id '' version '3.3.3' id '-management' version '1.1.6' id "" version "8.10.2" } java { toolchain { languageVersion = (22) } } repositories { mavenCentral() } ext { springBootVersion = '3.3.3' springCloudVersion = '2023.0.3' dependencyManagementVersion = '1.1.6' aopVersion = "1.9.19" hibernateValidatorVersion = '8.0.' testcontainersVersion = '1.20.2' jacksonVersion = '2.18.0' javaxValidationVersion = '3.1.0' } dependencyManagement { imports { mavenBom ":spring-boot-dependencies:${springBootVersion}" mavenBom ":spring-cloud-dependencies:${springCloudVersion}" } } dependencies { implementation ':spring-boot-starter-data-redis' implementation ':spring-boot-starter-web' implementation ':spring-boot-starter-actuator' implementation ":aspectjweaver:${aopVersion}" implementation ":jackson-core:${jacksonVersion}" implementation ":jackson-databind:${jacksonVersion}" implementation ":jackson-annotations:${jacksonVersion}" implementation ":-api:${javaxValidationVersion}" implementation ":hibernate-validator:${hibernateValidatorVersion}" testImplementation(':spring-boot-starter-test') { exclude group: '' } testImplementation ":testcontainers:${testcontainersVersion}" testImplementation ':junit-jupiter' } ('test') { useJUnitPlatform() }
We need to set up a connection to Redis.The Spring Data Redis properties in are as follows:
spring: data: redis: host: localhost port: 6379
Domain Model
CardInfo
It is the data object we will process. To make it more realistic, we have the card details delivered to the service as encrypted data. We need to decrypt, verify, and then store the incoming data. The domain model will have three levels:
- DTO: Request level, used for controller
- Model: Service level, used for business logic
- Entity: Persistence level, used in warehouses
The conversion between DTO and Model isCardInfoConverter
Completed in the middle. The conversion between Model and Entity isCardInfoEntityMapper
Completed in the middle. We use Lombok for easy development.
DTO
@Builder @Getter @ToString(exclude = "cardDetails") @NoArgsConstructor @AllArgsConstructor @JsonIgnoreProperties(ignoreUnknown = true) public class CardInfoRequestDto { @NotBlank private String id; @Valid private UserNameDto fullName; @NotNull private String cardDetails; }
Among them UserNameDto
@Builder @Getter @ToString @NoArgsConstructor @AllArgsConstructor @JsonIgnoreProperties(ignoreUnknown = true) public class UserNameDto { @NotBlank private String firstName; @NotBlank private String lastName; }
The card details here represent an encrypted string, andfullName
is passed as a separate object. NoticecardDetails
How fields are fromtoString()
Excluded from the method. Since the data is sensitive, it should not be recorded unexpectedly.
Model
@Data @Builder public class CardInfo { @NotBlank private String id; @Valid private UserName userName; @Valid private CardDetails cardDetails; }
@Data @Builder public class UserName { private String firstName; private String lastName; }
CardInfo
andCardInfoRequestDto
Same, justcardDetails
Have been converted (inCardInfoEntityMapper
done in).CardDetails
Now it is a decrypted object, which has two sensitive fields: pan (card number) and CVV (safety code):
@Data @Builder @NoArgsConstructor @AllArgsConstructor @ToString(exclude = {"pan", "cvv"}) public class CardDetails { @NotBlank private String pan; private String cvv; }
See again, wetoString()
Sensitive pan and CVV fields are excluded from the method.
Entity
@Getter @Setter @ToString(exclude = "cardDetails") @NoArgsConstructor @AllArgsConstructor @Builder @RedisHash public class CardInfoEntity { @Id private String id; private String cardDetails; private String firstName; private String lastName; }
In order for Redis to create a hash key for an entity, you need to add@RedisHash
Annotations and@Id
Annotation.
Here is how DTO is converted to Model:
public CardInfo toModel(@NonNull CardInfoRequestDto dto) { final UserNameDto userName = (); return () .id(()) .userName(() .firstName(ofNullable(userName).map(UserNameDto::getFirstName).orElse(null)) .lastName(ofNullable(userName).map(UserNameDto::getLastName).orElse(null)) .build()) .cardDetails(getDecryptedCardDetails(())) .build(); } private CardDetails getDecryptedCardDetails(@NonNull String cardDetails) { try { return (cardDetails, ); } catch (IOException e) { throw new IllegalArgumentException("Card details string cannot be transformed to Json object", e); } }
In this example,getDecryptedCardDetails
Methods just map strings toCardDetails
Object. In real applications, the decryption logic will be implemented in this method.
storehouse
Create a repository using Spring Data. On-serviceCardInfo
Retrieve by its ID, so there is no need to define a custom method, the code looks like this:
@Repository public interface CardInfoRepository extends CrudRepository<CardInfoEntity, String> { }
Redis configuration
We need the entity to be stored for only five minutes. To achieve this we need to set up TTL (Survival Time). We can passCardInfoEntity
Introduce a field and add@TimeToLive
Annotations to implement. It can also be done in@RedisHash
Add values to implement:@RedisHash(timeToLive = 5*60)
。
Both methods have some disadvantages. In the first case, we need to introduce a field that is independent of business logic. In the second case, the value is hardcoded. There is another option: implementKeyspaceConfiguration
. In this way, we can useTo set TTL, if necessary, you can also set other Redis properties.
@Configuration @RequiredArgsConstructor @EnableRedisRepositories(enableKeyspaceEvents = .ON_STARTUP) public class RedisConfiguration { private final RedisKeysProperties properties; @Bean public RedisMappingContext keyValueMappingContext() { return new RedisMappingContext( new MappingConfiguration(new IndexConfiguration(), new CustomKeyspaceConfiguration())); } public class CustomKeyspaceConfiguration extends KeyspaceConfiguration { @Override protected Iterable<KeyspaceSettings> initialConfiguration() { return (customKeyspaceSettings(, CacheName.CARD_INFO)); } private <T> KeyspaceSettings customKeyspaceSettings(Class<T> type, String keyspace) { final KeyspaceSettings keyspaceSettings = new KeyspaceSettings(type, keyspace); (().getTimeToLive().toSeconds()); return keyspaceSettings; } } @NoArgsConstructor(access = ) public static class CacheName { public static final String CARD_INFO = "cardInfo"; } }
In order for Redis to delete entities based on TTL, it is necessary to@EnableRedisRepositories
Added in the annotationenableKeyspaceEvents = .ON_STARTUP
. I've introducedCacheName
class to use constants as entity names and reflect that multiple entities can be configured differently if needed.
The value of TTL is fromRedisKeysProperties
Gets from the object.
@Data @Component @ConfigurationProperties("") @Validated public class RedisKeysProperties { @NotNull private KeyParameters cardInfo; @Data @Validated public static class KeyParameters { @NotNull private Duration timeToLive; } }
There is only the cardInfo entity here, but there may be other entities. Apply the TTL property in .yml:
redis: keys: cardInfo: timeToLive: PT5M
Controller
Let's add an API to the service so that data can be stored and accessed over HTTP.
@RestController @RequiredArgsConstructor @RequestMapping( "/api/cards") public class CardController { private final CardService cardService; private final CardInfoConverter cardInfoConverter; @PostMapping @ResponseStatus(CREATED) public void createCard(@Valid @RequestBody CardInfoRequestDto cardInfoRequest) { ((cardInfoRequest)); } @GetMapping("/{id}") public ResponseEntity<CardInfoResponseDto> getCard(@PathVariable("id") String id) { return (((id))); } }
Automatic deletion function based on AOP
We want to delete the entity immediately after it is successfully read through the GET request. This can be achieved through AOP and AspectJ. We need to create a Spring Bean and use it@Aspect
Make annotations.
@Aspect @Component @RequiredArgsConstructor @ConditionalOnExpression("${:false}") public class CardRemoveAspect { private final CardInfoRepository repository; @Pointcut("execution(* (..)) && args(id)") public void cardController(String id) { } @AfterReturning(value = "cardController(id)", argNames = "id") public void deleteCard(String id) { (id); } }
@Pointcut
Defines the entry point for logic applications. In other words, it determines the timing of triggering the execution of the logic.deleteCard
The method defines the specific logic, which passesCardInfoRepository
Delete by IDcardInfo
entity.@AfterReturning
The annotation indicates that the method will bevalue
The method defined in the attribute is executed after successfully returning.
In addition, I also used@ConditionalOnExpression
Annotation to enable or turn off this function according to configuration properties.
test
We will use MockMvc and Testcontainers to write the test case.
public abstract class RedisContainerInitializer { private static final int PORT = 6379; private static final String DOCKER_IMAGE = "redis:6.2.6"; private static final GenericContainer REDIS_CONTAINER = new GenericContainer((DOCKER_IMAGE)) .withExposedPorts(PORT) .withReuse(true); static { REDIS_CONTAINER.start(); } @DynamicPropertySource static void properties(DynamicPropertyRegistry registry) { ("", REDIS_CONTAINER::getHost); ("", () -> REDIS_CONTAINER.getMappedPort(PORT)); } }
pass@DynamicPropertySource
, we can set properties from the Redis Docker container that is launched. These properties are then read by the application to establish a connection to Redis.
Here are the basic tests for POST and GET requests:
public class CardControllerTest extends BaseTest { private static final String CARDS_URL = "/api/cards"; private static final String CARDS_ID_URL = CARDS_URL + "/{id}"; @Autowired private CardInfoRepository repository; @BeforeEach public void setUp() { (); } @Test public void createCard_success() throws Exception { final CardInfoRequestDto request = aCardInfoRequestDto().build(); (post(CARDS_URL) .contentType(APPLICATION_JSON) .content((request))) .andExpect(status().isCreated()) ; assertCardInfoEntitySaved(request); } @Test public void getCard_success() throws Exception { final CardInfoEntity entity = aCardInfoEntityBuilder().build(); prepareCardInfoEntity(entity); (get(CARDS_ID_URL, ())) .andExpect(status().isOk()) .andExpect(jsonPath("$.id", is(()))) .andExpect(jsonPath("$.cardDetails", notNullValue())) .andExpect(jsonPath("$.", is(CVV))) ; } }
Automatic deletion function test via AOP:
@Test @EnabledIf( expression = "${}", loadContext = true ) public void getCard_deletedAfterRead() throws Exception { final CardInfoEntity entity = aCardInfoEntityBuilder().build(); prepareCardInfoEntity(entity); (get(CARDS_ID_URL, ())) .andExpect(status().isOk()); (get(CARDS_ID_URL, ())) .andExpect(status().isNotFound()) ; }
I added this test@EnabledIf
Annotation, because the AOP logic can be closed in the configuration file, and the annotation is used to decide whether to run the test.
The above is the detailed content of the sample code for creating services that process sensitive data using Spring and Redis. For more information about Spring Redis processing sensitive data, please pay attention to my other related articles!