Data Protection for Sharing Data Between High and Low Environments

For most IT environments, protecting data as it moves from a high environment (where software or a product is live and customer-facing) to a low environment (where testing or refinement takes place) is a consistent challenge. When data is on the move, organizations have privacy, compliance and security issues to consider. 

In a high environment, data protection is relatively straightforward, as the main consideration is simply hiding data from users. In a low environment, where app updates, testing and debugging occur, data must be protected but yet be usable. 

The traditional approach to moving data from a high environment to a low environment involved custom development for cloning to produce data and share copies with developers, QA managers and others. However, organizations have often neglected security controls related to data access, data protection and how the data is used. Data privacy regulations like GDPR, PCI, HIPAA and the growing number of state data referendums require—to varying degrees—more stringent data protections.

When organizations suffer a data breach—whether it is the result of criminal activity or accidental exposure—the impact often comes in the form of lost revenue and reputational damage. And with the expansion of data privacy laws, there are regulatory fines to consider. Let’s take a closer look at methods for protecting data that moves from a high to a low environment.

High-to-low environment examples

We work extensively with companies in the financial services, healthcare and retail industries and have seen countless examples of how organizations in these industries move data from high to low environments.

Financial services – Banks are continuously updating their apps to adapt to customer preferences and user experience changes. As a result, they frequently copy data in the high environment and move it to a low environment to test UX features. 

Healthcare – During clinical trials, data analytics is performed using patient data to understand how participants react. Very often, multiple people analyze the data, which exacerbates risk—the more people who see sensitive data, the more likely it is that an accidental breach can occur. 

Retail – Sometimes a low environment exists outside of an organization’s four walls. For example, retailers often collect consumer data and resell it to third parties without considering compliance implications. Recently, a major retailer was fined more than $1 million for failing to process opt-out requests from certain customers.  

Data-Masking and Format-Preserving Encryption for Lower Environments 

When moving data from a high environment to a low environment, it is crucial to identify the best protection method based on how that data is to be used. Static data masking is an option, but it renders the masked data records unusable since they are not unique. However, it is a one-way transformation that is irreversible, in case those data records are not to be exposed at all in the lower environments. It is implemented during the process of writing or inserting data into the data store.

Format-preserving encryption, in contrast, yields unique data values as it writes to the data store; these values look like the original values (an encrypted text field still has readable characters, an integer is still a number). This process maintains referential integrity, so that applications that process the data still work, although the original data values cannot be obtained unless the encryption key is available and the access control requirements are met. 

Dynamic data masking can be used to create different views with data transformed to support users with different access to data during the reading or querying process. This can be implemented to support fine-grained access control without compromising security. We are seeing that this can be achieved with a “no code” approach to enable simplified dynamic data masking.

The operational aspect of adopting these controls using modern protection methods involves the deployment of a proxy layer between production databases and their replicas in lower environments used for development and testing. Data is encrypted “on the fly” as it moves from the higher to the lower environment. Then, masking or encryption can be specified on a column level, depending on how that data is utilized in its transformed state. This is revolutionary from a development perspective because it requires no custom development at all, just a network configuration change.

Furthermore, role-based access control enables different data-masking modes to be applied based on the user’s role, allowing organizations to protect sensitive data while providing responsible access. The same approach can also be used to share data externally with partners and customers while protecting PII, PHI and other sensitive information for security and to meet compliance with regulatory requirements such as GDPR, CCPA, PCI and HIPAA. 

Moving data between high and low environments is necessary for companies to continually update software, improve user experience and achieve myriad business goals. But data protection must be a top priority to ensure goals align with compliance requirements. Learn how Baffle can help protect your data in lower environments, or try it for yourself.