The Business and Technical Challenges:

The task involved developing a new version of the patient data management system using Python while supporting the legacy system on Scala. As part of their expansion into the European market, IND faced the challenge of complying with the stringent requirements of the General Data Protection Regulation (GDPR). This included ensuring the lawful processing of patient data, enabling data subject rights, and implementing robust data protection measures across its platforms. The new platform needed to manage patient data in full alignment with GDPR, emphasizing transparency, data minimization, and accountability.

 

The Solution:

To meet GDPR compliance and address regulatory complexity, our engineering team implemented the following strategies:

1. Data Segmentation by Region:

• Created a Europe-based data warehouse to store and process patient information in compliance with GDPR

• Maintained a U.S.-based data warehouse for regions outside GDPR jurisdiction

2. Dual-Database Structure:

• Established a central database for general patient information accessible across European countries, ensuring compliance with GDPR data transfer restrictions

• Designed country-specific auxiliary databases for detailed patient data, applying local GDPR mandates for pseudonymization and data retention policies

3. GDPR Compliance Measures:

• Ensured data minimization by collecting only necessary information

• Implemented mechanisms for patients to exercise their rights, including the right to access, rectify, erase, and port their data

• Incorporated “privacy by design and by default” principles into every aspect of the system’s architecture

4. Approval by EU Ethics Committees:

• The platform’s privacy and security policy was reviewed and approved by applicable EU Ethics Committees, ensuring adherence to regional regulations before the platform’s deployment.

​​

The Tech Stack, Used in the Project:

• Python

• Scala

• AWS

• PostgreSQL

• dbt​

The Result:

The new platform, built with GDPR compliance at its core, supports operations in the United Kingdom, Spain, Germany, Austria, and the Netherlands. It also includes functionality for seamlessly adding data storage in new European countries without requiring extensive platform modifications. Patients’ personal data is now securely managed and protected, reflecting IND’s commitment to ethical research practices and regulatory adherence.

​

The Data Security:

Security and confidentiality were ensured using AWS tools,

adhering to GDPR requirements for secure data storage, processing, and transfer. Key measures included:

Data Encryption: Ensured data is encrypted both in transit and at rest

Access Control: Applied strict access policies to limit data access based on roles and responsibilities

Audit and Monitoring: Enabled comprehensive logging and monitoring to detect and address potential security breaches proactively.

​These measures align with the GDPR’s accountability and security principles, ensuring IND’s research and data handling processes remain transparent and secure.

The platform’s GDPR-centric design underscores IND’s dedication to maintaining trust and protecting patient privacy while expanding its global research capabilities.