Data Security and Privacy

WhyLabs Secure is designed with a strong emphasis on data security and privacy, particularly for organizations handling sensitive information in their AI applications. Several key aspects of its architecture contribute significantly to maintaining the confidentiality and integrity of LLM data.

1. Deployment Within Your VPC

WhyLabs Secure utilizes a hybrid-SaaS model that prioritizes data protection. The core component of WhyLabs Secure, including the Guardrails API, is deployed directly within your own Virtual Private Cloud (VPC). This deployment strategy offers several security benefits:

• Data Isolation: By running within your VPC, WhyLabs Secure ensures that all sensitive data, including prompts and responses from your LLM applications, remains within your controlled environment.
• Network Security: Your existing network security measures, such as firewalls and access controls, automatically extend to the WhyLabs Secure deployment.
• Compliance: This approach helps maintain compliance with data residency and privacy regulations, as sensitive data does not leave your infrastructure.

2. Metrics-Only Telemetry

WhyLabs Secure is designed to provide comprehensive monitoring and analysis without exposing raw data:

• Aggregated Metrics: Instead of sending raw prompts and responses to the WhyLabs platform, only metrics and telemetry data are transmitted.
• Privacy-Preserving Analytics: This approach allows for in-depth analysis and monitoring of your AI applications without compromising the privacy of individual interactions.
• Reduced Data Exposure: By limiting the data sent to the monitoring platform to metrics and telemetry rather than duplicating the LLM data, the risk of exposing sensitive information is significantly minimized.
• Collaboration Without Compromise: Enable cross-functional collaboration on LLM projects while maintaining strict data privacy controls.
• Audit Trails: Maintain detailed logs of system interactions, allowing for transparency and accountability without exposing sensitive information.
• Secure Debugging: Allow technical teams to troubleshoot and optimize LLM performance using aggregated metrics and telemetry, without access to individual prompts or responses.
• Compliance Monitoring: Enable compliance teams to oversee LLM usage and ensure adherence to policies and regulations without direct access to potentially sensitive conversational data.
• Business Intelligence: Provide executives and product managers with high-level insights and KPIs derived from LLM usage, without risking exposure of customer data.

3. Batch Metrics

In addition to real-time metrics, WhyLabs Secure leverages the power of WhyLabs Observability for batch metrics:

• whylogs Integration: WhyLabs Secure uses whylogs, an open-source data logging library, to generate comprehensive statistical profiles of your data.
• Privacy-Preserving Profiles: whylogs creates compact, privacy-preserving summaries of your data, capturing important statistical properties without storing raw data points.
• Efficient Data Transfer: These compact profiles can be efficiently transmitted to the WhyLabs platform, reducing bandwidth usage while providing rich insights.

4. Comprehensive Observability

The metrics and telemetry data collected by WhyLabs Secure feed into the broader WhyLabs observability platform:

• Holistic Monitoring: Combine LLM-specific metrics from WhyLabs Secure with broader AI/ML observability features provided by the WhyLabs platform.
• Advanced Analytics: Leverage WhyLabs' powerful analytics capabilities to gain deeper insights into your AI applications' performance and behavior.
• Unified Dashboard: Access all your AI observability data, including WhyLabs Secure metrics, through a single, intuitive interface.

The combination of on-premises deployment, metrics-only telemetry, batch-telemetry via whylogs, and integration with WhyLabs observability creates a robust and comprehensive security and monitoring model. It allows organizations to benefit from WhyLabs Secure's advanced monitoring and guardrail capabilities while maintaining strict control over their sensitive data. This architecture is particularly valuable for industries with stringent data protection requirements or for applications handling personal or confidential information, all while providing deep insights into AI application performance and behavior.