Secure Everything Everywhere All at Once

Nikki Rouda
 | 
May 3, 2023

Security requires a comprehensive approach to be successful, and Replicated helps security-conscious vendors and enterprises meet their goals. We can think of 1/ security strategy for the enterprise who wants control of their data and the apps they use, 2/ security approaches used by the vendor building the app, and 3/ security by design that’s embedded within Replicated to facilitate both. This blog will explore the many layers of security that are necessary to protect apps and data in customer-controlled environments – everything, everywhere, all at once.

Some of the most common security reasons vendors and their enterprise customers like Replicated include: 

  1. Enabling enterprise private instances of apps

At our most basic level, a key value prop of Replicated is how our platform enables commercial software vendors to distribute a private, enterprise-ready instance of their application into their customers’ self-managed environments. By doing so, software vendors can give their customers full control of all the data that the application processes. Additionally, by controlling the environment, end-customers can monitor and block inbound and outbound traffic to improve compliance with data controls. This helps reduce the attack surface area and number of places where sensitive data is held. Bring all the apps to the data in one secure place, rather than distribute the data to many different SaaS apps each with their own strengths and vulnerabilities!

  1. Performing automatic, online updates and patches

One of the most frustrating scenarios for a vendor is when their customers have data exposed through vulnerabilities that already have patches available in the latest releases of their apps. If only the customers would apply the updates in a timely way! With automatic updates, this problem can be addressed and same day patching of all instances for all customers becomes much more achievable. If the customer is running the distributed software with “online updates” then, only specific metadata (the license ID & version information) according to criteria controlled by the customer is sent to Replicated to check if updates are available. At the very least, having reporting into which customers are still running vulnerable versions enables the vendors to send urgent reminders to patch.

  1. Air gap protection for apps and data

For the highest security, some organizations choose to implement an air gap approach - either with no physical external Internet connection at all, or more commonly an intermediate zone with virtual and/or operational air gaps. The idea is to completely isolate the apps and data from attackers, external or internal. If the customer is running the distributed software in “air gap mode” then no data leaves the system (or is even attempted) without direct action from the customer. Of course, this greatly complicates the ability to distribute an application, update it, or extract support log information. Replicated provides mechanisms to facilitate deployment into air gapped environments, and all data can be redacted before sharing. 

Layered Security for Replicated, our Vendors, and their Customers

These three pieces above don’t tell the whole story. Let’s now look at a more complete layered security model for distributing commercial software into customer environments. Here’s a diagram showing many of the components and roughly where they can be implemented. Some security controls should of course be implemented in all three locations: in the vendor’s dev environment and app, in the customer-controlled environment, and within Replicated’s own environment and platform offerings.

A diagram showing a layered security model with various approaches for vendors, enterprises, and Replicated
A layered security model

Physical Security Approaches

Facilities and location - Both vendors and enterprises should evaluate the physical security details of where they run their operations.

Replicated service providers' physical infrastructure is hosted and managed within Amazon Web Services (AWS) secure data centers and utilizes AWS technology. AWS continually manages risk and undergoes recurring assessments to ensure compliance according to the industry’s standards. AWS data center operations have been accredited under: 

● ISO 27001 

● SOC 1 and SOC 2/SSAE 16/ISAE 3402 (Previously SAS 70 Type II) 

● PCI Level 1 

● FISMA Moderate 

● Sarbanes-Oxley (SOX)

Replicated’s service providers’ data centers are all located in the United States, but of course data sovereignty, latency or other considerations could suggest other locations for vendors and enterprises.

​​On-site Security - For proper on-site security, physical access is strictly controlled both at the perimeter and at building ingress points by professional security staff utilizing video surveillance, intrusion detection systems, and other electronic means. Authorized staff must pass two-factor authentication no fewer than three times to access data center floors. All visitors and contractors are required to present identification and are signed in and continually escorted by authorized staff.

Replicated utilizes ISO 27001 and FISMA certified data centers managed by AWS. AWS data centers are housed in nondescript facilities, and critical facilities have extensive setback and military-grade perimeter control berms, as well as other natural boundary protection.

Network Security Approaches

Firewall and Web Application Firewall Protection - Access to sensitive systems should only be granted by a ZeroTrust implementation that analyzes the connection and the connecting device, with decisions made based on the characteristics of the request. Firewalls are utilized to restrict access to systems from external networks and between systems internally. By default, all access is denied and only explicitly allowed ports and protocols are allowed based on business need. Each system is assigned to a firewall security group based on the system’s function. Security groups restrict access to the ports and protocols required for a system’s specific function in order to mitigate risk. Host-based firewalls also provide the ability to further limit inbound and outbound connections as needed.

All firewall infrastructure (hardware and software) and management for Replicated is provided by our service providers: AWS and Cloudflare. Replicated also leverages a suite of Cloudflare web application firewall products to provide virtual web application firewall (WAF) that automatically blocks suspicious traffic and bots, and enables rate limiting. 

Vulnerability Scanning - Managed firewalls prevent IP, MAC, and ARP spoofing on the network and between virtual hosts helps to ensure spoofing is not possible. Packet sniffing is prevented by infrastructure including the hypervisor which will not deliver traffic to any interface other than that which it is addressed. Port scanning is prohibited and every reported instance is investigated by our infrastructure provider. When port scans are detected, they are stopped and access is blocked. 

Replicated’s service provider utilizes application isolation, operating system restrictions, and encrypted connections to further ensure risk is mitigated at multiple levels. 

Penetration Testing and Vulnerability Assessments - Third-party security testing of our service providers is performed by independent and reputable security consulting firms at least annually. Findings from each assessment are reviewed with the assessors, risk ranked, assigned to the responsible team for remediation, and then reviewed again. 

Security Incident Event and Response - In the event of a security incident, engineers gather extensive logs from critical host systems and analyze them to respond to the incident in the most appropriate way possible. Gathering and analyzing log information is critical for troubleshooting and investigating issues. 

Our service provider allows us to analyze four main log types: system, application, API, and audit logs from user accounts. Replicated can also facilitate the collection of the application version, Kubernetes cluster config, environment info, and application logs in a (optionally redacted) support bundle for vendors who need a deeper understanding of where and how their apps are being run.

DDoS Mitigation - A service provider's infrastructure should provide distributed denial-of-service (DDoS) mitigation techniques, including TCP Syn cookies and connection rate limiting, in addition to maintaining multiple backbone connections and internal bandwidth capacity that exceeds the Internet carrier supplied bandwidth. 

We work closely with our providers to quickly respond to events and enable advanced DDoS mitigation controls when needed.

Logical Access - Access to the Replicated production network is restricted by an explicit need-to-know basis. It utilizes least privilege, is frequently audited, and is closely controlled by our Engineering team. Employees accessing the Replicated production network are required to use multiple factors of authentication.

Encryption of Data in Transit and at Rest

Encryption - Strong encryption is a basic building block of any security strategy, and should be pervasively implemented for both data in transit (in motion) and stored data (at rest.) 

Communications between customers, vendors, and Replicated servers are encrypted according to industry best practices (HTTPS). Replicated supports encryption of sensitive customer data at rest.

Availability & Continuity 

Uptime - Uptime matters for enterprise applications because it directly impacts the availability and reliability of critical business systems and can result in financial losses, decreased productivity, and damage to reputation if service is disrupted.

Replicated availability has been 100% for the previous quarter and is continuously monitored. The availability reports are available at https://status.replicated.com 

Redundancy - Redundancy is also important for apps because it helps ensure uninterrupted service and prevent data loss in the event of hardware or software failures or other disruptions. 

Replicated leverages a cloud-native architecture, including clustering and network redundancies, to eliminate single point of failure. For additional redundancy, offsite data backup is available for qualifying accounts to copy the customer's data to a separate cloud provider to mitigate against data loss from AWS. 

Disaster Recovery - A good DR plan allows businesses to quickly resume operations following a catastrophic event or other unforeseen circumstances, minimizing downtime and mitigating potential financial and reputational losses.

Our service provider’s platform automatically restores customer applications and databases in the case of an outage. The provider’s platform is designed to dynamically deploy applications within its cloud, monitor for failures, and recover failed platform components including customer applications and databases.

Secure Development Life Cycle (SDLC) 

Framework Security Controls - Frame security controls help prevent unauthorized access, data breaches, and other cyber attacks, ensuring the confidentiality, integrity, and availability of sensitive information and critical systems. 

Replicated utilizes frameworks for security controls to limit exposure to OWASP Top 10 security flaws. These include inherent controls that reduce our exposure to Cross Site Scripting (XSS), Cross Site Request Forgery (CSRF), and SQL Injection (SQLi), among others. 

Quality Assurance (QA) - QA in development improves the security of enterprise applications because it helps identify and address vulnerabilities and weaknesses in the software development process, reducing the likelihood of security flaws and breaches in production.

In addition to automated testing, Replicated’s QA department reviews and tests our code base. Dedicated application engineers on staff identify, test, and triage security vulnerabilities in code. 

Separate Environments - Testing and staging environments should be separated from the production environment. 

No actual customer data or vendor data is ever used by Replicated in the development, staging, or test environments.

Application Vulnerabilities 

Static Code Analysis - Code analysis prior to shipping releases can detect potential security vulnerabilities and flaws in the source code before the software is deployed, reducing the risk of security incidents and protecting sensitive data. 

Replicated source code repositories are continuously scanned for security issues via our integrated static analysis tooling. 

Dependency Analysis - It’s important to identify and mitigate security risks associated with third-party libraries and components, reducing the potential for vulnerabilities and attacks.

Replicated platform application dependencies are continuously scanned for CVE information and remediated through automated pull requests when fixes are released. 

Continuous Updates to 3rd-party Components - With rapid changes in complex stacks of proprietary components and 3rd-party projects it’s challenging to stay up to date on the latest versions.

Replicated continuously scans upstream 3rd-party components for updates and automatically generates pull requests when new versions are available to install. These update requests are reviewed daily for integration. 

Vulnerability Reporting Program - The disclosure of relevant CVEs by Replicated to our customers and impacted parties is described in our Vulnerability Reporting Program white paper that is available to customers and prospects upon request.

Secure Disclosure - Secure disclosure allows for responsible reporting and patching of vulnerabilities before they can be exploited by attackers, enhancing the overall security posture of the application.

Replicated incentivizes secure disclosure through a private bug bounty program please contact security@replicated.com for inclusion in the program. 

Secure Credential Storage - Protecting credentials and secrets prevents unauthorized access to sensitive information and protects against credential-based attacks, ensuring the confidentiality and integrity of user data.

Replicated follows secure credential storage best practices by never storing passwords, but instead storing a 1-way hash of the salted password. 

API Security & Authentication - API security ensures that only authorized parties have access to protected resources and prevents unauthorized access and manipulation of data, safeguarding the integrity and availability of the API.

Replicated API is TLS-only and you must be a verified user to make API requests. You can authorize against the API using an API token that is controlled on the teams and tokens page. APIs are rate limited to prevent brute force attacks.

Access Privileges & Roles - Access to view and change account configuration should be  governed by access rights, and can be configured to define access privileges. 

Replicated has various permission levels for organization (admin & read-only) and a fully customizable RBAC system for Enterprise users. 

Account Audit Logs - Audit logs provide a record of all account activity, allowing for detection and investigation of suspicious or unauthorized behavior and aiding in forensic analysis in the event of a security incident.

Replicated accounts include a full audit log of the activity in the account. The audit logging system utilizes a Merkle tree design, which ensures data integrity. This data is available via the UI, and can also be accessed from the API for automated collection and centralization of event data. 

Authentication Options - Vendors may choose different authentication approaches for their applications to balance security and convenience. 

Replicated supports password based sign-in with two-factor authentication (2FA) and admin controlled password complexity options. Additionally, Replicated Enterprise Plan customers can leverage & enforce our SAML integration for user management. 

Transmission Security - Transmission security helps prevent interception, manipulation, and theft of sensitive data during transmission, ensuring the confidentiality and integrity of information sent between systems.

All communications with Replicated service provider servers are encrypted using industry standard HTTPS. This ensures that all traffic between you and Replicated is secure during transit.

Service Organization Control (SOC 2) 

SOC 2 Type 1 & 2 - SOC 2 provides a framework for evaluating and improving the effectiveness of their internal controls related to security, availability, processing integrity, confidentiality, and privacy of customer data.

Replicated has completed validation of our SOC 2 Type 1 and Type 2 compliance for security and confidentiality trust principles.

Obviously securing everything is no easy task, as it is a joint activity between Replicated, vendors, and their customers. This blog should provide you with a basic understanding of the various approaches and controls which Replicated has implemented.

For a downloadable look into Replicated’s security posture, including how we address security awareness, employee vetting, and more, please read our published Security White Paper.

To better understand the many ways in which Replicated helps vendors securely deliver software, please check out our Secure Everything use case page.