Organizations should take steps to protect the integrity of their log data by making logs read-only, restricting the ability to disable logging, and implementing separation of duties. Centralized log management solutions and SIEM tools can help monitor logs in real-time to detect and alert on any unauthorized access or modification attempts. Proper log protection is crucial for security investigations, auditing, and compliance.

Where did this come from?

This control comes from the CSA Cloud Controls Matrix v4.0.10 - 2023-09-26, which can be downloaded at https://cloudsecurityalliance.org/artifacts/cloud-controls-matrix-v4. The Cloud Security Alliance developed this framework to provide a set of security controls and best practices for securing cloud computing environments. For more information on logging and monitoring in AWS specifically, check out this documentation: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html

Who should care?

• Security analysts responsible for detecting and investigating security incidents
• Compliance officers needing to meet regulatory logging and auditing requirements
• Cloud administrators managing access controls and logging configurations
• Developers building applications that generate security-relevant log data

What is the risk?

If logs are not properly safeguarded, malicious insiders or external attackers who gain unauthorized access could potentially:

• Modify or delete log entries to cover their tracks and evade detection
• Disable logging entirely to operate undetected
• Leak sensitive data contained in unencrypted logs
• Tamper with logs to create false audit trails

While having immutable, always-on logging can't prevent all attacks, it greatly reduces the risk and impact of many threats. Proper logging is also required by many compliance frameworks.

What's the care factor?

For most organizations, safeguarding log integrity should be a high priority security control, especially for systems handling sensitive data or performing critical functions. Regulators and auditors will likely require it. While implementing it takes some effort, the risks of operating without trusted logging are often too high.

However, for very small or low-risk systems, the full set of controls may be overkill. A risk-based approach should be taken to prioritize logging efforts.

When is it relevant?

Implementing strict log integrity controls makes sense when:

• Systems process highly sensitive data like financial, health, or personal information
• Workloads must comply with regulations like PCI DSS, HIPAA, SOX, etc.
• Applications are business-critical or have high availability requirements
• Environments have many users and complex access control needs

It may be less relevant for:

• Test/Dev systems with dummy data
• Isolated workloads not exposed to users or networks
• Immutable or read-only workloads where tampering risk is very low

What are the trade offs?

Restricting access and making logs immutable comes with some operational overhead and complexity:

• Admins have to request and justify access to make any logging config changes
• Troubleshooting issues with logging pipelines can be more difficult
• Once written, log entries can't be deleted, requiring more storage over time
• Developers may push back on lack of access to application logs
• Encrypting log data in transit and at rest adds performance overhead

So while the security benefits are substantial, organizations need to plan for the administrative challenges and costs.

How to make it happen?

1. Use a managed cloud logging service like AWS CloudWatch Logs to durably store log data
2. Create a least-privilege IAM policy to make log groups and streams read-only
   • Use policy conditions to restrict logs:PutLogEvents to specific roles/users
   • Deny logs:DeleteLogGroup, logs:DeleteLogStream, logs:PutRetentionPolicy
3. Enable CloudTrail logging on the CloudWatch Logs API to record all access attempts
4. Stream CloudWatch Log data to a SIEM or centralized logging solution for monitoring
5. Configure metric filters and alarms to alert on any unauthorized API calls
6. Require multi-factor authentication (MFA) and approval workflows for any logging config changes
7. Implement separation of duties between teams managing logging infrastructure vs those using it
8. Encrypt log data at rest using KMS and in transit using HTTPS endpoints
9. Set log retention periods according to compliance and investigation needs
10. Document procedures for break-glass access to logs and perform periodic auditing

What are some gotchas?

• The IAM policy for CloudWatch Logs can be complex to get least-privilege right. Test extensively!
  • Specific permissions needed: logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents, logs:DescribeLogGroups, logs:DescribeLogStreams
• If a SIEM is used, ensure it also has strict access controls and auditing enabled
• Custom apps may use the AWS SDK to interact with CloudWatch APIs, so SDKs must be configured to use encrypted endpoints
• Encryption with KMS requires the calling principal to have kms:Generatkey and kms:Decrypt permissions on the CMK
• High volume log streams can incur significant storage and retrieval costs over time
• Some legacy apps may not natively support logging to CloudWatch and require a log shipping agent

Reference: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/auth-and-access-control-cwl.html

What are the alternatives?

Some alternative approaches to securing logs include:

• Using an immutable S3 bucket as the log storage backend, with versioning and MFA-delete enabled
• Forwarding logs directly into a SIEM via syslog/HTTP rather than CloudWatch
• Using 3rd party log shipping agents like FluentD, Logstash, Splunk UniversalForwarding
• Running a self-managed ELK stack with strict access control and auditing enabled

Each has pros and cons to consider, but generally the managed services approach using CloudWatch is simplest. What's most important is to use a solution that makes logs immutable and tightly restricts access.

Explore further

• AWS re:Invent SEC403 Presentation on logging best practices: https://www.youtube.com/watch?v=kf4pJBJdzAo
• CIS AWS Foundations Benchmark section 2 for additional logging guidelines: https://www.cisecurity.org/benchmark/amazon_web_services
• PCI DSS Requirement 10 for regulated entities: https://www.pcisecuritystandards.org/

Implementing this control helps satisfy several related compliance requirements and security best practices. Explore the linked references to dive deeper into the rationale and implementation details. With proper planning and execution, organizations can establish a robust logging solution that enables security while meeting business needs.

‍