The Amazon Web Services identity and access management (IAM) mechanism is complex, and not fully understanding its particularities often leads to misconfigurations and exposed cloud assets. Researchers from cloud security firm Lightspin identified quirks in S3 bucket permissions that appear to be a common source of confusion among administrators.
There are several ways in which access to data stored in S3 buckets can be restricted or granted, some more granular than others, but they're all interdependent. Over time, changes made to these individual policies can inadvertently expose data or open the storage buckets to operations from unauthorized users. This is particularly true when dealing with large buckets that have thousands or hundreds of thousands of objects, and the Lightspin researchers feel that AWS's warning messages are not clear or detailed enough for administrators to pinpoint the problems. That's why the company developed and released an open-source S3 bucket scanner that can identify public access and cross-account attack issues.
Objects can be public, but which ones exactly?
When dealing with S3 buckets, there are three methods of restricting public access: bucket ACLs (access control lists), which apply to the entire bucket, object ACLs, which apply to individual objects, and S3 bucket IAM policies. AWS also provides an S3 Block Public Access feature that's intended to help administrators secure their buckets by overriding existing ACLs, but this feature comes with four different options that have different effects, and while they give admins a lot of flexibility, they can also generate some confusion.
First of all, public access in the context of S3 ACLs refers to read or write permissions given to members of the AllUsers or AuthenticatedUsers groups. It's worth pointing out that AuthenticatedUsers means anyone with an AWS account, not just accounts from the same organization.
According to Lightspin researcher Noga Yam Amitai, the four options of the S3 Block Public Access feature and their respective impact are:
- BlockPublicAcls. When set to TRUE, no new ACL definitions are allowed, but existing ones still apply. Meaning, if there is an existing bucket with an ACL granting public access, the BlockPublicAcls is not going to affect it.
- IgnorePublicAcls. When set to TRUE, it causes Amazon S3 to ignore all public ACLs on a bucket and any objects that it contains. Meaning, all public access granted by a bucket or object ACL does not apply.
- BlockPublicPolicy. While set to TRUE, no new policies can be attached to the bucket, but existing ones still apply. Meaning, if there is an existing bucket policy that grants public access, the BlockPublicPolicy is not going to affect it.
- RestrictPublicBuckets. When set to TRUE, if there is a policy that allows public access, the access is restricted only to AWS service principals and authorized users within the bucket owner's account.
Amazon recommends in its documentation that in order to ensure all S3 access points, buckets, and objects have their public access blocked, all four settings should be turned on. However, in practice, companies can use the same bucket to store both public and non-public data for different applications, so blocking all public access to the entire bucket is not always practical. It's therefore important for administrators to understand how to effectively use different combinations of these 4 settings.
The AWS management console will provide an evaluation of the bucket policies with four possible statuses: Public; Bucket and objects not public; Only authorized users of this account; and Objects can be public.
"These four access options don’t necessarily allow you to provide definitive answers to whether your objects are public or not, and which buckets are secure," Yam Amitai said. "While ‘Public’ is a black and white outcome, and so is ‘Bucket and objects not public,’ the other two are open to confusion. In particular, the outcome of ‘Objects can be public’ leaves your security teams none the wiser about whether items are accessible or not."
As an example, the researcher says, let's say a developer sets up a new S3 bucket and creates a folder with non-sensitive information and uses ACLs to make its contents public. Later, the same developer decides to store some sensitive data in the same folder and decides to make the whole bucket non-public because its purpose has now changed, so it uses the IgnorePublicACL setting, which causes all existing public ACLs to become ineffective.
Months go by and the original developer moves on to a different company and is replaced by a new developer who decides to store another public folder in the existing bucket for a new application. They remove the IgnorePublicACL setting from the bucket, create the new folder and set its contents to public by using object ACLs. The status of the bucket will now be shown as "Objects can be public," which does not raise any alarm because the new developer just created a public folder inside. Unwittingly, he has also exposed the older folder created by the previous developer and which now has sensitive data inside.
This is what's known as a configuration drift, an all-too-common problem for cloud deployments. According to a report by cloud security firm Accurics, misconfigured storage buckets represent over 15% of cloud security issues identified in production environments. The company also found that 9 out of 10 organizations allow configuration changes to occur in runtime instead of configuration templates, which causes drifts to occur from the established secure baseline.
Lightspin recently scanned 40,000 S3 buckets and found that on average organizations have 4% of their buckets configured as public, but around 42% with the status “objects can be public," which could indicate potential misconfigurations. Over the years, misconfigured S3 buckets have resulted in serious data breaches and exposure of customer data for a number of high-profile companies.
Cross-account attack due to wildcard usage
During its S3 security research, the Lightspin researchers also identified a separate misconfiguration issue that has a high chance of occurring and could allow malicious users to write data into S3 buckets that don't belong to them.
Amazon provides several configuration, audit, and other tools to customers that run as services, such as AWS Config or CloudTrail. These services write the data they generate into S3 buckets and it's up to the customer to choose those buckets. They can create dedicated ones, use existing ones from their accounts, or direct these services to write in buckets from different accounts. The latter use case is for organizations that have multiple AWS accounts and want to use a dedicated one for all logging and auditing.
Configuring a bucket manually through policies to allow multiple accounts to write into it works well when the number of accounts is small, but doing it for a large number of accounts usually requires automation through the command line interface using resource paths and wildcards.
"When you have hundreds or thousands of AWS accounts in your organization, then you can't just go through each one of them and give permission to it, so we found a pattern that repeats with organizations using wildcards," Or Azarzar, the CTO of Lightspin, tells CSO. "They think that if they have a wildcard in one of the AWS accounts—the centralized one for logging or for auditing—and it's in their organization, that only accounts within the organization can access it. But what we figured out is basically any other AWS account can write logs or can write any data inside those buckets."
Examples of such problematic paths that use wildcards and open buckets to write access from any external accounts include:
- arn:aws:s3:::{bucket-name}/*
- arn:aws:s3:::{bucket-name}/AWSLogs/*
- arn:aws:s3:::{bucket-name}/AWSLogs/*/Config/*
The risk is somewhat limited with this particular misconfiguration because the external accounts only have write access, not read, and they can only write data that Amazon services generate, such as logs and configurations. But this can be an avenue for log poisoning, according to Azarzar. The information generated by the AWS audit services are used by security systems to detect potential malicious behavior. Attackers can take advantage of such a misconfiguration to inject rogue logs over time in order to poison the data and prevent monitoring systems from flagging future attacks against the organization's accounts.
Lightspin's latest research focused on AWS Config and CloudTrail, but these are not the only AWS services that can store data in S3 buckets, and others might require read permissions as well, which could lead to more serious data breaches. The researchers plan to continue and expand their analysis, Azarzar said.
Authorization misuse with IAM groups and users
Before the latest S3 research, Azarzar identified another potential source of confusion in the AWS authorization logic for IAM groups and users. While IAM allows administrators to define groups of users and apply identity-based policies to entire groups, the specific actions that policies can be enforced for are different between groups and individual users. This is somewhat unusual behavior when compared to other environments like Windows Active Directory or Linux.
Azarzar gives the example of a company that has three global AWS managers who should have full access to the company’s AWS account. The account administrator creates a “managers” group, attaches the “AdministratorAccess” policy to it, and adds the three users to the group. Then, the administrator wants to protect the group and the three managers from any potential account modifications in case some other high-privileged account gets hijacked, so it creates a policy called "ProtectManagers" with a DENY rule on all possible actions against the managers group and attaches this policy to all other users and groups that are not managers.
In AWS IAM, DENY policies override any existing ALLOW policies, so if there was an AWS account that's allowed to perform the iam:UpdateLoginProfile action—which can be used to change user passwords—the reasonable expectation would be that if the ProtectManagers policy is applied to that account, the account wouldn't be able to modify the passwords of members of the managers group. But that's not true. The policy only prevents group-specific actions from being performed and UpdateLoginProfile is a user-specific action. This means that in order to protect users of the managers group from password changes, separate deny policies need to be created for every individual user of the managers group, and then those policies need to be applied to all other non-manager users.
Another quirk is in how resource paths with wildcards in policies can mean different things in different contexts. For example, an identity-based policy that applies to the resource arn:aws:s3:::customersdata/* means it applies to all objects in an S3 bucket called customersdata. However, according to Azarzar, defining a policy for the path arn:aws:iam::123456789012:group/managers2/* would not apply to all users in the managers2 group, but only to the group itself as an object, which is counterintuitive.
"We approached AWS for a response, and they commented that this approach is by design, and not an error," he said in a blog post. "AWS treats groups as a separate object, and they don’t treat a user as part of a group when it comes to deny rules. As this can open up serious vulnerabilities for any organization who would assume that IAM works the same way as Active Directory on Azure for example [...,] we believe it’s important to draw attention to this issue."
"You know, the basic reason why I have a group is because I want this to be like sort of a container for my users, but if it doesn't work in all places in the same way, then it's really confusing," Azarzar tells CSO. "I think most of the security owners that are going from, let's say Active Directory to AWS IAM or from Linux administration to AWS IAM, expect this to be common sense: You have a group, you denied access to the group or denied the group access to certain assets, and you think that the objects in the group [the users] would have the same policies applied, but it doesn't work that way."
Changing the way groups work now would likely be problematic as it could break a lot of existing policies, but AWS IAM also supports defining access policies based on tags for resources and users and this can be used to achieve the behavior that one would normally expect with groups.
But if we have to replicate our group structures with tags and have huge lists of dedicated users we want to use, then what do we need groups for, Azarzar asks. "I think it creates confusion and more work for the other side to secure their environments."