AWS cli tool is written in python, and as python3 is the most recent, this is what will be installed. The awscli tool is installed through pip3. Consider installing this in a virtual environment (Python Virtual Env )

# yum install python3

... edited...

Install  1 Package (+3 Dependent packages)

Total download size: 11 M
Installed size: 51 M
Is this ok [y/d/N]: y
Downloading packages:
(1/4): python3-3.7.0-0.20.rc1.amzn2.0.1.x86_64.rpm                                     |  64 kB  00:00:01     
(2/4): python3-pip-9.0.3-1.amzn2.0.1.no                                                | 1.9 MB  00:00:03     
(3/4): python3-setuptools-38.4.0-3.amzn2.0.6.noarch.rpm                                | 617 kB  00:00:01     
(4/4): python3-libs-3.7.0-0.20.rc1.amzn2.0.1.x86_64.rpm                                | 8.0 MB  00:00:14     
------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                        487 kB/s |  11 MB  00:00:22    

# pip3 install awscli
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting awscli
  Downloading https://files.pythonhosted.org/packages/f8/ab/ab7b15a7a5524f47bb39279a59a7afdb1237162159ba7ff15cab28c96915/awscli-1.16.15-py2.py3-none-any.whl (1.3MB)
    100% |████████████████████████████████| 1.3MB 981kB/s 
...edited...    

AWS linux 2 does have a awscli tool in the linux repo, based on python2, but it is not as recent as the pip installed one, this is the python3 based version, check if you need py3 or py2 before installing:-

# aws --version
aws-cli/1.14.8 Python/2.7.14 Linux/4.14.47-64.38.amzn2.x86_64 botocore/1.8.12
[root@amazonlinux02 ~]#

Compare to the pip3 installed version:-

# /usr/local/bin/aws --version
aws-cli/1.16.15 Python/3.7.0rc1 Linux/4.14.47-64.38.amzn2.x86_64 botocore/1.12.5
[root@amazonlinux02 ~]#

The aws configure command will set up a default profile, if you need to use different accounts or keys, these can be set in a named profile, the two files, config and credentials in ~/.aws/ control this (needless to say, these are made up keys and secrets!):-

config

[default]
output = text


[profile admin1]
role_arn = arn:aws:iam::0123456781234:role/role_admin
source_profile = default
region = eu-west-1


[profile profile2]
region = eu-west-2
source_profile = default
output = text

credentials

[default]
aws_access_key_id = QWERTYUIOPASDFGHKEYQ
aws_secret_access_key = HaMPb65IFf0bVoEiLSKEJtuCUo3490nWlrJBES9n


[profile2]
aws_access_key_id = QWERTYUIOPASDFGHKEYA
aws_secret_access_key = wifisUMegS9pY_tpOnQpSY0YJYSiqgeKneMWqqIa

Single Sign On configuration.

$ aws configure list 
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                <not set>             None    None
access_key     ****************3AGN   container-role    
secret_key     ****************k3sa   container-role    
    region                eu-west-9              env    ['AWS_REGION', 'AWS_DEFAULT_REGION']

Returns details about the IAM user or role whose credentials are used to call the operation. No permissions are required to perform this operation.

This can be used to test AWS CLI setup and network connectivity.

$ aws sts get-caller-identity
{
    "UserId": "AROAQ8AB123CD4321ABCD:me1234",
    "Account": "123443210986",
    "Arn": "arn:aws:sts::123443210986:assumed-role/AWSReservedSSO_Access_9d3a256a5a2001321/me1234"
}
$

Roles allow proviledge escalation for a user to perform specific tasks. For this to be used in the cli, an extra section is added to the .aws/config file:-

[default]
output = json
region = eu-west-1


[profile sandbox]
role_arn = arn:aws:iam::12345678:role/roles-sandboxadmin
source_profile = default
region = eu-west-1

When a cli command is run, the –profile sandbox is added to awitch to that role:-

server:~/.aws$ aws ec2 describe-instances --filters "Name=instance-type,Values=t2.micro" --query Reservations[].Instances[].InstanceId --profile sandbox
[
    "i-0b99cb98dfabcdefg",
    "i-04888fe41agfedcba"
]
server:~/.aws$

$ aws ec2 describe-instances --profile nonprod_admin

An error occurred (InvalidClientTokenId) when calling the AssumeRole operation: The security token included in the request is invalid.
$

This was solved by updating the aws_access_key_id and aws_secret_access_key in ~/.aws/credentials. As it said “The security token included in the request is invalid.” Obvious in hindsight.

boto3 is Amazon's python library to interface with the aws cli commands.

A client needs to be set up, and for local cli usage, this needs to be linked with a profile as set above.

#!/usr/bin/env python
 
import boto3
 
profile = 'nonprod_admin'
 
# Create ec2 client
session = boto3.session.Session(profile_name=profile)
ec2 = session.client('ec2')
 
# Create SQS client
session = boto3.session.Session(profile_name=profile)
sqs = session.client('sqs')

This client (ec2, sqs etc) can be used to set or retreive information as the user in the profile:-

AMIResponse = ec2.describe_images(Filters=[{'Name': 'name', 'Values': [Regex]}, ], Owners=['self'])

It's worth pointing out that Code Commit repos are tied to a particular AWS account, so if you operate in a multiple account environment, you will need to ensure your code is stored in the right repo, eg if you have a prod and staging/dev environment, maybe codecommit may not be the easiest repo choice.

See:- https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-https-unixes.html.

AWS CodeCommit is a git compatible repository. It uses the git command locally, but if you are using roles, there is a restriction on using only https, not ssh to communticate to the remote repo. Also, there is a tie in with the aws command line which is why CodeCommit is here and not under Git Cheatsheet.

AWS's IAM requires HTTPS Git credentials for AWS CodeCommit to be created from the Security Credentials tab under Users. Click Generate and make a note of the values. These will become your git credentials to be used on the cli. Also, you will be required to configure a Credential Helper, the name of this sounds like it is optional, but it isn't.

IAM periodically resets the password used with the git credentials (above) and the Credential Helper is used to call out to IAM to get the updates password which is then used in the git command.

git config --global credential.helper '!aws codecommit credential-helper $@'
git config --global credential.UseHttpPath true

Profiles can be defined per repository by using –local instead of –global.

In the .gitconfig file, the commands above add the helper line, but to use it with roles, it needs the –profile adding:-

$ more  /home/user/.gitconfig 
[credential]
	helper = !aws --profile sandbox codecommit credential-helper $@
	UseHttpPath = true
$

My understanding is that git feeds a string of arguments to the credential-helper ($@) and consumes the string returned to forward on to CodeCommit as the user password. As it is an aws command it can take the –profile option. Without that, the helper will try to return the IAM users credentials not the role's credentials, git will present these and it will pushes and pulls will fail with fatal: unable to access <xx-repo> : The requested URL returned error: 403

If you get an error similar to The requested URL returned error: 403, you may need to export your AWS credentials:-

$ export AWS_PROFILE=shared-services

This is shown with the role option –profile sandbox

$ aws codecommit create-repository --repository-name CIS-Hardening --repository-description "Repo for ansible code to harden aws Linux2 image." --profile sandbox
{
    "repositoryMetadata": {
        "repositoryDescription": "Repo for ansible code to harden aws Linux2 image.",
        "cloneUrlSsh": "ssh://git-codecommit.eu-west-2.amazonaws.com/v1/repos/CIS-AWS_Linux2",
        "repositoryId": "91fb1234-833e-4705-8e1c-xxxxxxxx",
        "lastModifiedDate": 1537199788.236,
        "accountId": "1234567890",
        "repositoryName": "CIS-Hardening",
        "Arn": "arn:aws:codecommit:eu-west-2:987654321:CIS-Hardening",
        "cloneUrlHttp": "https://git-codecommit.eu-west-2.amazonaws.com/v1/repos/CIS-AWS_Linux2",
        "creationDate": 1537199788.236
    }
}
$
$ aws codecommit list-repositories --profile sandbox
{
    "repositories": [
        {
            "repositoryName": "AMI_PackerDefinitions-AWS_Linux2",
            "repositoryId": "12345678-a3af-4d07-8319-20f112345678"
        },
        {
            "repositoryName": "CIS-Hardening",
            "repositoryId": "87654321-14cd-495a-8b4f-121987654321"
        }
    ]
}

See the Git cheatsheet for adding files etc to the repo.

This is an error encountered on a Mac:-

me (ajs/cpe-1806) $ git push --set-upstream origin ajs/cpe-1806
fatal: unable to access 'https://git-codecommit.eu-west-1.amazonaws.com/v1/repos/myrepo/': The requested URL returned error: 403
me (ajs/cpe-1806) $ 

It's caused (probably) by the “Helper” having rotated the access keys at the AWS end and the Keychain Access app caching the original credentials which are no longer valid.

Solution is to search for and delete CodeCommit credentials in Keychain Access.

me (ajs/cpe-1806) $ git push --set-upstream origin ajs/cpe-1806
Enumerating objects: 11, done.
Counting objects: 100% (11/11), done.
Delta compression using up to 4 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 988 bytes | 988.00 KiB/s, done.
Total 6 (delta 4), reused 0 (delta 0)
To https://git-codecommit.eu-west-1.amazonaws.com/v1/repos/myrepo
 * [new branch]      ajs/cpe-1806 -> ajs/cpe-1806
Branch 'ajs/cpe-1806' set up to track remote branch 'ajs/cpe-1806' from 'origin'.
me (ajs/cpe-1806) $

Used jq to process output, this requires json, so the export forces json format. -r gives a raw output, ie, no quotes round variables.

get_repos.sh:-

#!/bin/bash
# List repos from AWS codecommit
 
export AWS_DEFAULT_OUTPUT="json"
 
aws codecommit list-repositories | jq -r '.repositories | .[].repositoryName'

credential-age-check
pyside6-stuff

STANDARD='arn:aws:securityhub:::ruleset/nist-800-53/v/5.0.0'
 
aws securityhub  batch-update-standards-control-associations \
    --standards-control-association-updates \
    --profile OrgDeployRole \
    '[
    {
    "SecurityControlId": "IAM.9",
    "StandardsArn": "'"${STANDARD}"'",
    "AssociationStatus": "DISABLED",
    "UpdatedReason": "Not applicable to environment"
    }
    ]'

"StandardsArn": "'"${STANDARD}"'" 

JSON keys and values need to be quoted, so that's the outside double quotes, STANDARD cannot be expanded by the aws command so it is single quoted, but lastly as BASH sees it, it is double quoted so BASH can expand the variable “${STANDARD}” to the value it represents.

I think…. Not 100% sure on the single quotes usage