How to Handle AWS CodeCommit Authentication

This article was automatically translated from Japanese by AI. Read the original Japanese version →

Table of Contents

Lately I've been using AWS CodeCommit, AWS's Git repository service, at work. Honestly, I think GitHub is more convenient, but there are various business reasons like cost and security, so it can't be helped.

So this time, I'm going to talk about how to authenticate with CodeCommit. There are roughly four ways to do it:

  1. SSH
  2. HTTPS
  3. git-remote-codecommit
  4. aws cli credential-helper

Options 1 and 2 are familiar from services like GitHub, and there's nothing particularly tricky about them. The problem lies with 3 and 4—these two are a bit quirky.

To give the conclusion up front: I don't recommend 3. I recommend 4, though the official documentation has a trap, so I'll share how to avoid it.

Using git-remote-codecommit

Setup Steps for HTTPS Connections to AWS CodeCommit with git-remote-codecommit - AWS CodeCommitGit を変更するユーティリティである git-remote-codecommit を使用して、HTTPS 経由で CodeCommit リポジトリに接続するように設定する手順を説明します。これは、CodeCommit リポジトリへのフェデ…docs.aws.amazon.com

The official documentation states the following:

If you connect to CodeCommit using a root account, federated access, or temporary credentials, you must set up access using git-remote-codecommit.

In AWS, it's a common pattern to switch roles from a user-management AWS account into a specific project or service AWS account. When using CodeCommit permissions in that kind of setup, git-remote-codecommit becomes necessary.

It leverages Git's credentials.helper feature. Roughly speaking, when authenticating to access a Git repository, instead of performing standard authentication, it runs a command with a specific interface.

The basic usage is written in the official docs linked above, and it goes like this:

  1. First, install the Python-based command called git-remote-codecommit locally.
    pip install git-remote-codecommit
  2. Next, set the protocol part of the URL to codecommit when running git clone.
    git clone codecommit://MyDemoRepo my-demo-repo

That's all.

To explain a bit more: when the protocol part of a repository URL is codecommit, the git command looks for the git-remote-codecommit command. Then git-remote-codecommit reads ~/.aws/config just like the AWS CLI and takes care of fetching temporary credentials for you.

It looks like a very helpful feature, but there's a pitfall.

It executes Python and a pip-installed library. When using Git from a CLI terminal, it's (probably) fine, but when using it from editor extensions or GUI tools, environment variables can make it hard to tell which git and Python installation will actually run. Some GUI tools don't even let you inspect or change environment variables easily. (And in some cases, expecting people who use GUI tools to deal with all that is asking too much.)

Whether you use Python regularly or not, messing with your Python environment just for this tool is kind of a hassle.

Using the aws cli credential-helper

Setup Steps for HTTPS Connections to AWS CodeCommit Repositories on Linux, macOS, or UNIX with the AWS CLI Credential Helper - AWS CodeCommitLinux、macOS、または Unix で HTTPS 経由で CodeCommit リポジトリに接続するように設定するための詳細な手順について説明します。これには、認証情報ヘルパーの設定が含まれます。docs.aws.amazon.com

The official documentation states the following:

However, if you connect to CodeCommit using a root account, federated access, or temporary credentials, you can use the credential helper included in the AWS CLI.

So the use cases where this is needed are the same as git-remote-codecommit. This is the method I recommend.

That said, the official documentation linked above has a rather questionable snippet:

git config --global credential.helper '!aws codecommit credential-helper $@'
git config --global credential.UseHttpPath true

Don't casually tell people to overwrite their global Git config!

If someone uses other Git repositories alongside CodeCommit, this is guaranteed to cause trouble.

Personally, I recommend the following setup. Change the region if you're using something other than ap-northeast-1.

  1. Run the following in your local Git so that httpPath is used when connecting to CodeCommit.
    git config --global credential."https://git-codecommit.ap-northeast-1.amazonaws.com".UseHttpPath true
  2. Run the following when cloning a repository:
    git clone -c credential.helper= -c credential.helper='!aws --profile XXXXX codecommit credential-helper $@' https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/my-demo-repo

Although UseHttpPath is set in the global config, it's scoped to that specific host, so it should have virtually no impact on other Git repositories. If even that bothers you, you can put it in each individual repository's .git/config instead. Then, when cloning each repository, specify credential.helper along with the --profile. If you have multiple AWS accounts, your profile names will differ as well. Cloning with these flags automatically writes credential.helper into the repository's .git/config, so you won't need to edit anything afterward.

Even when running git pull or similar from a GUI tool, it reads .git/config and works properly. Some tools can't find the AWS CLI on their own; in that case, specify the absolute path instead of just !aws.

By the way, if you Google something like git-remote-codecommit sourcetree, you'll find quite a few posts saying "if you cloned with git-remote-codecommit and can't pull from SourceTree, write the config above in .git/config"—which honestly just feels like doing the work twice.

If you understand how .gitconfig works, this is nothing special, but it's also a surprisingly common stumbling block, so I jotted this down as a note. Hope it helps someone out.

Update & Correction (2021/11/02)

I updated the git clone example in this post. In short, I added -c credential.helper=.

  • Before
    git clone -c credential.helper='!aws --profile XXXXX codecommit credential-helper $@' https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/my-demo-repo
  • After
    git clone -c credential.helper= -c credential.helper='!aws --profile XXXXX codecommit credential-helper $@' https://git-codecommit.ap-northeast-1.amazonaws.com/v1/repos/my-demo-repo

Here is the explanation for why this is needed.

On a Mac, Git's default authentication method is set to Keychain, which apparently causes passwords to expire and fail with a 403 error, so the following workaround may be required.

Troubleshooting the Credential Helper and HTTPS Connections to AWS CodeCommit - AWS CodeCommitGit 認証情報と HTTPS を使用して CodeCommit に接続する際の一般的な問題を説明します。問題をトラブルシューティングするためのガイダンスも含まれます。docs.aws.amazon.com

The default version of Git released with OS X and macOS uses the Keychain Access utility to store generated credentials. For security reasons, the password generated for access to CodeCommit repositories is temporary, and after about 15 minutes the credentials stored in the keychain stop working.

With the default Git on a Mac, ~/.gitconfig (the global gitconfig) contains the following:

[credential]
    helper = osxkeychain

In terms of Git's behavior, when you have:

  • A credential.helper setting in ~/.gitconfig with no hostname specified
  • A credential.helper setting in an individual repository's .git/config with a hostname specified

the former takes precedence. More precisely, it tries fetching credentials with the former first, and only falls back to the latter if that fails. When it fails with a 403, the authentication itself failed, but fetching the stored credentials succeeded—so Git stops after trying only the former.

For details on that behavior, please refer to the Git documentation.

The AWS doc linked above suggests three solutions:

  1. Remove the osxkeychain setting from the hostless [credential] section.
  2. Keep it, but specify concrete hostnames like GitHub so it doesn't trigger for CodeCommit.
  3. Set an empty-string helper in the CodeCommit (host-specific) credential section.

My recommendation is option 3.

Options 1 and 2 probably need no explanation. As for option 3, when Git encounters a helper set to an empty string, it clears the list of candidate helpers up to that point. So if you want to override the helper configuration from an individual repository, this is the cleanest way to handle it.

I've updated the command in this post accordingly.