In this blog post, I'll guide you through seamlessly syncing your Git repositories from Azure DevOps to AWS CodeCommit using the power of an Azure DevOps pipeline. This automated setup ensures that your source repository in Azure DevOps and its replica in AWS stay in perfect harmony. Whenever updates are pushed to the source repository, our pipeline springs into action, efficiently cloning the repository and pushing the changes to its AWS CodeCommit replica.

High-level architecture of the Pipeline

To set up repository replication in the AWS Cloud, I'll walk you through the following steps in this blog post:

1. Establish a CodeCommit repository.

2. Generate a policy, user, and secure HTTPS Git credentials using AWS Identity and Access Management (IAM).

3. Craft a pipeline within Azure DevOps.

Prerequisites

Before diving into the process, it's essential to ensure you have the following prerequisites in place:

1. An active AWS account.

2. An existing Azure DevOps repository.

Creating a repository in CodeCommit

To prepare your CodeCommit replica repository and obtain its URL and ARN, follow these step-by-step instructions:

1. Begin by creating a new CodeCommit repository in your preferred AWS region. Pick a distinctive name that reminds you it's a replica or backup repository, for instance, "my-repo" Please note that you should refrain from manually pushing any changes to this replica repository, as it could lead to conflicts when your pipeline syncs change from the source repository. Treat it as a read-only repository and ensure that all development changes are pushed to your source repository.

2. Navigate to the AWS CodeCommit console.

3. Select Repositories from the list of options.

4. Locate your newly created repository and click on View Repository.

5. Click on Clone URL then select Clone HTTPS This action will copy the repository's URL. Save this URL by pasting it into a plain-text editor for future reference.

6. In the navigation pane, under Repositories, choose Settings.

7. Copy the value of the Repository ARN and save it by pasting it into a plain-text editor.

You now have the URL and ARN for your CodeCommit replica repository, which will be essential for setting up the IAM Role and synchronization pipeline.

Creating a policy, user, and Git credentials in IAM

The pipeline needs permissions and credentials to push commits to your CodeCommit repository. In this example, you create an IAM policy, IAM user, and HTTPS Git credentials for the pipeline to give it access to your repository in AWS. You grant the least privilege to the IAM user so the pipeline can only push to your replica repository.

To create the IAM policy, complete the following steps:

1. On the IAM console, choose Policies.

2. Choose Create Policy.

3. Choose JSON.

4. Enter a policy that grants permission to push commits to your repository. You can use a policy that’s similar to the following. For the Resource element, specify the ARN of your CodeCommit repository:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": "codecommit:GitPush",
                "Resource": "arn:aws:codecommit:eu-west-2:<account-id>:my-repo"
            }
        ]
    }
   

5. Choose Review policy.

6. For Name, enter a name for your policy (for example, codecommit-azure-repo-policy).

7. Choose to Create policy.

Create an IAM user

1. On the IAM console, choose Users.

2. Choose Add user.

3. Enter a User name (for example, codecommit-azure-devops-pipeline-user).

4. Select Programmatic access.

5. Choose Next: Permissions.

6. Select Attach existing policies directly and select the IAM policy you created "codecommit-azure-repo-policy".

7. Choose Next: Tags.

8. Choose Next: Review.

9. Choose Create user.

10. When presented with security credentials, choose Close.

11. Choose your new user by clicking on the user name link.

12. Choose Security Credentials.

13. Under Access keys, remove the existing access key.

14. Under HTTPS Git credentials for AWS CodeCommit, choose Generate credentials.

15. Choose Download credentials to save the username and password.

16. Choose Close.

Creating a pipeline in Azure DevOps

In order to execute the pipeline discussed in this blog post, which mirrors your source repository and synchronizes it with your CodeCommit repository, you'll need to obtain the following essential details: the URL of your source repository and HTTPS Git credentials. Here's how to do it:

To identify the URL of your source repository and generate the necessary HTTPS Git credentials, please follow the steps below:

1. Go to the Repos page within Azure DevOps and choose your repository.

2. Choose Clone.

3. Choose HTTPS.

4. Copy and save the URL by pasting it into a plain-text editor.

5. Choose Generate Git Credentials.

6. Copy the user name and password and save them by pasting them into a plain-text editor.

Now that you have the URL and HTTPS Git credentials, create a pipeline.

1. Go to the Pipeline page within Azure DevOps.

2. Choose Create Pipeline (or New Pipeline).

3. Choose Azure Repos Git.

4. Choose your repository.

5. Choose Starter pipeline.

Enter or copy the following YAML code to replace the default pipeline YAML:

# This pipeline will automatically mirror an Azure DevOps repository in AWS CodeCommit

# Trigger on all branches
trigger:
- '*'

# Use latest Ubuntu image
pool:
  vmImage: 'ubuntu-latest'

# Pipeline
steps:
- checkout: none
- script: |

      # Install urlencode function to encode reserved characters in passwords
      sudo apt-get install gridsite-clients

      # Create local mirror of Azure DevOps repository
      git clone --mirror https://${AZURE_GIT_USERNAME}:$(urlencode ${AZURE_GIT_PASSWORD})@${AZURE_REPO_URL} my-mirror-repo

      # Sync AWS CodeCommit repository
      cd my-mirror-repo
      git push --mirror https://${AWS_GIT_USERNAME}:$(urlencode ${AWS_GIT_PASSWORD})@${AWS_REPO_URL}

  displayName: 'Sync repository with AWS CodeCommit'
  env:
    AZURE_REPO_URL: $(AZURE_REPO_URL)
    AZURE_GIT_USERNAME: $(AZURE_GIT_USERNAME)
    AZURE_GIT_PASSWORD: $(AZURE_GIT_PASSWORD)
    AWS_REPO_URL: $(AWS_REPO_URL)
    AWS_GIT_USERNAME: $(AWS_GIT_USERNAME)
    AWS_GIT_PASSWORD: $(AWS_GIT_PASSWORD)

Add the following variables to your pipeline using the steps below:

1. Choose Variables.

2. Choose New Variable.

3. Enter the variable Name and Value.

4. Select Keep this value secret when adding any user name or password variable.

5. Choose OK.

6. Repeat for each variable.

7. Choose Save.

8. Choose Save and Run.

Test the Pipeline

Upon saving the pipeline, it commits the pipeline's YAML file (azure-pipelines.yml) to the root of your primary branch in the source repository. Subsequently, the pipeline will be triggered to run automatically. To confirm the successful execution of the pipeline, follow these steps:

1. Azure DevOps Pipelines:

   • Go to the Pipeline page within Azure DevOps and choose your pipeline.

   • Choose the entry for the latest run.

   • Under Jobs, choose Job to view the output of your pipeline.

2. AWS CodeCommit Console:

   • On the AWS CodeCommit console, choose Repositories.

   • Choose your repository and choose View Repository.

   • On the navigation pane, choose Commits.

   • Verify that the CodeCommit repository contains the latest commit from Azure DevOps.

By following these steps, you can easily confirm that your pipeline ran without issues and that your replica repository in AWS CodeCommit is up-to-date and synchronized with your source repository.

Note: The pipeline runs whenever a new commit is pushed to the source repository. All updates are mirrored in the replica CodeCommit repository, including commits, branches, and references.

Cleaning up

When you’ve completed all steps and are finished testing, follow these steps to delete resources to avoid incurring costs:

1. On the CodeCommit console, choose Repositories.

2. Choose your repository and choose Delete Repository.

3. On the IAM console, choose Users.

4. Choose your pipeline user and choose Delete User.

5. On the navigation pane, choose Policies.

6. Choose your CodeCommit Git push policy and choose Policy Actions and Delete.

7. Go to the Pipeline page within Azure DevOps and choose your pipeline.

8. Choose More Actions and choose Delete.

Conclusion

This post showed how you can use an Azure DevOps pipeline to mirror an Azure repository in AWS CodeCommit. It provided detailed instructions on setting up your replica repository in AWS CodeCommit, creating a least privilege access IAM policy and user credentials for the pipeline in IAM, and creating the pipeline in Azure DevOps.