As Matt already wrote in his project introduction post we have been using Amazon CodeCatalyst to handle all of our project activities. This gives a very good indication into where CodeCatalyst is already "ready for prime time" and where further adjustments need to be made in order to develop the service into a fully fledge, all-or-nothing, integrated DevOps service.
The one pushing for using CodeCatalyst was myself, as I wanted to try out the new Amazon service that recently went "GA" after being announced at re:invent 2023 in a bigger team and in a real project. It was great fun - and we learned a lot, too!
Let's look at some details.

How we planned our tasks using "issues"

Our journey with CodeCatalyst started with a - very agile and intense - planning session using the "issues" component of CodeCatalyst. Given the nature of an hackathon, we created roughly the first ten issues in the system. Three of us were "new users" for CodeCatalyst, but they were able to quickly interact with the issues component - by default, this is a simple Kanban board and has a well structured user interface. One thing we directly missed was the ability to copy/paste images into comments within the issues...and to attach files to the issues themselves. Linking issues to one another is also only hardly possible right now.
We used a simple workflow and did a regular planning (twice per week) and created/updated/closed more than 50 issues (of partly very small scope) throught the course of less than four weeks. Overall, the available functionalities were good enough for our project and supported our project.

Collaborating source code - working with Git repositories and pull requests

The four of us are (Community) Builders from the bottom and all we need to be happy is a Git repository that we can push code to :-)
Of course this is not currect, but CodeCatalyst allows you to host you own, private Git repositories. You can have multiple repositories per project. In our case, we had decided to go with a mono-repo approach and so we strickly worked in a single repository. Working with the source code repository was "just fine" and working as expected. We also decided to merge as early and as often as possible to our "main" branch. The team however struggled in working with setting up Pull Requests and working in the UI when reviewing pull requests.
The most problematic things: UI response time, working with comments on pull requests (e.g. threaded comments) and creating pull requests. The whole UI does not yet feel as "structured" as in other competitors (e.g. Github) - examples: PR title/description proposals when creating a PR, link in terminal/commandline that allows your to create a PR.
Another thing we missed is "git blame" online - not to blame each other but to understand what changed and broke our "hacked" "hackathon" source code :-)
Cool: the markdown rendering for files named *.md. Missing here: including local images is not possible.

Working with Workflows - Github actions for the rescue!

We started off building our workflows with natively available actions in Amazon CodeCatalyst - using the "cdk deploy" action to deploy out infrastructure and the "s3 deploy" action to deploy our front end - but quickly shifted away from that and switched over to using Github Actions within the workflow to allow building our Flutter application as a CDK asset. As our Continuous Integration pieces matured within our npm implementation, there was less reason to go back to the native actions as the CI part was handled through npm.
Our current workflow and CI/CD pipeline:

The main preference here is to allow a continuous deployment to our production environment. This goes through an intial "sandbox" deployment, promotes to a "test" environment and then deploys to "production". The pipeline is prepared to execute integration tests, but they are not yet implemented - which is the nature of a _hack_athon :-)
This would definately one of the next things to change - adding real integration tests and also adding some security verifications.

Our workflow is, as I mentioned, pretty basic - here an excerpt of it until the deployment to our sandbox environment:

BuildAndTestCDKApp:
    Identifier: aws/github-actions-runner@v1
    Compute:
      Type: EC2
    Inputs:
      Sources:
        - WorkflowSource
      Variables:
        - Name: AWS_REGION
          Value: us-east-1
        - Name: NODE_ENV
          Value: sandbox
    Outputs:
      AutoDiscoverReports:
        Enabled: true
        ReportNamePrefix: rpt
        IncludePaths:
          - ./coverage/lcov.info
    Configuration:
      Steps:
        - name: Flutter Build Web
          uses: subosito/flutter-action@v2.8.0
          with:
            channel: stable
        - name: Java Install
          uses: actions/setup-java@v2
          with:
            distribution: zulu
            java-version: "11"
        - name: Setup Android SDK
          uses: android-actions/setup-android@v2
        - uses: actions/setup-node@v3
          with:
            node-version: 16
        - run: npm ci
        - run: npm run
        - run: npm run synth
    Environment:
      Connections:
        - Role: CodeCatalystWorkflowDevelopmentRole-AWSCommunityBuilders
          Name: community-builders-sandbox
      Name: speakers_sandbox
  DeploySandboxCDKApp:
    DependsOn:
      - BuildAndTestCDKApp
    Actions:
      DeployBase:
        Identifier: aws/github-actions-runner@v1
        Compute:
          Type: EC2
        Inputs:
          Sources:
            - WorkflowSource
          Variables:
            - Name: AWS_REGION
              Value: us-east-1
            - Name: NODE_ENV
              Value: sandbox
        Configuration:
          Steps:
            - name: Flutter Build Web
              uses: subosito/flutter-action@v2.8.0
              with:
                channel: stable
            - name: Java Install
              uses: actions/setup-java@v2
              with:
                distribution: zulu
                java-version: "11"
            - name: Setup Android SDK
              uses: android-actions/setup-android@v2
            - uses: actions/setup-node@v3
              with:
                node-version: 16
            - run: npm ci
            - run: npm run deploy
        Environment:
          Connections:
            - Role: CodeCatalystWorkflowDevelopmentRole-AWSCommunityBuilders
              Name: community-builders-sandbox
          Name: speakers_sandbox
  IntegrationTestSandbox:
    Identifier: aws/build@v1
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: ls -al
    Compute:
      Type: Lambda
    DependsOn:
      - DeploySandboxCDKApp

As you can see, a lot of the CD (and also the "deployment") runs within the corresponding task definitions in package.json.

No "Open Source" or "Readonly" view

We would really love to open-source our whole projet, to give you the possibility to get involved and learn from what we build, but unfortunately Amazon CodeCatalyst does currrently not offer an Open Source or a "Readonly" view and forces you to create an account for CodeCatalyst - these are things that make it difficult for us as we cannot link to our sources.
If you are an AWS Community Builder, please reach out to either of us in Slack and we can give you access to the project.

What did we really miss in the workflow component

• manual approvals before promotion to our "production" environment
• Notifications from workflows - could have been done through this implementation but we did not take time to implement that
• Integration of actions deployed through "npm - cdk deploy" with making the changes in infrastructure visible
• importing existing pipelines or render CDK pipelines to CodeCatalyst workflows The "killer feature" of the workflows, at least as far as we see it, is the possibility to use Github actions as part of the workflow. With only minor adjustmends an existing Github workflow can be transformed to a CodeCatalyst workflow. If either of you know about an existing tool that converts existing Github workflows to CodeCatalyst workflows in an automated way - please let me know.

The hidden gem: Development Environments

One of the "hidden gems" of CodeCatalyst are the Dev Environments - also known as Development Enviroments. Essentially, its a "service in the service" similar to Gitpod where you can host your IDE or the environment that you develop on. In our project we all didn't use it as the architecture was small and we only had this single project. Also, we were developing mainly on our main machines without switching between machines too often. That's why we all did not consider really using the DevEnvironments for this project. Last but not least, Flutter is not natively supported on the Dev Environments which would have forced us to prepare a devfile that includes the Flutter dependencies - and thats something that would have taken too much time out of the project.
Still, the De Environments within CodeCatalyst are hidden gems that I believe should get more traction than they are getting until now.

A summary of what we think works and what doesn't - and our biggest wishes

For a hackathon like this (3-6 weeks working time) CodeCatalyst seems to be a very good choice for a project that starts with a "new" or "fresh" codebase. The tool has most of the minimal requirements implemented to allow simple project management and good integration with AWS.
It also allowed us to quickly get started and to collaborate on our sources.

We really missed workflow notifications as well as manual approvals in the workflows.
As the tool matures I personally hope that more and more AWS internal teams will be using CodeCatalyst to develop, plan and deploy out their internal AWS services and with that the "flow" in the User Interface will definately be improved, too - as today there are some hickups in a real developers workflow.

And, last but not least - we would love to share this project with you, but given CodeCatalyst does not allow to "Open Source" a project, that's not possible :-)

Maybe something for the next hackathon?

If you would like to get involved in this project and help us to shape and build our AWS Speakers directory to make it easier for AWS User Group Leaders to find speakers - please reach out to either of us four about collaborating!

Who to reach out to?
Danielle, Matt, Julian or myself.