Assignment 4: Static Analysis and Continuous Integration

Possible Points	Due Date
50 pts	Monday February 19th - 11:59pm

Static and dynamic analysis tools help you keep your codebase healthy. In this recitation, we will learn how to set up these tools in CI (GitHub Actions).

Step 1: Setup Your Sample Python Repo

First, sign up for this assignment in GitHub Classroom using the button below. It will clone a template repo for you to work from.

Click Here to Accept the Assignment on GitHub Classroom

This repo implements a simple Python web application using Flask that returns the Fibonacci sequence value of a number. app.py implements the webserver, and fib.py implements the Fibonacci function.

You already learned that it's a generally not a good idea to push directly to main. We can actually enforce this using branch protect rules. Read the docs to understand what they are, and set the following rules:

• Requires a pull request before merging to main
• Requires tests to pass before merging to main:
• Search for the job name in the required checks (i.e. test in this case.) You may need to save the settings first before this search box appears.

Your setting should look like this:

The failing test cases ❌ wouldn’t have been there if we had these rules enabled in the first place. Now, let's fix our failing test.

Step 2: Fix the Broken CI

The ❌ really shouldn't have been there in the first place if we had these rules enabled. Now let's fix it. Branch off from main and create a PR to fix the broken CI.

Take a look in the Actions page to see which test is failing. Branch off from main and create a PR to fix the broken CI. (the fix should be VERY simple!)

The test job should pass on your PR. Click "Squash and merge"* to merge after the status checks pass.

*: It's just a lot cleaner than the default merge.

HINT: If you are really stuck on how to fix, click here, (but really, feel free to ask on Ed or come to office hours if you are having trouble 😉).

Step 3: Setup Your Local Development Environment

For local development below, we will be using python3.8 and the pipenv tool. Please make sure that you have both installed in your command line (either WSL, or your preferred command enviornment on Mac or Linux). I have included several links below to help you set these up, but if you still have questions, please feel free to reach out on Ed!

• Installing Python3 in WSL
• Installing Python3 on Mac
• Installing Python3 on Linux
• Installing pipenv on all OSes
• What is pipenv?: pipenv is a tool that allow us to easily configure our local development environment for Python. It allows us to easily install development tools and packages without having to worry too much about creating and managing different virtual environments. In python, in order for a program to run, it often requires a (sometimes very specific) set of dependencies. Virtual Environments allow us to create virtual "spaces" where we can install specific python packages (using the python package manager pip), and activate or deactivate the spaces to run different programs. Pipenv creates the virtual environments for you and manages them in your filesystem using Pipfiles and PipFiles.lock

Warning

Make sure you are running pipenv commands in the project directory so that the virtunalenv information gets saved to the proper location.

Step 4: Make your Code Pretty

Different tab sizes driving you crazy? Let's use a tool to standardize them all. A code formatter, a static analysis tool, helps one identify and fix formatting issues in the codebase. Let's use black as an example.

First, create another branch for setting up a code formatter.

Then, run the following commands to install it locally and try running it:

• pipenv install --dev black: black is only a development dependency. Your package doesn't actually use it.
• pipenv run black . --check:
• Runs black in the current directory. --check dry-runs black and don't alter any files.
• Observe some files on the list.
• pipenv run black .:
• This will actually change the files.
• Run git diff to observe the file changes.

Using CI, we can enforce formatting requirements using the same GH Actions + status checks. For popular tools, someone has done it before, and you can reuse their workflow.

• Go to this existing black Actions on GH Marketplace
• Click "Use latest version" to see what needs to be added to .github/workflows/main.yml
• Add another job called “format” to the main.yml file to use black to check the file formatting

After you are done, your main.yml file should look like the following:

Note

Check out the code annotations in the .yml file snippet below (marked as "+" signs), they explain what the different parts of the file are doing 🙂

name: build
on: push

jobs:
  format:
    runs-on: ubuntu-latest #(1)!
    timeout-minutes: 10
    steps:
      - name: Check out repository code
        uses: actions/checkout@v2 #(2)!

      - name: Setup Python
        uses: actions/setup-python@v2
        with:
          python-version: "3.8" #(3)!
      - name: Install pipenv
        run: |
          python -m pip install --upgrade pipenv wheel #(4)!
      - name: Install dependencies
        run: |
          pipenv install --deploy --dev #(5)!
      - name: Move to flask folder
        run: |
          cd app
      - name: Run black formatter
        uses: rickstaa/action-black@v1.3.3 #(6)!

  test:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - name: Check out repository code
        uses: actions/checkout@v2

      - name: Setup Python
        uses: actions/setup-python@v2
        with:
          python-version: "3.8"
      - name: Install pipenv
        run: |
          python -m pip install --upgrade pipenv wheel
      - name: Install dependencies
        run: |
          pipenv install --deploy --dev
      - name: Move to flask folder
        run: |
          cd app
      - name: Run test suite
        run: |
          pipenv run pytest

1. This command tells GitHub actions what type of server to boot up to run our CI. In this case, we are using the latest stable version of the Linux distro Ubuntu.
2. This command checks out our code and places a copy of our code repository from the latest commit on the ubuntu server that was just set up by GitHub actions
3. This command sets up and install Python version 3.8, which is what we would prefer to use to run this program.
4. This command installs pipenv and sets it to work in the sever using a Python wheel, which is a simple way to install python packages using pre-packaged binaries
5. This installs some of the packages specified in our PipEnv file.
6. This line runs the code formatter to check if there are formatting issues.

After you complete the above, you can finish up by:

• Pushing your formatted files to the branch and observe format passes.
• Squash and merge the PR

Step 5: Add Test Coverage to the CI Workflow

Finally, you can also do some dynamic analysis. Since we are already using pytest, let's use pytest-cov, a plugin that reports test coverage.

First, install and try to use it locally:

• Create another branch
• Install pytest-cov locally: pipenv install --dev pytest-cov
• Runs pytest with coverage report: pipenv run pytest --cov=app

Now, let's add another job in the workflow for reporting coverage:

• From the test workflow, copy the steps before pytest
• Now, run pipenv run pytest --cov=app to report coverage

After doing this, your main.yml file should look like this:

name: build
on: push

jobs:
  format:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - name: Check out repository code
        uses: actions/checkout@v2

      - name: Setup Python
        uses: actions/setup-python@v2
        with:
          python-version: "3.8"
      - name: Install pipenv
        run: |
          python -m pip install --upgrade pipenv wheel
      - name: Install dependencies
        run: |
          pipenv install --deploy --dev
      - name: Move to flask folder
        run: |
          cd app
      - name: Run black formatter
        uses: rickstaa/action-black@v1.3.3

  test:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - name: Check out repository code
        uses: actions/checkout@v2

      - name: Setup Python
        uses: actions/setup-python@v2
        with:
          python-version: "3.8"
      - name: Install pipenv
        run: |
          python -m pip install --upgrade pipenv wheel
      - name: Install dependencies
        run: |
          pipenv install --deploy --dev
      - name: Move to flask folder
        run: |
          cd app
      - name: Run test suite
        run: |
          pipenv run pytest --cov=app #(7)!

1. Here we modified our pytest command to also report the coverage, which you will be able to see in the results of the CI run.

Once you have updated this, you can push your changes and observe the new code coverage check running.

Bonus: report coverage in PRs

The coverage job doesn't really add much to the workflow now since it doesn't fail. Without being too strict about coverage, we can at least display the coverage status in the PR.

Somebody has already done it, so we can use it in our repo too. Hint: you should only need the last two steps in the workflow.

Note that this action will only run on pull request-based workflows, so you will need to modify your triggers.

If set up, the job will automatically comment on PRs with the coverage info:

Turning in the Assignment & Grading Rubric

To turn in Assignment 4 you just need to commit your changes to your repository created for you by GitHub classroom and by committing your changes implementing your fixes and setting up CI in your repo. You do not need to submit a link to the GitHub repository as we will be able to access it through GitHub classroom. You will see an Assignment 4 assignment in Webcourses, but it will not let you submit anything, this is normal!

The grading for this project will be broken down as follows:

• Part One: Fixing the Broken CI - (20 points total)
• Part Two: Making your Code Look Pretty - (15 points total)
• Part Three: Adding Test Coverage to the CI workflow - (15 points total)
• Extra Credit: Reporting Coverage in PRs - (5 points total)