Skip to content

Research Project Checkpoint 1: Artifact Assessment & Planning

Deliverables

Team Process & Planning – 70 points – due Tuesday, March 26th, 11:59pm

Artifact Selection

Your first task will be to select an artifact. You are free to choose any artifact from those accepted to the artifact tracks listed below:

Note

If there is another artifact for a conference not listed above that aligns with research interests on your team, let me know and I work with you to get approval for the artifact.

Warning

Make sure you select your artifact carefully. Read through the Abstracts and go and inspect the artifacts with your team before making a final decision.

You should be able to the find the artifact for the paper you are interested in by following the links in the abstract on the conf.reseachr page or by Googling the title of the artifact. Once you have located the artifact, you should either (i) fork the repository on Github (if the artifact is on GitHub) or (ii) create a private GitHub repository fir your team to work on. You should then create a Github Project board for your repository (you can follow the steps from the Assignment 1).

You must invite Dr. Moran as a contributor to your private repository! His GitHub username is kpmoran.

Warning

Note that there may be certain scenarios where you are not able to import the artifact into Github easily. If you have questions, please come and talk to Dr. Moran so that he can guide you.

You should use this team repository for all your development, and be sure to use good development practices, including keeping your commits cohesive and your commit messages informative. The same resources mentioned above provide more details on our guidelines.

We will be grading you on how well you follow the process we used for Assignment 1 and utilize this Project board:

  • Create issues for feature improvements or bug-fixes
  • When creating issue, assign team members and tag with appropriate labels
  • Create a pull request and reference the issue it will be resolving
  • Provide feedback to pull requests
  • Use a Kanban board to track your workflow

For this project, you will create issues or pull requests directly in your team repository. We will use your commit history and issue/pull request activity on Github to assess both your teamwork process and each member’s individual contributions towards the project. It is not acceptable for one person to commit all the work after synchronizing through other means.

For each code-based deliverable, we will look at a snapshot of your repository at the deadline.

Main Deliverables

Artifact Quality Assessment (35 pts)

Your first process task for this project will be to asses the current state of the artifact that you are working with. To this end, you should try to execute the artifact and explore whether it has any issues. Then you should write up a 1 - 2 page document containing your findings in the format below. We recommend that you keep it to around 1 page, 2 page is a hard limit.

You should minimally address the following sections:

  1. Artifact Availability
    Is everything stated in the artifact README available? If not what is missing? How does this impact the functionality of the artifact?

  2. Artifact Documentation
    Is the artifact well documented? What information is missing? What knowledge is assumed by the artifact? Is that assumption reasonable?

  3. Artifact Functionality
    Were you able to successfully run the artifact? Were there any errors or warnings when running it? Was it difficult to figure out how to run it? What difficulties did you encounter trying to run the artifact?

  4. Results
    Did running the artifact produce the results claimed? Do these results match the results from the paper?

Feel free to seek the assistance of the instructor in drafting this document.

You should save the file as a PDF and upload it to Webcourses. Only one team member needs to make the submission; as this will be a group submission on Webcourses.

Project Planning (35 pts)

Before your team jumps into development, you must first determine what features to implement. Schedule and hold an initial project planning meeting with your team to complete the steps outlined below.

Functional Requirements - Features & Bug Fixes

During this meeting, discuss potential functional requirements of this project. Consider what possible bug fixes or new features could be added based on your experience with running the tool for your writeup.

Then, document these functional requirements in the form of Feature Descriptions and Bug Fix Issues that follow the guidelines discussed in lecture.

You should come up with at least one feature/bug fix per student in your group.

As a team, you should then come up with a prioritization ranking for each user story. The prioritization should be based on two factors

  • Impact: how essential is this feature/bug fix to the overall functionality of the artifact to the scientific community, how beneficial it would be to the scientific community, and
  • Effort: how much time/effort is required to implement this feature/bug fix

Once you have your list of features/bug fixes, create a new GitHub Project board (click into Projects on your team repository). Then create two new columns to the left called "Features/Bug Fixes" and "Backlog". At this point in time, you should have "Features/Bug Fixes", "Backlog", "To-Do", "In Progress" and "Done", in that order from left to right. Feel free to add more columns if your teams decide that you need them during your development process

Add your feature/bug fix descriptions into the "Features/Bug Fixes" column using the “+ Add item” button at the bottom, create draft issues for each of your user stories.In the body of each of these draft issues, provide a brief but concrete justification of its prioritization ranking that your team decided on. You should order all the user stories in this column from highest to lowest priority.

Technical Requirements - Issues

Now that you have your prioritized list, consider the technical requirements of the various features/bug fixes and collectively decide on which one(s) you will be focusing on over the next software development sprint. In this project (as like most projects), your team is aiming to maximize the amount of value you are delivering to the research community given your constraints.

Your selected features/bug fixes should have relatively high priority based on your team's ranking, and you should actively take factors into consideration that may impact your development.

Selecting Appropriate Features/Bug Fixes

Given the amount of variations in each team's artifact, it's hard to give concrete guidelines on the number of features/bug fixes that a team needs to tackle. Teams could tackle 1 feature that requires major effort, or a few bug fixes that each requires lesser effort.

In general, we are expecting that user stories be selected given:

  • 2 sprints of 1.5 weeks each
  • number of team members on your team
  • assumption of 8 hours/week available per individual

The course staff is happy to discuss this with your team during OH and we highly recommend you do so if your team is unsure. We will also be providing you with feedback once you turn in this assignment.

The feature(s) you plan to implement should not be purely cosmetic or arbitrary. (e.g., only improving documentation is not sufficient).

Convert the feature(s) you decide to implement into technical requirements (i.e. what needs to be developed for this feature to work). Then, break these requirements down into manageable, assignable issues and document them on your Project board by adding them to “Backlog” column. At minimum, these issues should make use of GitHub features like milestones/labels/descriptions to denote:

  • Estimated effort for each task
  • Dependencies among tasks (if any)
  • Initial assignments for team members
  • Milestones to indicate which sprint the issue should be completed in (Sprint 1, Sprint 2, Stretch)
  • Acceptance criteria that describes how you know this task is done - think about how you will test this feature

You should actively assign and move these issues across your Kanban board as your team makes progress.

A Note on Grading

We will not assess how accurately you predicted your development process, nor will we be giving points based on the complexity or quality of your changes. The focus of our evaluations will be on how you decompose the problem, how you respond to unexpected circumstances, and how you analyze and reflect on your experience later on.

We will check your development progress at the end of each sprint. Please be proactive in your planning to ensure that you make notable progress in both of your sprints. We highly recommend reading ahead on what our expectations are for the end of each sprint as you plan your milestone goals.

Include a link to your GitHub board in your Webcourses submission. If you are unsure about your planned changes, you can ask for feedback from the instructor!

Grading

To receive full credit for the Artifact Quality Assessment, we expect:

  • All sections listed above are addressed in a roughly 1-2 page PDF document submitted to Webcourses
  • Document demonstrates a clear attempt to use, and a deep understanding of the artifact

To receive full credit for the project planning, we expect:

  • A GitHub project board linked to your team repository with:
    • A Feature/BugFix column containing at least one feature/bug fix per group member that satisfy the guidelines outlined in the sections above and in lecture
    • A Backlog column containing a series of GitHub issues describing the feature(s)/bug fixes that the team will tackle. Each issue makes use of GitHub features to denote all of the required information listed above