Skip to content

Research Project

Learning Goals

  • Assess the quality of scientific software
  • Practice contributing to an existing open-source project
  • Practice version control and development best practices within the context of a group assignment
  • Plan and schedule projects in terms of tasks, milestones, and time estimations, and re-plan as required
  • Improve research-related code using best software engineering practices

Project Context

Software Engineering has been moving rapidly in recent years. From automated documentation to program repair, new tools that facilitate software engineering processes and study developers have been transforming how software is built. Recently, the advent of deep learning techniques, including Large Language Models (LLMs), have accelerated research progress.

One aspect that has been fueling this rapid pace of research is open science practices, wherein researchers share their data and code, so that others may build upon it to create new techniques and study new phenomena. To help expand this effort, Software Engineering conferences (e.g., ICSE/FSE/ASE among others) have begun to institute artifact reviews where the code and data that supports a given paper is vetted by a group of three researchers to ensure that it is available and functions as intended. These Artifact Review Tracks typically follow ACM's Artifact Review and Badging Policies.

Despite the best efforts of artifact review committees, reviewers often have limited time to examine an artifact and they can still be difficult to use. Therefore, for this project, you will be selecting a reviewed artifact for a research paper published at either ICSE, ASE, or FSE, testing the artifact, and then creating a plan to improve the artifact over a two and half week sprint that ends on our last day of class.

Deliverables and Deadlines

This will be the second assignment with your group. There are two (2) deadlines for this project. Each of the core deliverables are described below. This project is worth a total of 200 points.

Detailed information for each of the deadlines has been split into its own subpage on the left.

Tip

This is a large assignment spanning from now until the end of the semester. We estimate that this project will take each student on the team on average 8 hours/week over the next 5 weeks, for a total of 40 hours. We highly recommend reading through the entire assignment before starting so you are aware of our expectations for the later deliverables.

To manage all of the write-ups, we recommend saving the pages as a PDF to print or annotate on as you work through the assignment with your team.

A) Artifact Quality Report and Implementation Plan – 70 points – due Tuesday, March 26th, 11:59pm

  • Artifact Quality Writeup (35 pts)
  • Project Planning (35 pts)

B) Development Sprint – 130 points – due Thursday, April 18th, 11:59pm

  • Process & Implementation Project Snapshot (80 pts)
  • Project Presentation (50 pts) - Short Video