---
layout: markdown_page
title: "Category Direction - Code Testing and Coverage"
---

- TOC
{:toc}

## Code Testing

Code testing and coverage ensure that individual components built within a pipeline perform as expected, and are an important part of a Continuous Integration framework.

- [Issue List](https://gitlab.com/groups/gitlab-org/-/issues?scope=all&utf8=%E2%9C%93&state=opened&label_name[]=Category%3AUnit%20Testing)
- [Overall Vision](/direction/verify)
- [UX Research](https://gitlab.com/groups/gitlab-org/-/epics/592)

Interested in joining the conversation for this category? Please join us in the issues where we discuss this topic and can answer any questions you may have. Your contributions are more than welcome.

This page is maintained by the Product Manager for Testing, James Heimbuck ([E-mail](mailto:jheimbuck@gitlab.com))

## What's Next & Why
We know that displaying the [current code coverage value](https://about.gitlab.com/blog/2016/11/03/publish-code-coverage-report-with-gitlab-pages/) for a project is nice and displaying the value in a [pipeline](https://docs.gitlab.com/ee/user/project/pipelines/settings.html#test-coverage-parsing) is valuable. We also know that without historical context it can be hard to understand what that value means today vs. yesterday. To solve this problem for users we are next going to add a [csv download](https://gitlab.com/gitlab-org/gitlab/-/issues/209121) and a [graph of the code coverage](https://gitlab.com/gitlab-org/gitlab/issues/33743) value(s) found in jobs.

## Maturity Plan

This category is currently at the "Minimal" maturity level, and our next maturity target is "Viable" (see our [definitions of maturity levels](/direction/maturity/)). Key deliverables to acheive this are:

* [Historic Unit Test Insights](https://gitlab.com/gitlab-org/gitlab/issues/33932)
* [Detect and report on flaky tests](https://gitlab.com/gitlab-org/gitlab/issues/3673)
* [Automatic flaky test minimization](https://gitlab.com/gitlab-org/gitlab/issues/3583)

We may find in research that only some of these issues are needed to move the vision for this category forward. The work to move the vision is captured and being tracked in this [epic](https://gitlab.com/groups/gitlab-org/-/epics/1941).

## Competitive Landscape

Many other CI solutions can consume standard JUnit test output to display insights natively like [CircleCI](https://circleci.com/docs/2.0/collect-test-data/) or through a plugin like [Jenkins](https://plugins.jenkins.io/junit). 

There are new entries in the space utilizing ML/AI tech to optimize testing like [Launchable](https://launchableinc.com/solution/) and write test cases like [Diffblue](https://www.diffblue.com/).

In order to stay remain ahead of these competitors we will continue to push forward to make unit test data visible and actionable in the context of the Merge Request for developers with [JUnit reports](https://docs.gitlab.com/ee/ci/junit_test_reports.html#viewing-junit-test-reports-on-gitlab) and historical insights to identify flaky tests with issues like [gitlab#33932](https://gitlab.com/gitlab-org/gitlab/issues/33932).

## Top Customer Success/Sales Issue(s)

TBD

## Top Customer Issue(s)

The most popular issue in the Unit Testing category today is an interesting request to [introduce a Checks API](https://gitlab.com/gitlab-org/gitlab/issues/22187). This is a good first step towards cleaning up the appearance of the MR widget as more data is added there over time.

## Top Internal Customer Issue(s)

The QA department has opened an interesting issue [gitlab#14954](https://gitlab.com/gitlab-org/gitlab/issues/14954), aimed at solving a problem where they have have limited visibility into long test run times that can impact efficiency.

## Top Vision Item(s)
The top vision item is [gitlab#3673](https://gitlab.com/gitlab-org/gitlab/issues/3673) which will start to address the problem of flaky test results which cause developers to not trust test runs or force unnecessary reruns of tests. Both of those outcomes are undesirable and counter to our goal of minimizing the lead time of changes.