Skip to main content

Track DORA metrics

This guide is designed to help you implement and track DevOps Research and Assessment (DORA) metrics within your organization in Port.

DORA Metrics are a set of key performance indicators that measure the effectiveness and efficiency of your software development and delivery process. By tracking these metrics, you can identify areas for improvement and ensure that your team is delivering high-quality software efficiently. This guide will cover the four key metrics: deployment frequency, lead time, change failure rate, and mean time to recovery.

Prerequisites​

  • Complete the Port onboarding process.
  • Access to a repository (GitHub, GitLab, or Azure Repos) that is connected to Port via the onboarding process.
  • While this guide demonstrates implementations using GitHub, GitLab, and Azure Repos, other Git providers can be used as well.
  • Optional for advanced strategies: If you're using workflows or pipelines, ensure they are configured for deployment tracking by following the relevant setup guides, such as CI/CD integrations or your platform-specific tools.

Tracking deployments​

In this section, we will cover how to track your team's deployments. Deployments refer to releasing new or updated code into various environments, such as Production, Staging, or Testing. Tracking deployments helps you understand how efficiently your team ships features and monitors release stability.

Deployments contribute to three key DORA metrics:

  • Deployment Frequency: How often changes are deployed to production or other environments.
  • Change Failure Rate: The percentage of deployments that fail and require intervention, rollback, or generate issues.
  • Lead Time for Changes: The time it takes from code commit to deployment into production.

To track the necessary data for these metrics, we will create a Deployment Blueprint with properties that capture the essential information for each deployment.

Data model setup​

  1. Navigate to your Port Builder page.
  2. Click the + Blueprint button to create a new blueprint.
  3. Name it Deployment and add the schema below:
Deployment blueprint (click to expand)
{
  "identifier": "deployment",
  "title": "Deployment",
  "icon": "Rocket",
  "schema": {
    "properties": {
      "createdAt": {
        "title": "Deployment Time",
        "type": "string",
        "format": "date-time",
        "description": "The timestamp when the deployment was triggered."
      },
      "environment": {
        "title": "Environment",
        "type": "string",
        "enum": [
          "Production",
          "Staging",
          "Testing"
        ],
        "description": "The environment where the deployment occurred."
      },
      "deploymentStatus": {
        "title": "Deployment Status",
        "type": "string",
        "enum": [
          "Success",
          "Failed"
        ],
        "description": "Indicates whether the deployment was successful or failed."
      }
    },
    "required": []
  },
  "mirrorProperties": {
    "leadTimeHours": {
      "title": "Lead Time (Hours)",
      "path": "pullRequest.leadTimeHours"
    }
  },
  "calculationProperties": {},
  "aggregationProperties": {},
  "relations": {
    "service": {
      "title": "Service",
      "target": "service",
      "required": false,
      "many": false
    },
    "pullRequest": {
      "title": "Pull Request",
      "target": "githubPullRequest",
      "required": false,
      "many": false
    }
  }
}
Missing Lead Time

If you do not have the lead time configured, you can follow the integration guide for your Git provider to add this property:

Adding JSON Schema Using Port's UI
  1. Go to the Builder in your Port portal.
  2. Click on "+ Blueprint" to create a new blueprint.
  3. Click on the {...} button in the top right corner, and choose "Edit JSON".
  4. Add this JSON schema to define the properties and relations needed for your blueprint.

By following these steps, you can paste and manage the JSON schema required to track DORA metrics in Port.

Tracking strategies​

Below are the main ways you can track deployments directly within Port:

One of the ways to track deployments is by monitoring when pull requests (PRs)/ merge request (MRs) are merged into a branch, typically the main/master branch. This is the recommended approach for tracking deployments and calculating lead time.

The lead time for these merges is calculated as the difference between when the PR/MR was created and when it was merged.

Below is a demonstration of how deployment tracking can be implemented using the PR merge strategy.

Example:

  • When a PR is merged, a deployment entity is created in Port to represent the deployment that took place.
  • The lead time for that PR is calculated and added to the deployment as part of the blueprint.

Here’s how you can implement this:

  1. Add Pull Request blueprint, sample can be found here.

  2. Add the configuration below to the data sources page in your Port portal, and select your GitHub integration:

Deployment config (click to expand)
 - kind: pull-request
   selector:
    query: .base.ref == 'main' and .state == 'closed'  # Track PRs merged into the main branch
   port:
     entity:
       mappings:
         identifier: .head.repo.name + '-' + (.id|tostring)
         title: .head.repo.name + " Deployment"
         blueprint: '"deployment"'
         properties:
          environment: '"Production"'  # Hardcoded value
          createdAt: .merged_at
          deploymentStatus: '"Success"' # Hardcoded value
       relations:
         pullRequest: .head.repo.name + (.id|tostring)
         service: .head.repo.name
Hardcoded values

The value for deploymentStatus is hardcoded to Success to treat all deployments as successful, and the environment is hardcoded to Production for the main branch in this example. You can modify these values based on your requirements.


Monorepo Tracking​

By using custom integrations, you can effectively track services or components within a monorepo.

Here’s how you can do this:

  • Mapping Service with Monorepo: You can track the individual services or features within a monorepo by adjusting the mappings associated with the file kind. Each microservice or feature should have distinct properties such as name, deployment status, and dependencies.

  • Ingest Data via Port API: Use Port's API to track changes, deployments, or pull request merges for each service individually. By using selectors, you can map specific parts of the monorepo (e.g., specific directories) to corresponding services.

The following YAML example demonstrates how to track multiple services (e.g., service-A, service-B) within a monorepo:

resources:
  - kind: repository
    selector:
      query: ".name == 'monorepo' && (.path | startswith('service-A/') || .path | startswith('service-B/'))"
    port:
      entity:
        mappings:
          identifier: .path
          title: .path | split('/')[0]
          blueprint: '"service"'
          properties:
            url: .html_url
            defaultBranch: .default_branch

In this setup:

  • The query checks if the repository is the monorepo and if the file path starts with either service-A/ or service-B/.
  • The identifier is mapped to the path, which uniquely identifies the service.
  • The title is derived from the service name in the file path.

By this method, individual services within a monorepo are mapped to Port blueprints.

Custom Integration Benefits

Custom integrations provide flexibility in mapping and tracking each service or microservice within a monorepo. With Port’s API, you can track deployments and updates for each component separately, giving you granular control over monitoring and managing services in a monorepo.

Tracking incidents​

Incidents are essential for tracking key DORA metrics, including Change Failure Rate (CFR) and Mean Time to Recovery (MTTR). Effective incident tracking reveals insights into how frequently deployments fail and how quickly teams can resolve issues. This section outlines how to:

  • Use incidents to calculate CFR and MTTR.
  • Link incidents to services to track the impact of failures.
  • Aggregate metrics across incidents for better monitoring.

Data model setup​

Ensure that your PagerDuty incident blueprint is properly configured to map incidents to the correct services(gitHub). This includes defining the appropriate properties and relations for incidents.Follow this PagerDuty Incident Blueprint Setup Link to implement.

  • Add the following properties to capture incident resolution time and recovery time:

    Additional properties for PagerDuty Incident Blueprint (click to expand)
     "resolvedAt": {
       "title": "Incident Resolution Time",
       "type": "string",
       "format": "date-time",
       "description": "The timestamp when the incident was resolved"
     },
     "recoveryTime": {
       "title": "Time to Recovery",
       "type": "number",
       "description": "The time (in minutes) between the incident being triggered and resolved"
     }

  • Add this mapping config to pagerduty incident data source:

    Incident mapping config for resolvedAt and recoveryTime (click to expand)
       resolvedAt: .resolved_at
       recoveryTime: >-
          (.created_at as $createdAt | .resolved_at as $resolvedAt |
          if $resolvedAt == null then null else 
          ( ($resolvedAt | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime) -
            ($createdAt | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime) ) / 60 end) # Time in minutes and divide by 3600 if you want it calculated in hours

Syncing incidents with PagerDuty and other tools​

To sync incidents from PagerDuty, follow the steps in the PagerDuty guide. The guide provides detailed steps for setting up integrations to track incidents related to deployments.

For other incident management tools, follow these respective guides:

Relating Incident to services​

Add this relationship to the Incident blueprint to link incidents to GitHub or Gitlab or Azure DevOps repository (service):

This section covers how to map incidents to GitHub repositories using the gitHubRepository relation.

{
  "gitHubRepository": {
    "title": "GitHub Service",
    "target": "service",
    "required": false,
    "many": false
  }
}

Update the mapping config to pagerduty incident data source:

Incident mapping config (click to expand)
   - kind: incidents
     selector:
        query: 'true'
        ...: # Add other selectors as needed
     port:
        entity:
           mappings:
              identifier: .id | tostring
              title: .title
              blueprint: '"pagerdutyIncident"'
              properties:
                 status: .status
                 url: .self
                 resolvedAt: .resolved_at
                 recoveryTime: >-
                    (.created_at as $createdAt | .resolved_at as $resolvedAt |
                    if $resolvedAt == null then null else 
                    ( ($resolvedAt | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime) -
                      ($createdAt | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime) ) / 60 end)
                 ... # Add other properties as needed
              relations:
                 pagerdutyService: .service.id
                 # Add this relation to map the incident to the correct service
                 gitHubRepository:
                    combinator: '"and"'
                    rules:
                       - property: '"$title"'
                         operator: '"="'
                         value: .service.summary
Mapping Incidents to Services

we use the search relation entity to map the pagerdutyIncident blueprint to the correct service based on the service's $title and the pagerduty incident service.summary. We have assumed that the service name/title exists in the service.summary property of the incident, but you can modify this query to map based on other properties that better match your setup To learn more about using search relations, see our documentation on Mapping Relations Using Search Queries.

Metrics​

We will now aggregate the DORA metrics at the service level, allowing us to track metrics such as Deployment Frequency, Change Lead Time, Change Failure Rate (CFR), and Mean Time to Recovery (MTTR) for each service.

Metrics Aggregation

If you want to track these metrics at higher levels, such as team or domain, makes sure the appropriate team or domain blueprints exist, and that they have relationships defined with the service blueprint. Then, you can apply the aggregation properties for these higher hierarchies, similar to how we are doing for the service blueprint below.

Aggregation​

The metrics in this guide are aggregated monthly. However, you can easily switch the timeframes to weekly, hourly, etc., based on your requirements.

Adding Aggregation to Blueprints

Before proceeding, follow these steps to add the aggregation and calculation properties to the Service Blueprint:

  1. Go to the Builder in your Port portal.
  2. Locate and select your Service blueprint.
  3. Click the {...} button in the top right corner, and choose Edit JSON.
  4. Insert the respective aggregation or calculation properties under the aggregationProperties or calculationProperties section in the Service blueprint's JSON schema.
  5. Save your changes to apply the new aggregation configuration.

Add this aggregation property to calculate deployment frequency:

Deployment Frequency (click to expand)

"deployment_frequency": {
   "title": "Monthly Deployment Frequency",
   "icon": "DeploymentIcon",
   "type": "number",
   "target": "deployment",
   "query": {
      "combinator": "and",
      "rules": [
         {
            "property": "deploymentStatus",
            "operator": "=",
            "value": "Success"
         }
      ]
   },
   "calculationSpec": {
      "func": "average",
      "averageOf": "month",
      "measureTimeBy": "$createdAt",
      "calculationBy": "entities"
   }
}

Aggregation Data Availability

At this point, you can already visit each service to view the aggregated DORA metrics. However, note that the aggregation data will only be calculated based on newly ingested data moving forward. click here for more details on aggregation properties.

Visualization​

By leveraging Port's Dashboards, you can create custom dashboards to track the metrics and monitor your team's performance over time.

Dashboard setup​

  1. Go to your software catalog.
  2. Click on the + New button in the left sidebar.
  3. Select New dashboard.
  4. Name the dashboard (e.g., DORA Metrics), choose an icon if desired, and click Create.

This will create a new empty dashboard. Let's get ready-to-add widgets

Adding widgets​

Setup Deployment Frequency Widget

  1. Click + Widget and select Number Chart.

  2. Title: Deployment Frequency - Monthly, (add the rocket icon).

  3. Select Display single property and choose Service as the Blueprint.

  4. Select an Entity and choose Monthly Deployment Frequency as the Property.

  5. Click Save.

Setup MTTR Widget

  1. Click + Widget and select Number Chart.

  2. Title: MTTR – Monthly Average (Seconds), (add the pagerduty icon).

  3. Select Display single property and choose Service as the Blueprint.

  4. Select an Entity and choose Mean Time to Recovery as the Property.

  5. Click Save.

Setup Change Lead Time Widget

  1. Click + Widget and select Number Chart.

  2. Title: Lead Time for Changes (Hour), (add LineChart icon).

  3. Select Aggregate by property and choose Service as the Blueprint.

  4. Select average as the function and choose Hour for Average of and createdAt as the Measure Time By.

  5. Add custom Unit for Unit of Measurement (Hours).

  6. Click Save.

Change Lead Time Over Time

  1. Click + Widget and select Line Chart.

  2. Title: Change Lead Time Over Time, (add the LineChart icon).

  3. Choose Service as the Blueprint.

  4. Select an Entity and choose Lead time for change as the Property.

  5. Set Time Interval to Month and Time Range to In the past 365 days.

  6. Click Save.

Deployments Frequency Over Time

  1. Click + Widget and select Line Chart.

  2. Title: Deployments Frequency Over Time (add the rocket icon).

  3. Select Service as the Blueprint.

  4. Select Monthly Deployment Frequency as the Property.

  5. Set Time Interval to Month and Time Range to In the past 365 days.

  6. Click Save.

Metric widget groupings

It would be visually cleaner and more informative to group related widgets, such as the Line Chart and Number Chart widgets, side by side for easier comparison. You can replicate more examples by checking our dora metrics dashboard on the demo environment.

Congrats 🎉 You have successfully set up DORA metrics tracking in your portal 🔥