Skip to main content

๐Ÿ“Š Promote Scorecards

What is a Scorecard?โ€‹

Scorecards enable us to create a set of rules that will determine the level of our Port entities, based on their properties. Each scorecard has a set of rules that affects its total level, a rule has a level property which is one of the following: Gold, Silver or Bronze. Each rule has specific conditions, and the scorecard level increases when they pass.

For example, to keep track of your organization's Services maturity, we can create a set of scorecards on top of a Service blueprint that will keep track of their progress.

๐Ÿ’ก Scorecard use casesโ€‹

Scorecards can be used to evaluate the maturity, producton readiness and engineering quality of any entity in your software catalog, for example:

  • Does a service has an on-call defined?
  • Does a README.md file exist in the repository?
  • Is Grafana defined for the K8s cluster?
  • etc.

In this live demo example, you can see the scorecards defined on a service and their evaluation. ๐ŸŽฌ

Scorecard structure tableโ€‹

A single scorecard defines a category to group different checks, validations and evaluations. Here is the structure of a single scorecard:

FieldTypeDescription
titleStringScorecard name that will be shown in the UI
identifierStringThe unique identifier of the Scorecard. The identifier is used for API calls, programmatic access and distinguishing between different scorecards
filterObjectOptional set of conditions to filter entities that will be evaluated by the scorecard
rulesObjectThe rules that we create for each scorecard to determine it's level

A scorecard contains and groups multiple rules that are relevant to its specific category, for example a scorecard for service maturity can contain 3 rules, while the production readiness scorecard can contain 2 completely different rules.

Rule elementsโ€‹

Rules enable you to generate checks inside a scorecard only for entities and properties.

A scorecard rule is a single evaluation consisting of multiple checks, each rule has a level which directly translates to how important it is for the check to pass (the more basic the check, the lower its level):

FieldTypeDescription
titleStringRule name that will be shown in the UI
descriptionStringDescription that will be shown in the UI when the rule is expanded. Value that contains markdown is also supported and will be displayed in a markdown format
identifierStringThe unique identifier of the Rule
levelStringone of Gold Silver Bronze
queryObjectThe query is built from an array of conditions and a combinator (or / and) that will define the

Combinatorโ€‹

There are two available combinators: and/or:

  • and - will apply a logical AND operation between all rules, requiring all of them to satisfy for a given asset in order to return it;
  • or - will apply a logical OR operation between all rules, requiring at least one of them to satisfy for a given asset in order to return it.
single rule queries

If you only have a single rule in your query, the combinator has no effect. But keep in mind that it still needs to be included to adhere to the query structure.

Single rule query example

In the following example, only a single rule appears in the rules array, so the combinator field has no effect:

{
"combinator": "and",
"rules": [
{
"property": "$blueprint",
"operator": "=",
"value": "myBlueprint"
}
]
}
{
"combinator": "and",
"conditions": [
{
"operator": "isNotEmpty",
"property": "on_call"
},
{
"operator": "<",
"property": "open_incidents",
"value": 5
}
]
}

Conditionsโ€‹

Conditions are small boolean checks that help when determining the final status of a query according to the specified combinator:

FieldDescription
operatorSearch operator to use when evaluating this rule, for example =, !=, contains, doesNotContains, isEmpty, isNotEmpty below
propertyProperty to filter by according to its value. It can be a meta-property such as $identifier, or any other standard entity property such as slack_channel including Mirror Properties and Calculation Properties
valueValue to compare to (not required in isEmpty and isNotEmpty operators)

Available operatorsโ€‹

OperatorSupported TypesDescription
=String, Number, Booleanchecks if the rule value is equal to the entity value
!=String, Number, Booleanchecks if the rule value is not equal to the entity value
<=String, Numberchecks if the rule value is less than or equal to the entity value
>=String, Numberchecks if the rule value is greater than or equal to the entity value
<String, Numberchecks if the rule value is less than the entity value
>String, Numberchecks if the rule value is greater than the entity value
containsString, Numberchecks if the rule value is contained within the entity value
doesNotContainsString, Numberchecks if the rule value is not contained within the entity value
endsWithString, Numberchecks if the rule value ends with the entity value
doesNotEndsWithString, Numberchecks if the rule value does not end with the entity value
beginsWithString, Numberchecks if the rule value begins with the entity value
doesNotBeginWithString, Numberchecks if the rule value does not begin with the entity value
isEmptyString, Number, Boolean, Array, Objectchecks if the rule value is an empty string, array, or object
isNotEmptyString, Number, Boolean, Array, Objectchecks if the rule value is not an empty string, array, or object

Scorecard total level calculationโ€‹

A Scorecard is built from several rules, and each one of them has a level property.

The available Scorecard levels are:

Basic -> Bronze -> Silver -> Gold

An entity always starts at the Basic level.

A rule's lowest possible level is Bronze.

Once an entity passes all of the rules for a certain level, its level changes accordingly, for example:

  1. An entity starts at level Basic;
  2. It has two rules with level Bronze;
  3. Once the entity passes those two rules, its level would be Bronze;
  4. It has four rules with level Silver;
  5. Once the entity passes those four rules (and the rules from Bronze level), its level would be Silver;
  6. etc.
multiple rules scenario

In the example listed above, let's assume the entity passes just one of the two Bronze rules, but it passes all of the Silver rules. The level of the scorecard will still be Basic, because not all Bronze rules have been satisfied.

Filter elementsโ€‹

Filters enable you to apply scorecard checks only for the entities and properties that you really care about.

Filters follow the same querying structure as rules.

A scorecard filter is used to make sure only relevant entities are evaluated, only entities that the filter evaluates to true on will have the specified rule checked:

FieldDescription
combinatorDefines the logical operation to apply to the query rules
conditionsAn array of boolean conditions to filter entities with

Scorecard exampleโ€‹

Please see the following example of an ownership scorecard.

It has one filter:

  1. Only evaluate entities that are related to production (indicated by checking that the is_production property is set to true).

It has two rules:

  1. Check that a defined on-call exists and that the number of open_incidents is lower than 5;
  2. Check if a team exists.
[
{
"title": "Ownership",
"identifier": "ownership",
"filter": {
"combinator": "and",
"conditions": [
{
"property": "is_production",
"operator": "=",
"value": true
}
]
},
"rules": [
{
"title": "Has on call?",
"identifier": "has_on_call",
"level": "Gold",
"query": {
"combinator": "and",
"conditions": [
{
"operator": "isNotEmpty",
"property": "on_call"
},
{
"operator": "<",
"property": "open_incidents",
"value": 5
}
]
}
},
{
"title": "Has a team?",
"identifier": "has_team",
"level": "Silver",
"query": {
"combinator": "and",
"conditions": [
{
"operator": "isNotEmpty",
"property": "$team"
}
]
}
}
]
}
]

Scorecard UI indicationsโ€‹

After configuring scorecards for the blueprint, entities matching the defined rules and filters will be evaluated and their scorecards will be displayed in the specific entity page:

Developer Portal Scorecards Tab

Next stepsโ€‹

Explore How to Create, Edit, and Delete Scorecards with basic examples

Dive into advanced operations on Scorecards with our API โžก๏ธ