Skip to main content

GitLab

Port's GitLab integration allows you to model GitLab resources in your software catalog and ingest data into them.

Overview​

This integration allows you to:

  • Map and organize your desired GitLab resources and their metadata in Port (see supported resources below).
  • Watch for GitLab object changes (create/update/delete) in real-time, and automatically apply the changes to your software catalog.
  • Manage Port entities using GitOps.

Supported Resources​

The resources that can be ingested from GitLab into Port are listed below.
It is possible to reference any field that appears in the API responses linked below in the mapping configuration.

Setup​

To install Port's GitLab integration, follow the installation guide.

Permission

Port's GitLab integration requires a group access token with the api scope. To create a group access token, follow the instructions in the installation guide

Configuration​

Port integrations use a YAML mapping block to ingest data from the third-party api into Port.

The mapping makes use of the JQ JSON processor to select, modify, concatenate, transform and perform other operations on existing fields and values from the integration API.

Capabilities​

Ingest files from your repositories​

Port allows you to fetch JSON and YAML files from your repositories, and create entities from them in your software catalog.
This is done using the file kind in your GitLab mapping configuration.

For example, say you want to manage your package.json files in Port. One option is to create a manifest blueprint, with each of its entities representing a package.json file.

Search Type

The file kind follows GitLab's Advanced Search type, adhering to its syntax, limitations, and capabilities.

The following configuration fetches all package.json files from "MyRepo" and "MyOtherRepo", and creates an entity for each of them, based on the manifest blueprint:

resources:
  - kind: file
    selector:
      query: 'true'
      files:
        # Note that glob patterns are supported, so you can use wildcards to match multiple files
        path: '**/package.json'
        # The `repos` key can be used to filter the repositories from which the files will be fetched. 
        # Remove the `repos` key if you want to fetch files from all repository
        repos:
          # The repository NAME should be used here, not the URL slug, e.g. "Repo Name" not "repo-name"
          - "MyRepoName"
          - "MyOtherRepoName"
    port:
      entity:
        mappings:
          identifier: .file.file_path
          title: .file.file_name
          blueprint: '"manifest"'
          properties:
            project_name: .file.content.name
            project_version: .file.content.version
            license: .file.content.license
Test your mapping

After adding the file kind to your mapping configuration, click on the Resync button. When you open the mapping configuration again, you will see real examples of files fetched from your GitLab organization.

This will help you see what data is available to use in your jq expressions.
Click on the Test mapping button to test your mapping against the example data.

In any case, the structure of the available data looks like this:

Available data example (click to expand)
{
  "file": {
    "file_name": "package.json",
    "file_path": "package.json",
    "size": 780,
    "encoding": "base64",
    "content_sha256": "d4dffc856dcacdaecb283ba9e47288beb6036d7c4ffff7c65f29057d890ecee9",
    "ref": "main",
    "blob_id": "7f8c2fea237a5cf0e1bcc17135c7c8b9e96edd49",
    "commit_id": "3ac75a99f6faa8ce4570368e6db1038c15f17cfc",
    "last_commit_id": "3ac75a99f6faa8ce4570368e6db1038c15f17cfc",
    "execute_filemode": false,
    "content": {
      "name": "my-awesome-project",
      "version": "1.0.0",
      "description": "A sample Node.js project",
      "main": "index.js",
      "scripts": {
        "start": "node index.js",
        "test": "echo \"Error: no test specified\" && exit 1",
        "build": "echo \"Building the project...\"",
        "lint": "eslint ."
      },
      "repository": {
        "type": "git",
        "url": "git+https://github.com/username/my-awesome-project.git"
      },
      "keywords": [
        "sample",
        "nodejs",
        "project"
      ],
      "author": "Your Name",
      "license": "ISC",
      "bugs": {
        "url": "https://github.com/username/my-awesome-project/issues"
      },
      "homepage": "https://github.com/username/my-awesome-project#readme",
      "dependencies": {
        "@docsly/react": "^1.9.1",
        "@docusaurus/core": "^3.4.0",
        "@docusaurus/plugin-client-redirects": "^3.4.0",
        "@docusaurus/plugin-google-tag-manager": "^3.4.0",
        "@docusaurus/plugin-ideal-image": "^3.4.0",
        "@docusaurus/preset-classic": "^3.4.0",
        "@docusaurus/theme-live-codeblock": "^3.4.0",
        "@easyops-cn/docusaurus-search-local": "^0.44.2",
        "@mdx-js/react": "^3.0.1",
        "@port-labs/docusaurus-plugin-openapi-docs": "^0.0.5",
        "@port-labs/docusaurus-theme-openapi-docs": "^0.0.5",
        "@stackql/docusaurus-plugin-hubspot": "^1.0.1",
        "clsx": "^2.1.1",
        "docusaurus-plugin-hotjar": "^0.0.2",
        "docusaurus-plugin-image-zoom": "^2.0.0",
        "prettier": "^3.3.2",
        "prism-react-renderer": "^2.3.1",
        "react": "^18.3.1",
        "react-dom": "^18.3.1",
        "react-tooltip": "^5.27.1"
      },
      "devDependencies": {
        "@docusaurus/module-type-aliases": "^3.4.0",
        "@docusaurus/tsconfig": "^3.4.0",
        "@docusaurus/types": "^3.4.0",
        "husky": "^9.0.11",
        "pretty-quick": "^4.0.0",
        "typescript": "~5.5.3"
      }
    }
  },
  "repo": {
    "id": 60625101,
    "description": null,
    "name": "pages-test-project",
    "name_with_namespace": "port-labs / pages-test-project",
    "path": "pages-test-project",
    "path_with_namespace": "port-org/pages-test-project",
    "created_at": "2024-08-06T10:31:47.509Z",
    "default_branch": "main",
    "tag_list": [],
    "topics": [],
    "ssh_url_to_repo": "git@gitlab.com:port-org/pages-test-project.git",
    "http_url_to_repo": "https://gitlab.com/port-org/pages-test-project.git",
    "web_url": "https://gitlab.com/port-org/pages-test-project",
    "readme_url": "https://gitlab.com/port-org/pages-test-project/-/blob/main/README.md",
    "forks_count": 0,
    "avatar_url": null,
    "star_count": 0,
    "last_activity_at": "2024-08-14T15:40:56.606Z",
    "namespace": {
      "id": 66136652,
      "name": "port-labs",
      "path": "port-org",
      "kind": "group",
      "full_path": "port-org",
      "parent_id": null,
      "avatar_url": null,
      "web_url": "https://gitlab.com/groups/port-org"
    },
    "container_registry_image_prefix": "registry.gitlab.com/port-org/pages-test-project",
    "_links": {
      "self": "https://gitlab.com/api/v4/projects/60625101",
      "issues": "https://gitlab.com/api/v4/projects/60625101/issues",
      "merge_requests": "https://gitlab.com/api/v4/projects/60625101/merge_requests",
      "repo_branches": "https://gitlab.com/api/v4/projects/60625101/repository/branches",
      "labels": "https://gitlab.com/api/v4/projects/60625101/labels",
      "events": "https://gitlab.com/api/v4/projects/60625101/events",
      "members": "https://gitlab.com/api/v4/projects/60625101/members",
      "cluster_agents": "https://gitlab.com/api/v4/projects/60625101/cluster_agents"
    },
    "packages_enabled": true,
    "empty_repo": false,
    "archived": false,
    "visibility": "private",
    "resolve_outdated_diff_discussions": false,
    "container_expiration_policy": {
      "cadence": "1d",
      "enabled": false,
      "keep_n": 10,
      "older_than": "90d",
      "name_regex": ".*",
      "name_regex_keep": null,
      "next_run_at": "2024-08-07T10:31:47.544Z"
    },
    "repository_object_format": "sha1",
    "issues_enabled": true,
    "merge_requests_enabled": true,
    "wiki_enabled": true,
    "jobs_enabled": true,
    "snippets_enabled": true,
    "container_registry_enabled": true,
    "service_desk_enabled": true,
    "service_desk_address": "contact-project-pages-test-project-60625101-issue-@incoming.gitlab.com",
    "can_create_merge_request_in": true,
    "issues_access_level": "enabled",
    "repository_access_level": "enabled",
    "merge_requests_access_level": "enabled",
    "forking_access_level": "enabled",
    "wiki_access_level": "enabled",
    "builds_access_level": "enabled",
    "snippets_access_level": "enabled",
    "pages_access_level": "private",
    "analytics_access_level": "enabled",
    "container_registry_access_level": "enabled",
    "security_and_compliance_access_level": "private",
    "releases_access_level": "enabled",
    "environments_access_level": "enabled",
    "feature_flags_access_level": "enabled",
    "infrastructure_access_level": "enabled",
    "monitor_access_level": "enabled",
    "model_experiments_access_level": "enabled",
    "model_registry_access_level": "enabled",
    "emails_disabled": false,
    "emails_enabled": true,
    "shared_runners_enabled": true,
    "lfs_enabled": true,
    "creator_id": 6152768,
    "import_url": null,
    "import_type": null,
    "import_status": "none",
    "open_issues_count": 0,
    "description_html": "",
    "updated_at": "2024-08-14T15:40:56.606Z",
    "ci_default_git_depth": 20,
    "ci_forward_deployment_enabled": true,
    "ci_forward_deployment_rollback_allowed": true,
    "ci_job_token_scope_enabled": false,
    "ci_separated_caches": true,
    "ci_allow_fork_pipelines_to_run_in_parent_project": true,
    "ci_id_token_sub_claim_components": [
      "project_path",
      "ref_type",
      "ref"
    ],
    "build_git_strategy": "fetch",
    "keep_latest_artifact": true,
    "restrict_user_defined_variables": false,
    "ci_pipeline_variables_minimum_override_role": "maintainer",
    "runners_token": null,
    "runner_token_expiration_interval": null,
    "group_runners_enabled": true,
    "auto_cancel_pending_pipelines": "enabled",
    "build_timeout": 3600,
    "auto_devops_enabled": false,
    "auto_devops_deploy_strategy": "continuous",
    "ci_push_repository_for_job_token_allowed": false,
    "ci_config_path": "",
    "public_jobs": true,
    "shared_with_groups": [],
    "only_allow_merge_if_pipeline_succeeds": false,
    "allow_merge_on_skipped_pipeline": null,
    "request_access_enabled": true,
    "only_allow_merge_if_all_discussions_are_resolved": false,
    "remove_source_branch_after_merge": true,
    "printing_merge_request_link_enabled": true,
    "merge_method": "merge",
    "squash_option": "default_off",
    "enforce_auth_checks_on_uploads": true,
    "suggestion_commit_message": null,
    "merge_commit_template": null,
    "squash_commit_template": null,
    "issue_branch_template": null,
    "warn_about_potentially_unwanted_characters": true,
    "autoclose_referenced_issues": true,
    "approvals_before_merge": 0,
    "mirror": false,
    "external_authorization_classification_label": "",
    "marked_for_deletion_at": null,
    "marked_for_deletion_on": null,
    "requirements_enabled": true,
    "requirements_access_level": "enabled",
    "security_and_compliance_enabled": true,
    "pre_receive_secret_detection_enabled": false,
    "compliance_frameworks": [],
    "issues_template": null,
    "merge_requests_template": null,
    "ci_restrict_pipeline_cancellation_role": "developer",
    "merge_pipelines_enabled": false,
    "merge_trains_enabled": false,
    "merge_trains_skip_train_allowed": false,
    "only_allow_merge_if_all_status_checks_passed": false,
    "allow_pipeline_trigger_approve_deployment": false,
    "prevent_merge_without_jira_issue": false,
    "permissions": {
      "project_access": null,
      "group_access": {
        "access_level": 50,
        "notification_level": 3
      }
    }
  }
}

Create multiple entities from a single file​

In some cases, we would like to parse a single JSON/YAML file and create multiple entities from it.
For this purpose, we can use the itemsToParse key in our mapping configuration.

For example, say you want to track/manage a project's dependencies in Port. One option is to create a package blueprint, with each of its entities representing a dependency from a package.json file.

The following configuration fetches a package.json file from a specific repository, and creates an entity for each of the dependencies in the file, based on the package blueprint:

resources:
  - kind: file
    selector:
      query: 'true'
      files:
        path: '**/package.json'
        # Note that in this case we are fetching from a specific repository
        repos:
          - MyRepoName
    port:
      itemsToParse: .file.content.dependencies | to_entries
      entity:
        mappings:
          # Since identifier cannot contain special characters, we are using jq to remove them
          identifier: >-
            .item.key + "_" + if (.item.value | startswith("^")) then
            .item.value[1:] else .item.value end
          title: .item.key + "@" + .item.value
          blueprint: '"package"'
          properties:
            package: .item.key
            version: .item.value
          relations: {}

The itemsToParse key is used to specify the path to the array of items you want to parse from the file. In this case, we are parsing the dependencies array from the package.json file.
Once the array is parsed, we can use the item key to refer to each item in the array.

Multi-document YAML files​

For multi-document YAML files (a single file containing multiple YAML documents separated by ---), .file.content will not resolve to an object, but to an array of objects.

You can use one of these methods to ingest multi-document YAML files:

  1. Use the itemsToParse key to create multiple entities from such a file (see example above).
  2. Map the result to an array property.
Mixed YAML types

If you have both single-document and multi-document YAML files in your repositories, you can use the itemsToParse key like this to handle both cases:

itemsToParse: .file.content | if type== "object" then [.] else . end

Limitations​

  • Currently only files up to 1MB in size are supported.
  • Only JSON and YAML formats are automatically parsed. Other file formats can be ingested as raw files.
  • GLOB patterns are supported for file pattern matching, but wildcards at the end (e.g., **/*) are not allowed, in order to prevent matching all files indiscriminately.
  • Currently only the default branch of the repository is supported.

For a list of known limitations with GitLab’s Advanced Search, see GitLab's Advanced Search documentation.

Examples​

Refer to the examples page for practical configurations and their corresponding blueprint definitions.

Relevant Guides​

For relevant guides and examples, see the guides section.

GitOps​

Port's GitLab integration also provides GitOps capabilities, refer to the GitOps page to learn more.

Advanced​

Refer to the advanced page for advanced use cases and examples.