Automatize a verificação de licenças OSS

Let's Get to Work: The Main Job

1. Fetch the Source Code and License Config

Your job starts with fetching your repo and a separate repo containing your license configuration. Here's the code:

jobs:
  deps-lic-check:
    runs-on: ubuntu-latest
    steps:
      - name: Generate token
        id: generate_token
        uses: tibdex/github-app-token@v1
        with:
          app_id: ${{ secrets.APP_ID }}
          private_key: ${{ secrets.PRIVATE_KEY }}

      - name: Checkout current repo
        uses: actions/checkout@v4
        with:
          ref: ${{ env.CHECKOUT_REF }}
          fetch-depth: 1

      - name: Checkout configuration
        uses: actions/checkout@v3
        with:
          repository: perfectscale-io/gha-store # replace to your org/repo
          ref: v1
          token: ${{ steps.generate_token.outputs.token  }}
          path: gha-store
          fetch-depth: 1

Regarding "Checkout current repo" everything is simple, but pay attention to "ref". We use the to ensure that the PR will get Commit from PR because the default behavior on PR is to checkout ephemeral commits of merging PR to the target branch.

To access configuration, we need to checkout another repo (inside our org). To do so, we created a Github Application with a specific set of permissions.

So, as result, we have the next working directory structure:

pkg/
main.go
go.mod
go.sum
gha-store/

2. Setting Up Go and Vendor

If you're dealing with a Golang repo, you'll need to set up Go and Govendor like so:

      - uses: actions/setup-go@v4
        with:
          go-version: 'stable'

      - name: Setup vendor
        run: |
          go env -w GOPRIVATE="github.com/perfectscale-io" # replace to your org
          git config --global url." "
        env:
          TOKEN: x-access-token:${{steps.generate_token.outputs.token}}

Here we setup “go” and again reuse the token from the "Generate token" step.
If you don't have private dependencies, you can omit the "Setup vendor" step.

3. Generate Software Bill of Materials (SBOM)

Why SBOM? It makes it easier to list all your dependencies.
To do it with Golang we will use "CycloneDX/gh-gomod-generate-sbom@v2" from CycloneDX.

Note, we used OWASP CycloneDX as it is a full-stack Bill of Materials (BOM) standard that provides advanced supply chain capabilities for cyber risk reduction. It is managed by the CycloneDX Core Working Group, is backed by the OWASP Foundation

As an output, it will generate a JSON file with all our dependencies.
Take a look:

      - name: Generate SBOM
        uses: CycloneDX/gh-gomod-generate-sbom@v2
        with:
          version: v1
          args: app -licenses=true -assert-licenses=true -json=true -output sbom.json .

By using SBOM as a standard to get dependencies information, we can avoid further refactoring to support multiple languages and make our code simple.

The only thing left is to parse this SBOM and compare dependencies licenses with the allowed list.
To do it, we will run a simple Python script which we will run directly in GitHub actions step using "shell: python"

But before we start, let's look at our "allowed-licenses.yaml" (or "configuration") file:

---
# array of allowed licenses
allowed:
  - MIT
  - Apache-2.0
# array of package names to ignore during check
ignore:
  - github.com/golang/protobuf

Here we have 2 arrays:

allowed - array ([]string) of allowed licenses
ignore - array ([]string) of packages which we ignore and allow any license for them

In this configuration file we store in our gha-store repo, folder "helpers" and are able to update it as often as we want. Every new workflow we run will take an updated configuration.

4. Python Magic: Analyzing Licenses

Now, the real fun begins! We've written a Python script that does all the heavy lifting.

      - name: Analyze licenses
        shell: python
        run: |
          import sys
          import json
          import yaml
          def generate_markdown_table(non_allowed_licenses):
              table = "| License ID | Packages |\n"
              table += "|------------|----------|\n"
              for license_id, packages in non_allowed_licenses.items():
                  table += f"| {license_id} | {', '.join(packages)} |\n"
              return table
          def extract_licenses(json_data):
              license_package_mapping = {}
              for component in json_data['components']:
                  package_name = component['name']
                  licenses = component.get('licenses', [])
                  for lic in licenses:
                      license_id = lic['license']['id']
                      license_package_mapping.setdefault(
                          license_id, []).append(package_name)
              return license_package_mapping
          def check_licenses(license_package_mapping, allowed_licenses, ignored_packages):
              non_allowed_licenses = {}
              for license_id, packages in license_package_mapping.items():
                  if license_id not in allowed_licenses:
                      non_ignored_packages = [
                          pkg for pkg in packages if pkg not in ignored_packages]
                      if non_ignored_packages:
                          non_allowed_licenses[license_id] = non_ignored_packages
              return non_allowed_licenses
          json_data = json.load(open('sbom.json'))
          with open('gha-store/helpers/allowed-licenses.yaml', 'r') as f:
              yaml_data = yaml.safe_load(f)
          extracted_licenses = extract_licenses(json_data)
          allowed_licenses = yaml_data['allowed']
          ignored_packages = yaml_data['ignore']
          non_allowed_licenses = check_licenses(
              extracted_licenses, allowed_licenses, ignored_packages)
          if non_allowed_licenses:
              print("Non-allowed licenses and their packages:")
              for license_id, packages in non_allowed_licenses.items():
                  print(f"License: {license_id}, Packages: {', '.join(packages)}")
              markdown_table = generate_markdown_table(non_allowed_licenses)
              with open("result.md", "w") as f:
                  f.write("# Non-Allowed Licenses and Their Packages\n")
                  f.write(markdown_table)
              sys.exit(1)
          else:
              with open("result.md", "w") as f:
                  f.write("#No problems with Licenses\n")
              print("All licenses are allowed.")
              sys.exit(0)

Here we compare dependencies licenses from SBOM with the allowed licenses and log everything that doesn't have the proper license and is not ignored.
Also, we generate a markdown table with the "failed" packages.

5. Keeping It User-Friendly: GitHub Actions Summary

We want the output to be as developer-friendly as possible, so we use GitHub Actions Summary:

      - name: Update summary
        if: always()
        run: echo -e "$(cat result.md)" >> $GITHUB_STEP_SUMMARY

No magic, just GitHub Actions.

6. Don't Let It Slide: Slack Notifications

If anything goes south, you'll get a Slack notification. Trust me, you want to set this up!

      - name: Notify about deploy
        if: failure()
        uses: slackapi/slack-github-action@v1.23.0
        with:
          payload: |
            {"text": "License issue in ${{ github.event.repository.name }}\nRef: ${{ github.ref_name }}\nWorkflow url: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"}
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.GLOBAL_SLACK_DEPENDENCY_LICENSE }}
          SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK

To setup, please create a new Slack application and add it to your workspace. Then add a new WebHook and save it as a global secret, for example, "GLOBAL_SLACK_DEPENDENCY_LICENSE" as we do.
You can read more about it in Slack documentation: https://api.slack.com/messaging/webhooks

Call me ...

We finished our Reusable workflow, now we need to call it somehow:

name: PSC Golang Workflow

on:
  push:
    branches:
      - 'main'
  pull_request:
  workflow_dispatch:
    inputs:
      ref:
        type: string
        description: 'Optional: Commit, branch etc.'
        required: false

permissions:
  checks: write
  id-token: write

jobs:
  deps-lic-check:
    uses: perfectscale-io/gha-store/.github/workflows/_deps_lic_check.yaml@v2
    with:
      inherit-inputs: ${{ toJSON(inputs) }}
    secrets: inherit

As promised example of how to pass parent's inputs to child workflow. We just call this Reusable workflow to check our licenses.

Bringing It All Together

And there you have it, folks! Automating license checks is easier than you thought. With just a bit of Python and GitHub Actions, you're all set to ensure you're not stepping on any legal landmines.
So, go ahead, give it a try, and let us know how it goes. Happy coding! 🎉

Full reusable workflow:

on:
  workflow_call:
    inputs:
      inherit-inputs:
        required: false
        type: string

env:
  CHECKOUT_REF: ${{ fromJSON(inputs.inherit-inputs).ref || github.event.pull_request.head.sha }}

jobs:
  deps-lic-check:
    runs-on: ubuntu-latest
    steps:
      - name: Generate token
        id: generate_token
        uses: tibdex/github-app-token@v1
        with:
          app_id: ${{ secrets.APP_ID }}
          private_key: ${{ secrets.PRIVATE_KEY }}

      - uses: actions/checkout@v4
        with:
          ref: ${{ env.CHECKOUT_REF }}
          fetch-depth: 1

      - uses: actions/checkout@v3
        with:
          repository: perfectscale-io/gha-store
          ref: v2
          token: ${{ steps.generate_token.outputs.token  }}
          path: gha-store
          fetch-depth: 1

      - uses: actions/setup-go@v4
        with:
          go-version: 'stable'

      - name: Setup vendor
        run: |
          go env -w GOPRIVATE="github.com/perfectscale-io"
          git config --global url." "
        env:
          TOKEN: x-access-token:${{steps.generate_token.outputs.token}}

      - name: Generate SBOM
        uses: CycloneDX/gh-gomod-generate-sbom@v2
        with:
          version: v1
          args: app -licenses=true -assert-licenses=true -json=true -output sbom.json .

      - name: Analyze licenses
        shell: python
        run: |
          import sys
          import json
          import yaml
          def generate_markdown_table(non_allowed_licenses):
              table = "| License ID | Packages |\n"
              table += "|------------|----------|\n"
              for license_id, packages in non_allowed_licenses.items():
                  table += f"| {license_id} | {', '.join(packages)} |\n"
              return table
          def extract_licenses(json_data):
              license_package_mapping = {}
              for component in json_data['components']:
                  package_name = component['name']
                  licenses = component.get('licenses', [])
                  for lic in licenses:
                      license_id = lic['license']['id']
                      license_package_mapping.setdefault(
                          license_id, []).append(package_name)
              return license_package_mapping
          def check_licenses(license_package_mapping, allowed_licenses, ignored_packages):
              non_allowed_licenses = {}
              for license_id, packages in license_package_mapping.items():
                  if license_id not in allowed_licenses:
                      non_ignored_packages = [
                          pkg for pkg in packages if pkg not in ignored_packages]
                      if non_ignored_packages:
                          non_allowed_licenses[license_id] = non_ignored_packages
              return non_allowed_licenses
          json_data = json.load(open('sbom.json'))
          # yaml_data = yaml.load(open('allowed-licenses.yaml'))
          with open('gha-store/helpers/allowed-licenses.yaml', 'r') as f:
              yaml_data = yaml.safe_load(f)
          extracted_licenses = extract_licenses(json_data)
          allowed_licenses = yaml_data['allowed']
          ignored_packages = yaml_data['ignore']
          non_allowed_licenses = check_licenses(
              extracted_licenses, allowed_licenses, ignored_packages)
          if non_allowed_licenses:
              print("Non-allowed licenses and their packages:")
              for license_id, packages in non_allowed_licenses.items():
                  print(f"License: {license_id}, Packages: {', '.join(packages)}")
              markdown_table = generate_markdown_table(non_allowed_licenses)
              with open("result.md", "w") as f:
                  f.write("# Non-Allowed Licenses and Their Packages\n")
                  f.write(markdown_table)
              sys.exit(1)
          else:
              with open("result.md", "w") as f:
                  f.write("#No problems with Licenses\n")
              print("All licenses are allowed.")
              sys.exit(0)

      - name: Update summary
        if: always()
        run: echo -e "$(cat result.md)" >> $GITHUB_STEP_SUMMARY

      - name: Notify about deploy
        if: failure()
        uses: slackapi/slack-github-action@v1.23.0
        with:
          payload: |
            {"text": "License issue in ${{ github.event.repository.name }}\nRef: ${{ github.ref_name }}\nWorkflow url: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"}
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.GLOBAL_SLACK_DEPENDENCY_LICENSE }}
          SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK

Common Open source software (OSS) libraries licenses

Open source software (OSS) libraries typically use a variety of licenses, but some are more common than others. Always read and understand the terms of a license before using or contributing to a project.

License	Type	Permissive/ Restrictive	Usage Examples	Obligations
MIT License	Permissive	Permissive	Commercial and non-commercial projects, proprietary software.	Include the original license and copyright notice in any copy or substantial portion of the software.
GNU GPL (v2 or v3)	Copyleft	Restrictive	Free and open source projects, may impact commercial use and proprietary software.	Distribute derivative works under the same license. Provide access to the source code of the software when distributing binaries.
Apache License	Permissive	Permissive	Commercial and non-commercial projects, widely used in corporate environments.	Include the original copyright, license, and notice in any copy or substantial portion of the software.
BSD License (2 or 3-Clause)	Permissive	Permissive	Commercial and non-commercial projects, allows for use in proprietary software.	Include the original copyright, license, and disclaimer in any copy or substantial portion of the software.
Creative Commons	Various	Varies	Not typically used for software, but for creative works like images, music, etc. Usage varies based on specific Creative Commons license chosen.	Varied obligations depending on the specific Creative Commons license (e.g., attribution, non-commercial use, share-alike).
Mozilla Public License (MPL)	Copyleft	Partially Restrictive	Free and open source projects, allows for proprietary derivative works. Mozilla products often use this license.	Distribute any derivative works under the MPL, provide access to the source code of derivative works, include an original copy of the MPL with any substantial portions.