Cheat notes · Module 04 · The portal and its catalog

Catalog at scale: let it find the repos

A five-minute review for when you have done this module and want it back in your head. Open the full module for the steps and the lab, or the full text to print.

The idea

One URL per repo does not scale. A discovery provider scans your GitHub account on a schedule and reads every catalog-info.yaml it finds, so adding the file is the whole onboarding, and catalog rules stop team repos from inventing teams or users.

Remember

Entity provider vs location: the catalog reads locations in its processing loop; a provider runs on its own schedule and writes entities in (GitHub discovery, org imports).
A backend module, not a plugin: plugin-catalog-backend-module-github plus one backend.add line plugs into the catalog plugin. A new backend module needs a restart.
catalog.providers.github.<id>: organization, catalogPath, filters (branch, repository), schedule. The id names the scheduled task: github-provider:<id>:refresh.
Rules: catalog.rules lists what any source may add (Component, System, API, Resource, Location). Only the platform's own file locations widen it (Group, User, Domain, Template). Anything else is refused with a warning in the log.
Orphans: an entity its source no longer mentions. The default strategy delete removes it in a cleanup every 30 seconds.
Discovery reads main: a catalog-info.yaml only counts once it is merged.

Pitfalls

A team defining its own Group in its repo: the group never appears and nothing on the entity page says why. The log says not of an allowed kind for that location. Groups go to catalog/org.yaml by pull request.
Leaving create-app's own catalog-info.yaml owned by [email protected]: discovery finds it. Make it true before you merge.

Commands

`yarn --cwd packages/backend add @backstage/plugin-catalog-backend-module-github`	Install the discovery module
`curl -s -H "Authorization: Bearer $PORTAL_API_TOKEN" -X POST localhost:7007/api/catalog/.backstage/scheduler/v1/tasks/github-provider:teamRepos:refresh/trigger`	Run the scan now
`... \| jq -r '.metadata.annotations["backstage.io/managed-by-origin-location"]'`	Where an entity now comes from

Provider and rules

catalog:
  rules:
    - allow: [Component, System, API, Resource, Location]
  providers:
    github:
      teamRepos:
        organization: YOUR_GITHUB_USER
        catalogPath: /catalog-info.yaml
        filters: { branch: main, repository: '.*' }
        schedule:
          frequency: { minutes: 30 }
          timeout: { minutes: 3 }

Test yourself

Answer in your head first, then open the answer.

What is the difference between a location and an entity provider?

The catalog reads a location; a provider runs on its own schedule and writes entities in.

You remove a location. What happens to its entities by default?

They become orphans and are deleted within about 30 seconds.

Why can a discovered team repo not add a User?

catalog.rules does not allow User for general sources; only the platform's org.yaml location does.

Where module 04 leaves you: The catalog finds checkout and portal by scanning your GitHub account. Team repos may add components, APIs and resources, but not groups or users.