Use dynamic YAML in pipelines

Hi @Oliver

Thank you for reaching out to Microsoft Q&A.

Your expectation of having a single Branch Validation Policy that dynamically picks and runs the YAML pipeline defined in the repository where the Pull Request was created is logical, but unfortunately it does not work and is not supported due to Azure DevOps YAML pipeline design limitations. In Azure DevOps, branch policies require a statically defined pipeline, and the pipeline YAML must exist and be registered ahead of time in the Pipelines section. The pipeline compiler resolves the pipeline YAML and repository resources at compile time, which means it cannot dynamically load or substitute the pipeline definition from the PR source repository. As a result, Azure DevOps does not support scenarios where a branch policy pipeline dynamically executes the pipeline YAML from whichever repository triggered the PR, forcing many teams into maintaining multiple pipelines unless a workaround pattern is used.

Refer below points to resolve this issue or use this workaround:

Use a single PR validation pipeline that always runs in the context of the PR repository

Create one central PR validation pipeline and configure it as a Branch Validation Policy for all repositories. This pipeline must always check out self, which automatically resolves to the repository where the PR originates.

trigger: none
 
pr:
  branches:
    include:
      - "*"
steps:
- checkout: self
- template: /.azuredevops/pr-validation.yml

Store repo-specific logic as templates inside each repository

Instead of dynamic pipelines, move variability into repo-local templates. Each repository defines its own PR logic in a known location, for example:

/.azuredevops/pr-validation.yml

Each repository controls its own build, test, scan, or validation steps:

steps:
- script: echo "Running PR validation for this repository"
- template: templates/build.yml
- template: templates/test.yml

Enforce a convention instead of managing 100 pipelines

Adopt a simple convention that every repository must contain a PR validation template at a fixed path. With this approach:

• You maintain only one pipeline
• You apply only one branch policy
• Onboarding a new repo requires no pipeline creation, only adding the template file

This is the most scalable and maintainable design and aligns with how Azure DevOps YAML pipelines are intended to be used at enterprise scale.

Azure DevOps does not support a single YAML pipeline that dynamically reads and runs arbitrary YAML files from 100 different repositories at PR time. Each YAML pipeline is bound to a specific repository and YAML path, and templates from other repositories are only used at compile time, not as independent pipelines.

Key constraints from the platform:

1. Templates from other repositories are not separate pipelines
   • Repository resources can be declared and templates referenced across repos:

          resources:
            repositories:
              - repository: templates
                type: git
                name: MyProject/BuildTemplates
          
          jobs:
          - template: common.yml@templates
     

   • Only template files are used; once expanded, the final pipeline runs as if defined entirely in the source repo. Scripts or logic cannot be executed as if they were separate pipelines from those other repos.
   • Repositories are resolved once at pipeline start; there is no mechanism to dynamically “switch” to another repo’s YAML as the pipeline definition at runtime.
2. Triggers and branch policies are repo‑scoped
   • For multi‑repo checkout, each repository resource can have its own triggers, but the pipeline definition still lives in a single repo:

          resources:
            repositories:
            - repository: A
              type: git
              name: MyProject/A
              ref: main
              trigger:
              - main
          
            - repository: B
              type: git
              name: MyProject/B
              ref: release
              trigger:
              - main
              - release
     

   • PR validation is configured per repository via branch policies. To validate PRs in each of 100 repos, each repo needs a build validation policy pointing to a pipeline.

Given these constraints, the desired model of “one pipeline that discovers and runs each repo’s own YAML” cannot be implemented natively.

Practical options within the platform model:

1. Single shared template, many minimal pipelines
   • Create one core template repo with the actual build logic (jobs, steps) in templates. Example:

          # Core template repo: BuildTemplates/common.yml
          parameters:
          - name: vmImage
            type: string
            default: 'ubuntu-22.04'
          
          jobs:
          - job: Build
            pool:
              vmImage: ${{ parameters.vmImage }}
            steps:
            - script: npm install
            - script: npm test
     

   • In each of the 100 repos, define a tiny azure-pipelines.yml that only references the shared template and passes repo‑specific parameters:

          resources:
            repositories:
              - repository: templates
                type: git
                name: Contoso/BuildTemplates
          
          pr:
          - main
          - feature/*
          
          jobs:
          - template: common.yml@templates
            parameters:
              vmImage: 'ubuntu-22.04'
     

   • Each repo then has its own pipeline (and PR build policy), but maintenance is centralized in the template repo. Changes to build logic are made once in the template.
2. Multi‑repo pipeline with branch policies (still not fully dynamic)
   • A single pipeline can check out multiple repositories and run generic validation logic, but it cannot interpret each repo’s own YAML as a pipeline. It can only run scripted logic (e.g., build/test commands) against each checked‑out repo.
   • To trigger on PRs in other repos, declare them as repository resources and configure branch policies in those repos so that PRs trigger the shared pipeline.
3. Security and secrets via templates
   • For shared logic that uses secrets, store scripts/templates in a secure repo and reference them as templates, passing secrets via variable groups or Key Vault, as recommended for secure template usage.

Within the documented capabilities, the closest to the requested behavior is: one shared template repo + 100 minimal YAML pipelines (one per repo) that all reference the shared templates and are wired via PR branch policies. The platform does not provide a way to avoid having per‑repo pipeline definitions if each repo must have its own PR validation.

References:

• Use YAML templates in pipelines for reusable and secure processes (templates-includes)
• Use YAML templates in pipelines for reusable and secure processes (templates-extends)
• Check out multiple repositories in your pipeline
• Build multiple branches in Azure Pipelines
• Secrets in Azure Pipelines