Can't create Data Collection Rule association with bicep

Hello Rachel, you are getting this error when creating a Data Collection Rule Association (DCRA) from Bicep: Rule Id '/subscriptions/XXXX/resourceGroups/XXXX/providers/Microsoft.Insights/dataCollectionRules/XXXX' does not exist. You have verified the DCR exists and the id string matches, and the same Bicep template works in other subscriptions. The association deployment cannot resolve the DCR id in this failing subscription.

• The DCR must be visible from the scope where the association is created. If the association is deployed in a different subscription or wrong scope, the service cannot resolve the id and returns “does not exist”.  If the association is deployed in the same transaction or immediately after DCR creation, the resource may not yet be resolvable by the service; the deployment engine can attempt the association before the DCR is fully registered.
• If Microsoft.Insights is not registered or you use an incompatible API version, the resource type may not be recognized in that subscription.
• The deployment identity needs read access to the DCR; without it the resource appears invisible to the deployment.  Subscription differences: Subscriptions can differ in provider registration, policies, or RBAC which explains why the same template works in some subscriptions and not others.

Please review the following items on your end:

Verify the DCR exists and is accessible: Confirm the rule can be resolved using the same identity used for the deployment:

az monitor data-collection rule show \
--name <name> \
--resource-group <rg> \
--subscription <sub>

If this fails, the issue is typically RBAC or subscription‑scope related.

Check provider registration: Ensure Microsoft.Insights is registered in the failing subscription:

az provider show -n Microsoft.Insights --subscription <sub> --query registrationState

Register if needed:

az provider register -n Microsoft.Insights --subscription <sub>

Use supported API versions: Prefer stable, non‑preview API versions:

• Microsoft.Insights/dataCollectionRules@2023-03-11
• Microsoft.Insights/dataCollectionRuleAssociations@2022-06-01

Use the correct Bicep reference pattern Data Collection Rules are resource‑group scoped. Reference them correctly:

resource existingDcr 'Microsoft.Insights/dataCollectionRules@2023-03-11' existing = {
name: 'myDcrName'
scope: resourceGroup('myDcrResourceGroup')
}

If the DCR is created in the same deployment: Reference, it directly and enforce ordering with dependsOn:

resource association 'Microsoft.Insights/dataCollectionRuleAssociations@2022-06-01' = {
name: 'assoc'
properties: {
dataCollectionRuleId: dcr.id
}
dependsOn: [ dcr ]
}

Verify deployment identity permissions: The deployment identity must have at least: Microsoft.Insights/dataCollectionRules/read

Roles that satisfy this include Reader, Monitoring Contributor, and Contributor.

Without read access to the DCR resource group, the rule may appear nonexistent during validation.

Reference:

Manage data collection rule associations in Azure Monitor - Azure Monitor | Microsoft Learn 

Microsoft.Insights/dataCollectionRuleAssociations - Bicep, ARM template & Terraform AzAPI reference | Microsoft Learn

Microsoft.Insights/dataCollectionRules - Bicep, ARM template & Terraform AzAPI reference | Microsoft Learn

Hope this helps. and please feel free to reach out if you have any further questions. Thanks

Hi Bharath,

1. I can't use the same identity as the deployment to run that command because of how our deployment system works, but the identity that is trying to create the association made the relevant data collection rule so it has enough access. I confirmed that the data collection rule is in the the same subscription and resource group as the bicep deployment. It is in a different region than the aks cluster that the rule is being associated with, but it needs to be. The region we need our aks cluster to be in doesn't support Azure Monitor. I confirmed that this pattern worked in our test subscription where we have a data collection rule in eastus associated with an aks cluster in eastus2. The same bicep templates were used to provision these resources in our test subscription.
2. Provider registration confirmed
3. The bicep is using that pattern except it does not have the dependsOn. I would have expected that to explain the failure the first time this was run if this were a race condition while the DCR was being provisioned, but I ran this template several hours after the DCR was provisioned, so at that point, it did exist, and the correct resource id was being passed into the rule association object.

Thanks for the update,

Other possible primary resolution: For AKS in a subscription where cross-region DCR associations fail:

az aks enable-addons \   
--addons monitoring \   
--resource-group <rg> \   
--name <aksName>

Running this initializes the Azure Monitor backend metadata for AKS in the subscription. Before this, the ARM engine cannot fully resolve a DCR association for AKS because the backend doesn’t yet see AKS + Monitor connections. After initialization, the DCR association resolves correctly, without changing your Bicep templates.

To confirm the problem is AKS-specific:

1. Associate the same DCR with a VM or Arc-enabled server in the same subscription.
2. If it succeeds while AKS still fails, the issue is not the DCR or IAM, but the AKS Monitor backend initialization.

This helps isolate the problem from other potential causes like identity, permissions, or DCR visibility.

If you are using a Data Collection Endpoint (DCE) in your architecture:

1. Ensure the DCE association exists and resolves first.
2. Then create the DCR association.

Reasoning: AKS resolves associations in this order internally: AKS > DCE > DCR

• If the DCE isn’t ready, the DCR association cannot resolve.
• Creating the DCE first ensures the chain is fully resolvable.

Let us know if this works, please feel free to reach out if you have any further questions. Thanks

Hi, I'm sorry, I realized that this same bicep template has worked before in this subscription, but it is in a different resource group. It was for an aks and azure monitor workspace in a different region.

Also, this is a production environment for my team, and I don't want one of my clusters to have an add-on that our other environments don't have. For the same reason, I am not really able to create random resources and connect DCRs to them. I can perform read-only operations and/or bicep deployments of production resources that we need. The relevant DCE does exist and was created in the same deployment.

Hello, Thanks for the update, If DCR references a DCE, ARM must validate that the DCE exists before it will accept the DCR as fully provisioned. The DCRA then validates the DCR in the same way. If Bicep loses the symbolic reference at any point in that chain typically when an ID is passed as a string parameter across module boundaries it deploys all three resources with no enforced ordering. Regional ARM latency differences mean this race condition surfaces in some regions and not others, which explains why the same template works in one region pairing but fails in another within the same subscription.

Check how IDs are passed between your modules from start to end:

You can map out how the DCE ID flows into the DCR, and how the DCR ID flows into the DCRA. Look for any point where the ID becomes a plain string parameter rather than a direct module output reference:

// String passed across module boundary — breaks implicit dependsOn
module dcra 'dcra.bicep' = {
  params: {
    dcrId: dcrModule.outputs.dcrId  // becomes a plain string inside dcra.bicep
  }
}
 
//  Symbolic reference used directly on the resource — preserves dependsOn
resource dcra 'Microsoft.Insights/dataCollectionRuleAssociations@2022-06-01' = {
  properties: {
    dataCollectionRuleId: dcrModule.outputs.dcrId  // direct reference
  }
}

You can add explicit dependsOn on the DCRA covering both DCR and DCE

This is a safe, non-destructive Bicep-only change that closes the ordering gap regardless of how IDs are passed:

resource dcra 'Microsoft.Insights/dataCollectionRuleAssociations@2022-06-01' = {
  name: 'dcra-name'
  scope: aksCluster
  properties: {
    dataCollectionRuleId: dcrId
  }
  dependsOn: [
    dcrModule
    dceModule
  ]
}

Confirm region details of the working vs failing deployment

Compare the DCE, DCR, and AKS cluster regions between the previously working deployment in this subscription and the current failing one. This confirms whether regional ARM commit latency is a consistent factor, which would further validate the dependency ordering as the root cause.