Share via

1 ES Hosted Pool issue - US EAST

Aaron Paterson 0 Reputation points Microsoft External Staff
2026-04-29T16:30:06.3566667+00:00

We are attempting a deployment utilizing the 1ES Hosted Pool from Microsoft and are having requests come into the pool seen in our networking logs but nothing is attempting to provisions, start, or begin a process that then fails since there are also no failure logs.

The deployment would be happening in the US-EAST region and last week we had another impact of the same exact variety that caused the same issue of not being able to provision a build agent.

I don't suppose anything will be done with this Q&A but, in an effort to produce some sort of up channel notification, I've created this.

Azure DevOps

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Rakesh Mishra 8,420 Reputation points Microsoft External Staff Moderator
    2026-04-29T18:41:29.31+00:00

    Hey Aaron, it sounds like your pipelines are queuing against the 1ES Hosted Pool in East US but no new scale-set VMs ever spin up (and you’re not even seeing extension-orchestration errors). Let’s walk through the most common culprits for VMSS agent pools, then I’ll list some questions so we can zero in on exactly what’s happening:

    What to check first

    1. Diagnostics tab in Azure DevOps
      • In Azure DevOps → Project Settings → Agent pools → select your 1ES pool → Diagnostics.
      • Do you see scale-out jobs running every 5 minutes? Any failures or “no scale action needed” messages?
    2. Idle vs. desired standby count
      • What’s your “minimum” and “maximum” agent count configured?
      • If your current idle count is ≥ minimum, the pool won’t scale out. Likewise, if you’re already at max capacity, it won’t spin up new instances.
    3. Azure subscription & region capacity
      • Check Azure Portal → Subscriptions → Usage + quotas for East US cores.
      • If you’re out of VM cores in that region, new VMs simply won’t start and you won’t see agent extension logs because provisioning never kicks off.
    4. Firewall/proxy/NSG rules
      • Even though VMs aren’t creating, the scale-set orchestration layer still needs to reach Azure DevOps endpoints. Make sure the pool (or your corporate network) isn’t blocking: • download.agent.dev.azure.com • vstsagentpackage.azureedge.net
    5. Azure DevOps Services health
      • Check https://status.dev.azure.com for any ongoing incidents in East US. Last week’s recurrence could point at a transient service issue in the region.
    6. RBAC & resource-group visibility
      • Since 1ES pools are Microsoft-managed, you don’t handle the VMSS yourself—but if your subscription had any recent policy or role changes it might impact Microsoft’s backing scale-set.

    With those bits of info we can pinpoint whether it’s a capacity/quota issue, a transient service incident, or something on your networking/RBAC side.

    Reference docs

    If above troubleshooting steps did not help, please share the requested information in Private messages.

    Note: This content was drafted with the help of an AI system.

    0 comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.