Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Important
Legal and regulatory considerations- Organizations need to evaluate potential specific legal and regulatory obligations when using any AI services and solutions, which may not be appropriate for use in every industry or scenario. Additionally, AI services or solutions are not designed for and may not be used in ways prohibited in applicable terms of service and relevant codes of conduct.
Some or all of this functionality is available as part of a preview release. The content and the functionality are subject to change.
Monitor usage of Personalized Shopping Agent
Use the following methods to monitor how the Personalized Shopping Agent is used and performing.
Built-in analytics: Use the Copilot Studio (and underlying PVA) analytics dashboard to track sessions, resolution rate, abandonment rate, and common topics/queries. For example, if 30% of 500 sessions trigger the fallback topic, investigate those queries to identify gaps.
Transcripts review: During preview, full conversation logging for custom agents might be limited due to privacy. When available and in compliance with privacy rules, review transcripts (or internal test conversations) to understand user phrasing and identify where the agent succeeds or fails.
Feedback mechanism: You can embed an in-agent feedback prompt. For example, at the end of a session or after an answer, the agent could ask, "Did that answer your question? (Yes/No)". If the answer is No, the agent can reply, "Thanks—I'm still learning. I pass this question on." You can manually capture these negative responses for training. The feedback system isn’t built in, but you can simulate it by having a fallback on negative user responses. Alternatively, solicit feedback through other channels (for example, an email survey to pilot users).
Plan for continuous improvement
Use monitoring data to prioritize improvements and iterate on the agent. Work with your Microsoft account representative to discuss improvements and enhancements.
Data refreshes: Keep the data current. Ensure scheduled dataflow refreshes complete successfully and that catalog changes (new or discontinued products) are reflected in the agent. Regularly verify data freshness to avoid stale recommendations.
If you have seasonal product rotations, plan a major data update and test each season’s new data with the agent. Also update any seasonal dialogue. For example, around holidays customers ask for "gifts for Christmas"—adjust the agent to handle that specifically (for example, point to a holiday collection).
Scaling to more users or customers: After a successful pilot, you can roll it out wider. For internal associates that might mean to inform all stores about the new Personalized Shopping Agent they can use in Teams. Provide them with a quick, user-friendly, engaging how-to guide. For public web or mobile deployments, launch cautiously (consider a beta phase) and monitor satisfaction closely. Public usage can be unpredictable, so ensure the agent is robust against random chatter.
Best practices for maintaining Personalized Shopping Agent
Follow these best practices to maintain the agent and use available resources for continuous improvement.
Regular updates
Treat the agent as a living product. Schedule periodic reviews (for example, monthly) to:
- Update the knowledge sources (new documentation, policy changes, product updates).
- Retrain or adjust behavior if products or strategy change (for example, add filters or prompts to prioritize sustainable products if the business emphasizes them).
- Check for template updates released by Microsoft. These updates could come through solution updates in the Power Platform environment. Have an admin check if an update is available for Personalized Shopping Agent, especially after the solution goes GA. Test any updates in a sandbox if possible, then apply them to production.
Important
Topics update only if they're unmodified. If a customization layer exists, the agent doesn't receive template updates. If you added a customization layer, remove the layer, apply the update, and then reapply the layer to re-enable updates.
Scaling dataflows
As data volume increases, refine your data management strategy:
- For large catalogs with slow refreshes, implement incremental refresh using keys and last-modified timestamps to reduce full refresh times.
- Archive or mark discontinued products so the agent excludes them from responses or explicitly states "currently unavailable."
Security reviews
If the agent gains capabilities that access transactions or customer data, perform a new security and privacy review and ensure compliance with data-handling requirements (especially when logging conversations).
- Follow Microsoft’s guidance and terms of use for AI solutions. For example, disclose data collection and recording to users when required.
- For customer-facing agents, include a visible disclaimer such as: "This agent is AI‑powered and has limitations; verify critical information."