AgentforceAdvancedevaluationtestingquality
Implement evaluation harness with golden transcripts before agent launch
Real World Scenario
Leadership demands go-live Friday; QA has no automated way to verify 200 critical intents still work after prompt tweak.
Expected Answer
• Curate golden set: user utterance, expected topic, required citations, forbidden phrases
• Automated regression scoring containment and action correctness
• Block release if score drops below threshold on golden set
• Include adversarial and edge cases not only happy path
• Human review sample of failed regression cases
• Track metrics over time not one-time pre-launch
• Version golden set with agent metadata releases
Follow-Up Questions & Answers
Click to expand — each follow-up includes a direct, interview-ready answer
Direct answer: Curate golden set: user utterance, expected topic, required citations, forbidden phrases Also consider: Automated regression scoring containment and action correctness In practice: Block release if score drops below threshold on golden set Optimize for scale and operational observability.
Architect Perspective
Agents without eval harness regress silently every prompt edit — treat eval as CI gate.