Managing Compliance Questions
Accessing the Compliance Page
To manage your agency's compliance questions, click Compliance in the left sidebar. This opens the Compliance page where you control everything about how your calls are scored.
What You'll See When You Arrive
The Compliance page has a clean header area at the top. If you're a superadmin or agencies admin, you'll see an agency selector dropdown that lets you switch between agencies to manage their compliance questions individually. If you're an agency admin, you'll see your agency's name displayed as a badge in the header, confirming which agency's questions you're managing.
Below the header, the page is organized into two tabs:
- Agency Questions — where you create and manage the compliance questions specific to your agency.
- AI Context — where you provide additional background information that helps the system more accurately evaluate your calls.
The Agency Questions tab is selected by default when you first arrive.
---
The Agency Questions Tab
This is where you'll spend most of your time on the Compliance page. It contains everything you need to build and maintain your compliance question set.
The Questions List
If you already have questions configured, they appear in a list below the Add Question button. Each question in the list displays the following information at a glance:
- Question Text — the full text of the compliance question.
- Question Type Badge — a small badge indicating the type (Boolean, Carrier Selection, Agent Selection, Disposition Selection, Custom Options, or Free Text).
- Position Number — the display order position, shown as a number.
- Required Badge — if the question is marked as required, a badge indicates this.
- Pass Rate Badge — if the question counts toward the pass rate calculation, a badge shows this.
- Vertical Badges — if the question is restricted to specific industry verticals, those verticals appear as badges. If no verticals are selected, the question applies to all agencies.
- Answer Options — for Custom Options questions, the configured answer choices are displayed.
- Edit Button — opens the question in the editing modal so you can make changes.
- Delete Button — removes the question entirely (you'll be asked to confirm).
This list gives you a comprehensive snapshot of your entire compliance question set without needing to click into each one.
---
Creating a New Compliance Question
To add a new question to your compliance scorecard:
- Click the Add Question button at the top of the Agency Questions tab.
- The Add/Edit Question Modal appears with the following fields.
Field-by-Field Walkthrough
#### Question Text (Required)
This is a textarea with 3 rows where you type the actual compliance question the system will evaluate against call transcripts.
Tips for writing effective questions:
- Be specific and unambiguous. Instead of "Did the agent do a good job?", write "Did the agent clearly state the plan's monthly premium amount?"
- Focus on one concept per question. Don't combine multiple requirements into a single question. Instead of "Did the agent introduce themselves and read the disclosure?", create two separate questions — one for the introduction and one for the disclosure.
- Describe observable behaviors. The system evaluates what was said on the call, so frame questions around things that would be spoken aloud. "Did the agent mention the enrollment deadline?" is better than "Did the agent understand the enrollment timeline?"
- Use complete sentences. Write questions as full, clear sentences that anyone on your team could understand without additional context.
Strong question examples:
- "Did the agent state their full name and the name of the agency within the first 30 seconds of the call?"
- "Did the agent confirm the caller's current insurance coverage before presenting new options?"
- "Did the agent clearly explain the difference between Medicare Advantage and Medicare Supplement plans?"
- "Was the caller informed about the plan's out-of-pocket maximum?"
- "Did the agent ask the caller if they have any existing health conditions that might affect coverage?"
Weak question examples to avoid:
- "Good greeting?" — too vague, not a complete sentence.
- "Compliance?" — impossible for the system to evaluate.
- "Did the agent do everything right?" — too broad, not tied to a specific behavior.
#### Question Type (Required)
This is a dropdown menu that determines how the system will answer your question. Select the type that best matches what you're trying to measure. Below is a detailed explanation of each type and when to use it.
---
##### Boolean (Yes/No)
What it does: The system reads the transcript and answers with a simple Yes or No.
When to use it: Whenever you have a straightforward pass/fail question about whether something happened or didn't happen during the call.
How results appear: The answer shows as a color-coded badge — green for Yes (pass) and red for No (fail).
Example questions:
- "Did the agent state their full name and company name within the first 30 seconds?"
- "Was the caller informed that the call is being recorded for quality assurance?"
- "Did the agent verify the caller's date of birth before providing any quotes?"
- "Did the agent read the required CMS disclaimer before discussing plan benefits?"
- "Did the agent ask for permission before transferring the caller?"
Pro tip: Boolean questions are the backbone of most compliance programs. If you're just getting started, begin with Boolean questions for your most critical compliance requirements.
---
##### Carrier Selection
What it does: The system identifies which insurance carrier was discussed on the call and selects it from your pre-configured list of compliance carriers.
When to use it: When you need to track which carrier's products are being discussed, recommended, or sold on each call.
How results appear: The selected carrier appears as a purple badge in the compliance results.
Prerequisite: You must have compliance carriers configured in your Agency Settings before this question type will work properly. The system matches transcript content against your carrier list.
Example questions:
- "Which insurance carrier was discussed on this call?"
- "Which carrier's plan did the agent ultimately recommend to the caller?"
- "What carrier was the caller most interested in?"
Pro tip: If your agency represents multiple carriers, Carrier Selection questions help you track market share and identify which carriers are being presented most often — useful for both compliance and business intelligence.
---
##### Agent Selection
What it does: The system evaluates the agent's behavior and identifies specific actions, adherence to scripts, or conduct patterns.
When to use it: When you want to assess whether the agent followed specific protocols, used approved language, or exhibited particular behaviors during the call.
How results appear: The answer appears as a badge in the compliance results.
Example questions:
- "Did the agent follow the approved script for the initial greeting?"
- "Did the agent use the compliant language when describing plan benefits?"
- "Did the agent follow the proper escalation procedure when the caller expressed a complaint?"
---
##### Disposition Selection
What it does: This type links to call disposition data, allowing the system to evaluate and categorize what happened as a result of the call.
When to use it: When you want to tie compliance evaluation to how the call was ultimately dispositioned or what the call outcome was.
How results appear: The disposition appears as a badge in the compliance results.
Example questions:
- "What was the primary outcome of this call?"
- "Did the call result in a completed application or enrollment?"
- "Was the call dispositioned appropriately based on the conversation content?"
---
##### Custom Options
What it does: You define your own list of possible answers, and the system selects the best match from your options based on what was discussed in the transcript.
When to use it: When you need a multiple-choice evaluation with specific answer categories that you control. This is more flexible than Boolean but more structured than Free Text.
How to set up options: When you select Custom Options as the question type, an Answer Options textarea appears in the modal. Enter each possible answer on its own line — one option per line. The system will choose the single best match from your list.
How results appear: The selected option appears as a badge in the compliance results.
Example 1 — Product type tracking:
Question: "What type of insurance product was discussed on this call?"
Answer Options (entered one per line):
```
Medicare Advantage
Medicare Supplement
Part D Prescription Drug Plan
Life Insurance
Dental and Vision
Final Expense
None Discussed
```
Example 2 — Enrollment period identification:
Question: "What enrollment period is the caller currently in?"
Answer Options:
```
AEP (Annual Enrollment Period)
OEP (Open Enrollment Period)
SEP (Special Enrollment Period)
Not Applicable
Unknown / Not Discussed
```
Example 3 — Caller motivation tracking:
Question: "What was the primary reason the caller expressed interest?"
Answer Options:
```
Reducing monthly premiums
Improving coverage
Turning 65 / new to Medicare
Lost current coverage
Referred by another agent
Responding to a mailer or advertisement
Other
```
Pro tip: Always include a catch-all option like "None Discussed," "Not Applicable," or "Other" so the system has a valid selection even when the call doesn't clearly match any of your primary categories.
---
##### Free Text
What it does: The system provides an open-ended, written response based on the transcript. There are no predefined answers — the system writes its analysis in its own words.
When to use it: When you need subjective analysis, summaries, or detailed observations that can't be captured in a Yes/No or multiple-choice format.
How results appear: The response appears as a blue badge that can be expanded to read the full text.
Example questions:
- "Provide a brief summary of any compliance concerns identified during this call."
- "Summarize the agent's closing pitch and whether it included all required elements."
- "What objections did the caller raise, and how did the agent address them?"
- "Describe the key benefits the agent highlighted during the call."
- "Were there any moments during the call where the agent appeared to deviate from the approved script? If so, describe what happened."
Pro tip: Free Text questions are excellent for capturing nuance, but they don't contribute to pass/fail scoring. Use them alongside Boolean questions — the Boolean questions give you the score, and the Free Text questions give you the context.
---
#### Answer Options (Custom Options Type Only)
This textarea only appears when you select Custom Options as the question type. Enter each answer option on a separate line. The system will evaluate the call transcript and select the single most appropriate option from your list.
Formatting rules:
- One option per line.
- Press Enter / Return to start a new option.
- Keep options concise but descriptive.
- Avoid duplicate or overly similar options.
- Include a fallback option (e.g., "Other" or "Not Discussed").
#### Applicable Verticals
This section displays a set of checkboxes representing the industry verticals configured on the platform. Use these checkboxes to restrict the question so it only applies to agencies operating in specific verticals.
How it works:
- Leave all checkboxes unchecked = the question applies to all agencies regardless of their vertical. This is the default and recommended setting for general compliance questions.
- Check one or more verticals = the question only applies to agencies that match at least one of the selected verticals. This is useful for industry-specific compliance requirements.
Example use case: If you have a question like "Did the agent read the CMS Medicare disclaimer?", you might restrict it to the Medicare vertical since it's not relevant to agencies selling auto insurance.
#### Display Position
A number field accepting values from 1 to 100 that controls where this question appears in the compliance results scorecard.
How to use it effectively:
- Lower numbers appear first; higher numbers appear later.
- Group related questions with consecutive position numbers.
- Suggested organization:
- 11–20: Identity verification and consent questions
- 21–40: Core compliance and disclosure questions
- 41–60: Product-specific questions
- 61–80: Closing and summary questions
- 81–100: Optional / supplementary questions
Pro tip: Leave gaps between position numbers (e.g., use 10, 20, 30 instead of 1, 2, 3) so you can easily insert new questions later without having to renumber everything.
#### Required Question Toggle
This toggle switch determines whether the question is mandatory in your compliance scorecard.
- ON (Required): The question will always be evaluated and will appear prominently in results. A Required badge is shown next to the question in the questions list.
- OFF (Optional): The question is still evaluated, but it's treated as supplementary.
When to make a question required: Use this for questions that represent non-negotiable compliance requirements — legal disclosures, consent statements, identity verification steps, and any behavior mandated by regulation.
#### Pass Rate Calculation Toggle
This toggle switch controls whether the question's result counts toward the overall Pass Rate percentage shown on the Compliance Overview card.
- ON (Counts toward pass rate): A passing answer on this question improves the pass rate; a failing answer lowers it. A Pass Rate badge appears next to the question in the list.
- OFF (Excluded from pass rate): The question is still evaluated and the answer is still shown in results, but it doesn't affect the pass rate number.
When to exclude from pass rate: Exclude questions that are informational rather than evaluative. For example:
- Free Text summary questions ("Summarize the agent's pitch") aren't pass/fail, so they shouldn't affect the pass rate.
- Carrier Selection questions ("Which carrier was discussed?") identify information but don't represent a compliance pass or fail.
- Custom Options questions that categorize rather than evaluate ("What type of product was discussed?") are often better excluded from the pass rate.
When to include in pass rate: Include questions where there's a clear right/wrong answer that reflects compliance, such as Boolean questions about required disclosures, consent, and verification steps.
---
Saving Your Question
Once you've filled out all the fields in the modal:
- Review your settings — check the question text, type, options (if applicable), position, and toggles.
- Click the Save button at the bottom of the modal.
- The modal closes and your new question appears in the questions list on the Agency Questions tab.
- The question will be used to evaluate all future calls from the moment it's saved.
---
The Questions List — Managing Existing Questions
After you've created questions, they all appear in the list view on the Agency Questions tab. Here's how to manage them.
Reviewing Your Questions
Scan the list to get a quick overview of your compliance program. Each question shows its full text, type badge, position number, and status badges (Required, Pass Rate, Verticals). For Custom Options questions, the configured answer choices are also displayed inline.
Editing a Question
- Find the question you want to modify in the list.
- Click the Edit button on that question's row.
- The Add/Edit Question Modal opens with all fields pre-populated with the question's current settings.
- Make your changes — update the text, change the type, adjust the position, toggle the Required or Pass Rate switches, or modify verticals.
- Click Save to apply the changes.
Important note: Editing a question does not retroactively change the compliance results of calls that have already been scored. The updated question will be used for all future calls only.
Deleting a Question
- Find the question you want to remove.
- Click the Delete button on that question's row.
- Confirm the deletion when prompted.
Important note: Deleting a question is permanent. Historical compliance results from calls already scored will retain the answers for the deleted question, but no future calls will be evaluated against it. If you're unsure, consider simply toggling off the Pass Rate Calculation and Required switches instead of deleting.
---
The AI Context Tab
Click the AI Context tab (next to Agency Questions) to access the context configuration area.
What Is AI Context?
AI Context lets you provide additional background information that helps the system more accurately evaluate your calls. Think of it as giving the scoring engine a briefing before it reviews your transcripts.
The Context Message Textarea
The tab contains a Context Message textarea with 6 rows of input space. The placeholder text in the field explains what kind of information to include.
What to Include in Your Context
Write a clear paragraph or set of bullet points that describe:
- Your industry and business type. Example: "We are a Medicare insurance agency that primarily sells Medicare Advantage and Medicare Supplement plans during AEP and OEP."
- Your typical call flow. Example: "Calls typically begin with a greeting and disclosure, followed by needs assessment, plan presentation, and enrollment."
- Important terminology. Example: "When agents say 'SOA', they mean Scope of Appointment. 'T65' refers to people turning 65 and becoming Medicare-eligible."
- Specific compliance requirements. Example: "Agents are required to read the CMS disclaimer verbatim before discussing any specific plan benefits. The exact wording is: [your disclaimer text]."
- Call structure expectations. Example: "Most calls are inbound from mailer responses. Agents should verify the caller received our mailer before proceeding."
Saving Your Context
After writing your context message, click the Save Agency Context button below the textarea. The context is saved immediately and will be used when evaluating all future calls for your agency.
Updating Your Context
You can return to the AI Context tab at any time to update your context message. Simply edit the text in the textarea and click Save Agency Context again. The updated context applies to all future evaluations.
Pro tip: Review and update your AI Context whenever you:
- Change your call scripts or procedures.
- Start selling new products or representing new carriers.
- Receive feedback that compliance scoring seems inaccurate for certain questions.
- Onboard new agents with different training backgrounds.
---
Best Practices for Building Your Question Set
Write Specific, Actionable Questions
The more specific your question, the more accurate the evaluation. Compare:
- Vague: "Was the call compliant?" → Too broad for accurate scoring.
- Specific: "Did the agent state their full name and agency name within the first 30 seconds of the call?" → Clear, measurable, tied to a specific moment.
One Concept Per Question
Resist the temptation to combine multiple requirements into one question. Each question should evaluate exactly one thing.
- Bad: "Did the agent introduce themselves, read the disclosure, and verify the caller's identity?" → Three separate requirements jammed together.
- Good: Create three questions:
2. "Did the agent read the required disclosure statement?"
3. "Did the agent verify the caller's identity by confirming their date of birth?"
Use Clear, Simple Language
Write questions that any team member could understand without specialized knowledge. Avoid jargon, abbreviations, or internal shorthand — unless you've defined those terms in your AI Context.
Focus on Observable Behaviors
Since the system evaluates what was spoken during the call, frame your questions around things that would appear in a transcript. Internal thoughts, intentions, or actions taken outside the call (like updating a CRM) can't be detected.
- Observable: "Did the agent verbally confirm the caller's ZIP code?"
- Not observable: "Did the agent correctly enter the ZIP code into the system?"
Start with 5–10 Critical Questions
Begin with a manageable set of questions covering your most important compliance requirements. This lets you:
- Validate that questions are being scored accurately.
- Adjust wording if results seem off.
- Build familiarity with the compliance workflow before scaling up.
- Avoid overwhelming agents with a massive scorecard on day one.
You can always add more questions as your compliance program matures.
---
Understanding Platform Questions vs. Agency Questions
How Platform Questions Affect You
If your platform administrator has configured Platform Questions, those questions automatically appear in your compliance scorecards alongside your Agency Questions. You cannot edit, disable, or delete Platform Questions from the agency level.
What this means for you:
- Your call compliance results will include answers to both Platform Questions and your Agency Questions.
- The pass rate calculation includes both sets of questions (unless individual questions have pass rate calculation turned off).
- When reviewing scorecards, you'll see all questions together in a single list, ordered by their display position numbers.
Coordinating with Platform Questions
When building your Agency Questions, consider what's already covered by Platform Questions to avoid duplication:
- Review your compliance scorecards for a few recent calls to see which Platform Questions are already being evaluated.
- Focus your Agency Questions on requirements that are unique to your agency, your specific carrier partnerships, or your particular regulatory environment.
- If a Platform Question partially covers something you need, create an Agency Question that addresses the specific gap.
---
Reviewing AI Answers to Check Question Effectiveness
After your questions have been active for a while, review the results to make sure the system is interpreting your questions as intended.
How to Check
- Go to Call Logs and click Compliance Check on several recent calls.
- Read through the question-by-question results in the Compliance Summary section.
- For any answers that seem incorrect, listen to the call recording and compare what was said to how the question was answered.
Common Issues and Fixes
- Question is too vague → inconsistent answers. Rewrite the question to be more specific. Add context in the AI Context tab if needed.
- Question asks about something not discussed on calls → mostly "Unanswered." Consider whether this question is relevant to your call type, or adjust your call scripts to ensure the topic is covered.
- Wrong carrier selected in Carrier Selection questions. Check that your compliance carrier list is up to date and includes all carriers your agents discuss.
- Free Text answers are too generic. Make the question more specific. Instead of "Summarize the call," try "Summarize the specific plan benefits the agent discussed, including premium, deductible, and out-of-pocket maximum."
---
Tips for Ongoing Question Management
Review Your Questions Regularly
Set a recurring calendar reminder — monthly or quarterly — to review your compliance questions. Ask yourself:
- Are all questions still relevant to current regulations?
- Have any new requirements been introduced that need new questions?
- Are there questions that consistently score 100%? If so, they might be too easy and could be tightened up.
- Are there questions with very low pass rates? If so, investigate whether the issue is agent behavior, question wording, or a training gap.
Update Questions When Regulations Change
Regulations evolve. When CMS issues new guidelines, when state insurance departments update requirements, or when your agency changes its approved scripts, update your questions immediately. The system automatically begins scoring against updated questions on all future calls.
Use Position Numbers to Organize Logically
Think of position numbers as an outline for your compliance scorecard. Organize questions in the order they'd naturally occur in a call:
- Opening (positions 1–10): Greetings, introductions, recording disclosures.
- Verification (positions 11–20): Identity confirmation, eligibility checks.
- Core Compliance (positions 21–40): Required disclaimers, mandated disclosures.
- Product Discussion (positions 41–60): Benefits explanation accuracy, carrier identification.
- Closing (positions 61–80): Enrollment confirmation, next steps, farewell.
- Supplementary (positions 81–100): Summary questions, general observations.
This logical flow makes it easier for supervisors and quality analysts to scan scorecards quickly and consistently.
Leverage Both Pass Rate and Non-Pass Rate Questions
Build your question set with a mix:
- Pass Rate ON for hard compliance requirements (disclosures, consent, verification) — these determine the compliance score.
- Pass Rate OFF for informational tracking (carrier discussed, product type, call summary) — these provide valuable data without penalizing agents.
This gives you a compliance score that truly reflects adherence to requirements, while still capturing rich analytical data for business insights.
Last updated: February 24, 2026