On this page Jump to section
Published October 27, 2026
Question and scope
Write the primary question and one supporting question, then stop; a long list of questions usually hides an undefined unit. If the population is too small to sample, say so and report the complete set rather than implying a larger study. Write the primary measure and one supporting measure; a measure without a denominator or observation window is not ready. The protocol should be short enough for an operator to follow and specific enough for a reviewer to repeat. Use the same note in the case file and the study extract; do not create a second version in a personal document.
The question should name the unit, the start event, the stop event, and the decision the result is meant to inform. A count without a denominator is not a measurement, especially when the queue changes size during the day. A case is complete only when the bank feed shows the approved destination value and the vendor W-9 supports it. Write the decision the study is meant to inform before choosing the measure. When a case needs an exception, record the reason, the owner, and the date before continuing the review.
Population and unit
When several vendor W-9 updates belong to one request, state whether the request or each update is the unit and apply that rule consistently. A denominator that changes during the window should be reported separately for each period. Define how a case that waits for controller is counted: paused, open, or excluded, but not silently dropped. Record the extraction date and the queue filter so another reviewer can reproduce the population count. Keep the measurement note short enough to read with the case and specific enough to support the reported missing receipt rate.
When several approval email updates belong to one request, state whether the request or each update is the unit and apply that rule consistently. The population should include unresolved cases, not only closures, or the study will overstate completion. Write the rule for a missing approval email before the first case so the treatment is consistent. A case that never received an owner still belongs in the population; do not drop it because it is difficult to classify. If two reviewers disagree, preserve both classifications until the definition or the evidence is resolved.
Definitions and measures
Test each definition on a routine case, a missing-evidence case, and a case that crossed owners. Keep the original text and the coded value together so a reviewer can check the classification. Define whether a case that waits for a third party is open, paused, or excluded, and apply that rule to every owner. Give one concrete example and one boundary case to each reviewer before collection begins. Link the observation to the exact field in the bill pay system and to the approval email that supports it.
Define the controlled event as the moment a bill intake request becomes work, not the moment someone starts typing a note. Keep the original text and the coded value together so a reviewer can check the classification. A duplicate request is a separate intake event even when it points to the same underlying need; record the link. If two reviewers disagree, preserve both classifications until the definition is clarified. Do not let a summary written later replace the evidence that was available at the first decision.
Data dictionary and evidence
Keep free-text notes separate from coded fields so the analysis does not depend on inconsistent wording. Use a controlled list for exception reasons; do not let free text become the analysis category. Run a five-case pilot before the full sample and revise any field that two reviewers interpret differently. Define who may change a coded value and how the change is logged. If the case moved between owners, include both owner periods in the review note and name the handoff time.
The data dictionary should name the allowed values for status, exception reason, owner, and close evidence. Give every case a study key that is not the client name, member number, or candidate email. Use one row per case and one column per required field; do not bury evidence in a long free-text note. Pilot the dictionary with two reviewers and compare their classifications before the main sample. Keep the review note with the case so the next sampler does not have to reconstruct the decision from memory.
Record the version or date of the missing receipt note and the the close checklist value so a later reviewer can see what was known at the first action. Record the extraction time separately from the event time so a late entry is visible. A missing value should have a reason code, not a blank cell or a zero. Use a separate field for reviewer notes so the coded values stay comparable. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract.
Sampling and collection
Use consecutive bill intake cases where possible, then stratify by owner, channel, and risk class. A second reviewer should re-extract a small subsample and record disagreements before the full analysis. Stratify by risk, channel, and owner so the sample is not dominated by the easiest queue. A case that is difficult to access is still part of the frame; report the access failure rather than replacing it silently. At QBO Assistant, the bill intake review should end with a dated note and a named owner for the next check.
Sample across the full observation window, including the days when the queue was busiest and the days when coverage changed. A second reviewer should re-extract a small subsample and record disagreements before the full analysis. Keep the sample selection reproducible by recording the export date, filter, and random seed or start rule. Include at least one case from each owner in scope when the frame allows it. If the evidence is sensitive, record the approved reference and access path rather than copying the content.
A convenience sample of recently closed cases will overrepresent easy work and underrepresent the tail that needs attention. A second reviewer should re-extract a small subsample and record disagreements before the full analysis. Keep the sample selection reproducible by recording the export date, filter, and random seed or start rule. Do not let the reviewer choose the sample after seeing the results. The note should distinguish what was observed, what was inferred, and what still needs an owner decision.
Analysis and reporting
Compare correction rate across owners only after checking the case mix, the channel, and the evidence standard. Compare periods only after checking the case mix, the definitions, and the queue rules. Keep the case-level table available for review and summarize only the measures that answer the question. When a rate changes, check whether the case mix or the evidence standard changed at the same time. Record this decision in the study log with the case key, the review date, and the person who approved the classification.
A difference between two periods is a signal to inspect, not proof that a process change caused the change. Keep descriptive findings separate from causal claims. If the result is unstable, say which cases create the variation and what a larger sample would test. Report the number of cases that drive a small-sample result and the direction of the effect. Keep the evidence reference beside the coded value so a reviewer can test the same case without rebuilding the queue.
Limitations
The cited public sources frame recordkeeping, privacy, security, or control practice; they do not provide a local result. A clean summary can hide missing evidence, especially when the operator who did the work is also the reviewer. State that the result describes the observed sample and not the entire industry or every team in the service. State that the result describes the observed sample, not the whole industry or every team in the service. If the source changes after the observation, preserve the earlier value and add a dated note instead of replacing it.
The cited public sources frame recordkeeping, privacy, security, or control practice; they do not provide a local result. Small samples are unstable; report the range and the cases that create the variation. Name the follow-up evidence that would reduce the largest uncertainty. State what a follow-up study would measure and which population it would include. At QBO Assistant, the reviewer should be able to explain this step to bookkeeper from the case record alone.
Review and next action
Keep the review note with the bill intake evidence so the next reviewer can see why the process changed. Name the owner, the review date, and the decision the study is expected to inform. Record the owner, the due date, and the exact change that will be tested at the next review. Link the action to the case evidence that prompted it, not only to the meeting notes. Use the same note in the case file and the study extract; do not create a second version in a personal document.
Keep the review note with the bill intake evidence so the next reviewer can see why the process changed. Name the owner, the review date, and the decision the study is expected to inform. If the action requires a policy or access change, route it to the owner and record the decision. Record the owner, due date, and exact change that will be tested at the next review. When a case needs an exception, record the reason, the owner, and the date before continuing the review.