On this page Jump to section
Published October 27, 2026
Question and scope
State the observation window, the sites or teams included, and the exclusions before collecting the first case. The unit definition should be stable enough that two reviewers can classify the same case the same way. Distinguish an administrative pause from a failure: a pause waits for a named decision, while a failure leaves the next action unowned. Write the decision the study is meant to inform before choosing the measure. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract.
A useful scope is narrow enough to reproduce and specific enough to show whether unreconciled line is being detected. If one request produces several reconciliation report updates, define whether the unit is the request or each update before sampling. Write the primary measure and one supporting measure; a measure without a denominator or observation window is not ready. State whether the study covers one team, one client, or all cases in the defined service. At QBO Assistant, the receipt collection review should end with a dated note and a named owner for the next check.
Population and unit
Test the unit definition on a case that crosses owners, a case that is still open, and a case that was reopened. A case that moves between owners stays one case unless the process defines a new requirement at the handoff. The stop event should be the verified destination update, not the last message in the thread. If a request was reopened after the window, keep it in the original population and report the later event separately. If the evidence is sensitive, record the approved reference and access path rather than copying the content.
Test the unit definition on a case that crosses owners, a case that is still open, and a case that was reopened. Record the queue at the start and end of each observation day so additions and withdrawals are visible. If the definition of duplicate bill count changes, keep the old and new values side by side for the transition period. Record the extraction date and the queue filter so another reviewer can reproduce the population count. The note should distinguish what was observed, what was inferred, and what still needs an owner decision.
Definitions and measures
A pause should have a reason, an owner, and a review date; otherwise it is an unowned case with a different label. Use a stable case key and a separate identity crosswalk stored in an approved system. A completeness measure should name the required fields and the evidence state; an overall percentage is not reproducible. Give one concrete example and one boundary case to each reviewer before collection begins. Record this decision in the study log with the case key, the review date, and the person who approved the classification.
Measure elapsed time from the first recorded request to the verified destination update, and report waiting intervals separately. Keep the original text and the coded value together so a reviewer can check the classification. Use the same status vocabulary in the queue, the evidence extract, and the report so a case does not change category. Record the pause rule and the person authorized to release a paused case. Keep the evidence reference beside the coded value so a reviewer can test the same case without rebuilding the queue.
Data dictionary and evidence
Test the dictionary on five cases before the full sample; ambiguous fields should be fixed before collection, not after. Give every case a study key that is not the client name, member number, or candidate email. Use one row per case and one column per required field; do not bury evidence in a long free-text note. Archive the dictionary with the study so a later reviewer can reproduce the categories. If the source changes after the observation, preserve the earlier value and add a dated note instead of replacing it.
The data dictionary should name the allowed values for status, exception reason, owner, and close evidence. Name the required fields for each article type and mark conditional fields in the dictionary. Use one row per case and one column per required field; do not bury evidence in a long free-text note. Keep the original request, the approval, and the destination value linked to the same key. At QBO Assistant, the reviewer should be able to explain this step to vendor from the case record alone.
Keep free-text notes separate from coded fields so the analysis does not depend on inconsistent wording. Record the extraction time separately from the event time so a late entry is visible. A missing value should have a reason code, not a blank cell or a zero. Record the source location, not just the file name, so a reviewer can reopen the exact evidence. Use the same note in the case file and the study extract; do not create a second version in a personal document.
Sampling and collection
Stop expanding the sample only when the review question is answered, not when the result starts to look favorable. Keep the collection window and the number of collectors in the report; both affect how the result should be interpreted. Record the eligible count and every exclusion before the sample is drawn. Record the sampling frame as an export with a date, filter, and row count. When a case needs an exception, record the reason, the owner, and the date before continuing the review.
Stop expanding the sample only when the review question is answered, not when the result starts to look favorable. Keep the collection window and the number of collectors in the report; both affect how the result should be interpreted. Record the eligible count and every exclusion before the sample is drawn. Record the date of the sample and the version of the queue export. Keep the measurement note short enough to read with the case and specific enough to support the reported duplicate bill count.
A convenience sample of recently closed cases will overrepresent easy work and underrepresent the tail that needs attention. Record the time of each observation so a later reviewer can distinguish a late entry from a changed decision. Include reopened, unresolved, and exception cases in the sample frame. A case that is difficult to access is still part of the frame; report the access failure rather than replacing it silently. If two reviewers disagree, preserve both classifications until the definition or the evidence is resolved.
Analysis and reporting
Separate observation from interpretation; a high completion rate on a small easy sample is not evidence of a strong control. Compare periods only after checking the case mix, the definitions, and the queue rules. Use a simple before-and-after comparison only when the definitions and observation window are the same. Use a case-level table to explain an outlier before treating it as a process signal. Link the observation to the exact field in the receipt inbox and to the vendor W-9 that supports it.
Report the eligible count, the sample, the missing-field rate, the exception rate, and the unresolved count at the close of the window. Keep descriptive findings separate from causal claims. Write the finding in plain language and separate what was observed from what the team believes caused it. End with the decision the evidence supports and the decision it does not support. Do not let a summary written later replace the evidence that was available at the first decision.
Limitations
Selection bias remains if the team samples only records that are easy to open or cases handled by cooperative owners. Small samples are unstable; report the range and the cases that create the variation. Do not treat a missing record as proof that the work did not happen; it is evidence about the record. Report the limits of the the vendor master data, including fields that were unavailable or changed. If the case moved between owners, include both owner periods in the review note and name the handoff time.
The record may be incomplete, delayed, or maintained outside the team, so the method cannot infer missing work from a clean summary. The record may be incomplete or delayed, so the method cannot infer work that never entered the queue. State that the result describes the observed sample and not the entire industry or every team in the service. Keep the limitations with the result so a local observation is not turned into a general claim. Keep the review note with the case so the next sampler does not have to reconstruct the decision from memory.
Review and next action
Report what moved, what did not, and which result is still unexplained. If the result is inconclusive, say so and choose a narrower question rather than claiming a trend. Keep the old process note until the new process has been tested; do not erase the baseline. Link the action to the case evidence that prompted it, not only to the meeting notes. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract.
If the action affects wrong vendor, route it to vendor and record the approval before implementation. Keep the review note with the receipt collection evidence so the next owner can see why the process changed. Close the review only when the owner has accepted the result and the next check is scheduled. Link the action to the case evidence that prompted it, not only to the meeting notes. At QBO Assistant, the receipt collection review should end with a dated note and a named owner for the next check.