On this page Jump to section
Published October 27, 2026
Question and scope
The question should name the unit, the start event, the stop event, and the decision the result is meant to inform. If the population is too small to sample, say so and report the complete set rather than implying a larger study. Define age from the first recorded request, not from the first time someone remembered to open the case. Write the decision the study is meant to inform before choosing the measure. Keep the measurement note short enough to read with the case and specific enough to support the reported duplicate bill count.
The scope is one defined population of bank reconciliation cases observed from first request through verified close in the bill pay system. The population should include cases handled by every owner and channel in scope, not only the queue with the cleanest records. The definitions should be short enough to use at the queue and precise enough to defend in a review with controller. Write the decision the study is meant to inform before choosing the measure. If two reviewers disagree, preserve both classifications until the definition or the evidence is resolved.
Population and unit
If a reopened case creates a new requirement, count it as a new case and retain a link to the earlier record. Report the number of cases that could not be classified and the reason, rather than forcing them into a category. A correction should be classified by the original case and by the correction event, with both dates retained. Use the same case key across the queue export and the evidence extract to avoid double counting. Link the observation to the exact field in the bill pay system and to the reconciliation report that supports it.
If a reopened case creates a new requirement, count it as a new case and retain a link to the earlier record. Record the queue at the start and end of each observation day so additions and withdrawals are visible. Define how a case that waits for business owner is counted: paused, open, or excluded, but not silently dropped. Separate cases opened before the window from cases opened and closed inside it, and report both counts. Do not let a summary written later replace the evidence that was available at the first decision.
Definitions and measures
Define the controlled event as the moment a bank reconciliation request becomes work, not the moment someone starts typing a note. Do not store sensitive content in the analysis file when a reference to the approved repository is enough. Record the person who approved the definition and the date it became effective. Give one concrete example and one boundary case to each reviewer before collection begins. If the case moved between owners, include both owner periods in the review note and name the handoff time.
A case that has no destination change should be defined as cancelled or rejected, not counted as complete. Do not store sensitive content in the analysis file when a reference to the approved repository is enough. A duplicate request is a separate intake event even when it points to the same underlying need; record the link. Give one concrete example and one boundary case to each reviewer before collection begins. Keep the review note with the case so the next sampler does not have to reconstruct the decision from memory.
Data dictionary and evidence
Use a pseudonymous case key and keep the identity crosswalk in an approved system with restricted access. Keep the identity crosswalk in the approved system and restrict it to the review owner. Run a five-case pilot before the full sample and revise any field that two reviewers interpret differently. Do not use a zero or an empty string to represent missing evidence. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract.
Each case needs the source, the observation time, the field supported, the owner, and the reviewer. If a field is derived, record the formula and the source fields used to calculate it. Record the data extraction date and the version of the close checklist used for the population count. Use a separate field for reviewer notes so the coded values stay comparable. At QBO Assistant, the bank reconciliation review should end with a dated note and a named owner for the next check.
The data dictionary should name the allowed values for status, exception reason, owner, and close evidence. Name the required fields for each article type and mark conditional fields in the dictionary. Record the source identifier, the field name, the observation time, and the reviewer for every extracted value. Keep the original request, the approval, and the destination value linked to the same key. If the evidence is sensitive, record the approved reference and access path rather than copying the content.
Sampling and collection
Include adverse cases: a missing bill image, a conflicting source, an unavailable bookkeeper, and a tight deadline. If a source cannot be accessed, mark it unavailable and record the access problem rather than asking the operator to reconstruct it from memory. Keep the sample selection reproducible by recording the export date, filter, and random seed or start rule. Record the sampling frame as an export with a date, filter, and row count. The note should distinguish what was observed, what was inferred, and what still needs an owner decision.
Stop expanding the sample only when the review question is answered, not when the result starts to look favorable. Use the same collection form for every case and keep the original record linked to the extracted fields. Stratify by risk, channel, and owner so the sample is not dominated by the easiest queue. A case that is difficult to access is still part of the frame; report the access failure rather than replacing it silently. Record this decision in the study log with the case key, the review date, and the person who approved the classification.
Include adverse cases: a missing approval email, a conflicting source, an unavailable vendor, and a tight deadline. Collect the case from the bank feed, the approval email, and the approval record, and stop at the verified close. Use a random or systematic start rule and then take consecutive cases within each stratum. Include at least one case from each owner in scope when the frame allows it. Keep the evidence reference beside the coded value so a reviewer can test the same case without rebuilding the queue.
Analysis and reporting
Keep case-level detail available for review, because a summary chart without records cannot support a decision. Report denominators with every rate and separate missing evidence from negative evidence. Record the sample date, the denominator, the numerator, and the exclusion count for every reported rate. Report the numerator, denominator, observation window, and exclusion count for each measure. If the source changes after the observation, preserve the earlier value and add a dated note instead of replacing it.
Compare reconciliation age across owners only after checking the case mix, the channel, and the evidence standard. Compare periods only after checking the case mix, the definitions, and the queue rules. When two groups differ, check whether the difference survives after adjusting for case mix and evidence standard. End with the decision the evidence supports and the decision it does not support. At QBO Assistant, the reviewer should be able to explain this step to business owner from the case record alone.
Limitations
Report what remains unknown and what evidence would be needed to answer the next question. Small samples are unstable; report the range and the cases that create the variation. Record the response or access problems that prevented a case from being reviewed. State what a follow-up study would measure and which population it would include. Use the same note in the case file and the study extract; do not create a second version in a personal document.
The record may be incomplete, delayed, or maintained outside the team, so the method cannot infer missing work from a clean summary. The study cannot determine whether a professional judgment was correct, only whether the record supports it. Do not treat a missing record as proof that the work did not happen; it is evidence about the record. Record the access problems that prevented a case from being reviewed. When a case needs an exception, record the reason, the owner, and the date before continuing the review.
Review and next action
Repeat the protocol after the change with the same definitions, window, and exclusions. Report what moved, what stayed the same, and what the study still cannot explain. Link the action to the specific bank reconciliation cases that motivated it. If the action needs approval, record the approver and the date before the change is applied. Keep the measurement note short enough to read with the case and specific enough to support the reported close days.
Record the decision and the next check so the study does not end as an archive. Name the owner, the review date, and the decision the study is expected to inform. Set the next sample date before closing the review, so the action does not drift. Record the owner, due date, and exact change that will be tested at the next review. If two reviewers disagree, preserve both classifications until the definition or the evidence is resolved.