Skip to content
AtomicReps
Surviving the SOC 2 Audit

Your fix landed outside the window

Lesson 1 of 8

Your fix landed outside the window

A lesson from Surviving the SOC 2 Audit. Play it above, or read it through below.

  • A Type 2 opinion is about a period. The period already ran.

    The common read: The audit tests the state of our systems, so a remediation sprint before fieldwork buys a clean report.

    Broken, fix, green. That loop has worked your entire career, and this is the one system where it does not close. A type 2 report says the controls operated effectively throughout a specified period: here, 2026-01-01 to 2026-06-30, 181 dates, all behind you. There is no git revert for a control that did not operate on 12 January. A type 1 report opines as of a specified date, which is why remediating into a deadline works there and only there. Same controls, same firm. Different tense.

    The control in question emits one artifact per day: an MFA-status snapshot per in-scope account, written by the identity provider export with a snapshot_date on every row. That population is the thing tested, and 2026-01-12 through 2026-01-14 are three rows inside it. Nothing about the state of the account on the day of fieldwork edits those rows, because they are records of dates that already happened, and a record is not a state you can bring back to green.

    Detection is not automatic, and the honest sentence is narrower than the one on the readiness deck. The gap is discoverable, and if it is discovered the auditor will likely document an exception. What it never is, is out of scope for being old. That arrives as a testing-matrix row reading no exceptions noted for five months and one deviation for the sixth, filed against the identity team, whose enforcement policy was detached by a group migration during a platform cutover and reattached the same week.

    Takeaway: "We fixed it in May" is a sentence about June. The report is about January.

    So the operational move is not a sprint, it is a date. Put the earliest date you would defend on the window's left edge, and treat every day after it as a day that will be read back to you. The sprint still has a job: it is what makes the NEXT window clean, and the report you are being asked for now covers the one that already ran.

  • A daily control leaves 181 chances to disagree with you. A quarterly one leaves two.

    The common read: The auditor samples, so a two-day gap on one account is a needle in a haystack and the sample will miss it.

    Say how many instances of your quarterly access review exist inside a six-month window, then read on. Two. No sample size is prescribed for a SOC engagement, and the sizes you will see come from the AICPA's audit sampling guidance applied by analogy: a quarterly control with four instances a year gets about two tested, a monthly one gets two to four, a weekly one gets five to nine. A daily system-generated check gets tested against something much closer to the whole population, because that population is a query.

    That asymmetry runs the opposite way to the intuition it replaces. Continuous monitoring is sold as the thing that keeps you clean, and it is also the thing that makes a two-day gap findable: it emits one dated row per account per day, so a two-day gap is two of the 181 rows that account contributes rather than a needle in anything. The sparse control is the one whose deviations hide, and the sparse control is also the one where a single miss is half the population.

    The version of this that reaches a report is quieter than a breach. It arrives as a request for the population of privileged access grants for the period, met with a chat channel and console screenshots, and the sample that gets drawn is drawn from whatever the team could enumerate. The exception that follows names the access review, not the fact that nobody could produce a list with a date field on it.

    So the cheap thing to know per control is its cadence and its population size inside the window, and the two together tell you where a single miss is fatal. A quarterly control with two instances has no room: one missed cycle is half the sample. A daily control has room and no cover, because every day of it is a row somebody can count. Neither of those is an argument for fewer checks. Both are arguments for knowing which of your controls cannot survive one bad week.

  • Fieldwork is not one date. Some of it happens while the window is open.

    The common read: Nobody is looking until fieldwork opens, so a gap I close before the auditor arrives is a gap nobody saw.

    Put a date on the day the auditor starts looking, then read on. There is more than one. Some controls leave no evidence that can be tested later, so the service auditor may test them at various times throughout the reporting period, and interim procedures inside the window are routine. The ones that qualify are the ones whose trace does not survive: an observed walkthrough, an approval that lives in a chat channel, a log stream whose retention is shorter than the window.

    Two things follow, and the second is the one nobody prices. Mid-window testing does not shorten the period or freeze it early, because the period rather than the visit is what gets opined on. And a gap you closed in week two was not necessarily unobserved: if the monitoring feed the auditor pulls at the April visit covers January, January is in front of them in April, nearly three months before the environment gets its final look.

    That arrives as an evidence request in April for a control the team was still planning to stand up in May, answered honestly and quoted back in the test-results section with the word deviation on it, filed against the engineer who answered rather than against the model that said the audit had not started. So ask which controls the engagement team tests at interim, and write the answer beside those controls in your matrix. One email buys the list of controls whose audience arrives in April.

  • The window starts where the artifacts start. A kickoff date is not a start date.

    The common read: The evidence pipeline has to be running by the time the auditor asks, so switching it on at kickoff covers the window we declared.

    Your dashboard is green and it is green about April onward. The declared window opens 2026-01-01 and the first row the exposure table can produce is dated 2026-04-12, which is the day the compliance platform was connected. Eighty of the window's 181 dates carry an exposure row. The other 101 carry nothing, and no query written after the fact turns a date with no artifact into a date with one.

    The MFA snapshots cover all 181, which is exactly what makes this expensive to spot. Coverage is a fact about one control and its own table, so a report holding a control instrumented before the window opened and a control instrumented four months into it has two left edges, and the one you can defend for the whole report is the later of them unless you drop the newer control out of scope.

    Backfill is the move everyone reaches for and it is worth being precise about what it can do. A configuration history in the cloud provider can often reconstruct what a security group ALLOWED on a January date, which is evidence of state rather than the artifact the control was described as emitting, and a control described as a daily review of an exception report has no state to reconstruct at all. Reconstructed evidence gets argued; emitted evidence gets sampled.

    So the number to have per control is one date: the first date its own artifact exists, taken from the artifact and not the vendor's onboarding email. Operating effectiveness cannot be concluded for dates the population does not cover, so you open the window on the latest of those dates or the auditor raises January. It arrives as an evidence request for month two, answered with a screenshot taken in July, filed against the monitoring platform for not having the data. That is cheaper in March than in August.

  • Every deviation prints. Printing is not the opinion.

    The common read: One miss in a sample of eight is inside the rate the sample allows, so a single deviation with a documented cause and a dated fix costs nothing.

    You sent a cause and it bought testing rather than closure. What the auditor does with a deviation is investigate its nature and cause, then decide one of three things: that it is within the expected rate and acceptable, that more testing is needed, or that the control did not operate effectively throughout the period. Your explanation is the input to that decision, and a cause that reaches instances the sample never drew argues for the middle one.

    Two teams can report one deviation and be in different engagements. One missed week with a named reviewer, a dated catch-up and no other affected instance is a rate question. A filing job that stopped writing on 2026-06-01 with nothing watching it is a population question, and it converts one sampled deviation into a request for every weekly record from that date on, which is the moment the count stops being the unit. The count was never the unit. The cause is.

    The deviation prints either way. Materiality is not applied when the results of tests are described, and the report carries identified deviations with their number and nature even where the auditor concludes the control operated effectively. The cleanest statement of that rule is AT-C 320 paragraph .A30, which is the SOC 1 section; the AICPA SOC 2 guide carries the same rule in practice, so take it as the practice and not as a SOC 2 citation. A clean opinion and a printed deviation are not in tension.

    What the auditor does not vouch for is the sentence beside the deviation. The response is management's own, printed beside the test result at many firms, and it is the part a buyer's security review reads. That arrives months later as a questionnaire asking whether the audit was passed, filed against the deviation, when the part nobody could check was a response reading "the issue was promptly remediated". So write it as an engineer: name the cause, the date the fix landed, and what stops the cause next time.

  • There is no reset. There are three prices.

    The common read: A period that failed can be re-run, so the fix plus a fresh window replaces the exception.

    No mechanism restarts an observation period after a deviation. What people call a reset is always one of three trades. Take the exception in the report and answer it. Move the left edge past the failure and declare a window that opens after it. Accept a modified opinion, run a fresh period, and pay for the second engagement. The first is what almost everyone does, and it is also the one nobody plans for.

    The second trade is the one worth pricing, because it is the one that sounds free. Moving the left edge past 2026-01-14 costs one of two things. Hold the six months and the window closes 2026-07-31, so the report arrives roughly a month late. Hold the close date and the window is five months, and buyers read window length: a report covering three months answers a smaller question than one covering twelve. Practice clusters between three and twelve months, so a short window is legible, not invisible.

    One move is worse than the exception it hides. Backdate an approval, edit a completion record, regenerate an export with a friendlier timestamp, and the deviation is gone. It arrives as a ticket to correct a timestamp, filed against the export job, and nothing is filed against the approval whose date moved. A deviation that resulted from fraud by service organization personnel sends the auditor back to whether the description is fairly presented and whether the controls are suitably designed at all. One exception becomes the whole report.

    So the deliverable after a deviation is a paragraph and a date, not a new period. Write the cause in the words the auditor will have to defend, attach the artifact that shows the fix landed and when, and let the exception print. A single deviation with a credible cause and a dated fix does not usually cost the opinion, and practitioner consensus is the only authority behind that sentence, which is precisely why the sentence you write next to it is worth an hour.