How closing a survey early works - and why we may hold back data (for Math nerds)

This FAQ explains the formal rule behind closing a survey early at Salary Confidential.

It is tightly connected to our "Safe batches of three" rule, which is explained on the math side in this FAQ item

Background: what early close is solving

Under normal operation, survey results are released incrementally to the requester (owner) using our safe batches of three logic. That logic answers the question:

Is it safe to release the next batch of results right now?

Specifically, it evaluates whether releasing a batch at a given moment would leave behind an unsafe remainder of responses — for example, a final group that is too small to preserve anonymity.

Early close answers a different question:

Given what the requester has already seen, is it safe to finalize the survey now?

The key difference is that early close must account for history — specifically, which responses have already been revealed to the requester through partial releases.

Depending on how many responses have been collected since the requester’s last released batch, we may or may not be able to safely include additional data when a survey is manually closed early.

This is what the early close rule evaluates.

---

Definitions

Let:

• $N$ = total survey size (paid / expected responses)
• $r$ = total responses received at the moment the survey is closed early
• $m$ = number of responses already released to the requester via normal batch logic
  (by construction, $m \in \{0, 3, 6, 9, \dots\}$ and $m \le r$)

Let:

$$u = r - m$$

where $u$ is the number of responses that exist but have not yet been released.

Minimum realized survey size

Before any early close can occur, we enforce:

$$r \ge 4$$

This guarantees that no final report can ever be based on fewer than four responses — our minimum survey buy size — even if those responses form a complete batch of 3.

This prevents the edge case of “buy 4 slots, close at 3” or “buy 25 slots, close at 3”. The minimum buy size of surveys — and their reports — is there to discourage any attempts to create a triangulation trap for a specific individual. We don't let the early close feature be a way around this principle.

The early-close rule

Given that $r \ge 4$, the early-close inclusion rule is:

$$\text{Include unreleased responses } u \quad \iff \quad u = 0 \;\; \text{or} \;\; u \ge 3$$

Equivalently:

$$\text{Hold back unreleased responses } u \quad \iff \quad u \in \{1, 2\}$$

There are no other possible cases.

Why this rule exists

1. Preventing new singleton or pair exposure

By the time early close is evaluated, the requester has already seen $m$ responses via partial releases.

If closing early were to reveal only 1 or 2 new responses on top of that, those responses would be trivially attributable.

Therefore:

• revealing $u \in \{1,2\}$ is unsafe
• revealing $u \ge 3$ is safe

2. Consistency with safe batch release

This rule is not a new privacy model.

It extends the same invariant used by normal batch release:

No release step may expose fewer than three newly revealed responses.

Early close is simply a final release step, evaluated under the same constraint.

3. Why “exactly 3” is sometimes safe and sometimes not

A subtle point:

• If $u = 3$ and $m > 0$, the three unreleased responses were withheld earlier to avoid creating a remainder of 1 or 2 at full completion. Releasing them at early close is safe.
• If $u = 3$ and $m = 0$, then $r = 3$, which violates the minimum realized size rule $r \ge 4$. Early close is therefore not allowed in that case.

This is why the minimum size condition matters.

Relationship to the normal release function

The normal batch release function determines when partial results unlock:

$$R(r, N) = 1 \quad \text{iff} \quad (r \ge 3) \;\land\; \Big((r \bmod 3 = 0 \land N - r \ge 3) \;\lor\; (r = N)\Big)$$

Early close is evaluated after this function has been applied as far as possible.

That is:

• all releases permitted by $R(r, N)$ have already occurred,
• early close only evaluates the fate of the remaining tail $u = r - m$.

Why early close is history-aware

Normal batch release is state-based: it depends only on the current values of $r$ and $N$.

Early close is state + history-aware:

• respondents never see partial data,
• but the requester does.

Therefore, privacy must be evaluated relative to what the requester has already learned — which is why the rule depends on $m$, not just $r$.

Summary (TL;DR for nerds)

A survey can be closed early iff:

$$r \ge 4 \quad \text{and} \quad (r - m) \notin \{1, 2\}$$

Where:

• $m$ is the number of responses already released via batch logic.

If this condition fails, the survey may still be closed — but the unsafe remainder is permanently withheld from the final report.

What happens to held-back respondent submissions?

We discuss this in our main (non-technical) FAQ on early close surveys. But the simple thing: held-back data is locked away forever.

This doesn't change access given to respondents — whether theirs was part of the held-back data or not. If a respondent received a results key, it will be valid and they get access to the report. They participated in good faith, so they get access to the result report as participating peers.

Why this matters

This rule ensures that:

• no respondent is exposed through a late singleton or pair,
• requesters cannot infer individual responses from timing or order,
• and the privacy guarantees of Salary Confidential remain intact even when surveys end early.

Early close provides flexibility — without relaxing the math that keeps people safe.