Closest Rate

The Closest Rate metric combines the results of the Distance to Closest and Closest Distances Ratio metrics to determine the percentage of individuals whose most similar avatar is not too close, and whose original data/avatar pair is not easily distinguishable from the rest of the dataset. It provides an overall measure of the dissociation between the original data and their avatars in the synthetic dataset.

Working half (WO), holdout half (HO), working avatars (WA)

Like the two metrics it combines, the Closest Rate is reference-based: it compares the anonymization (WO, the anonymized working half, vs its avatars WA) against the natural variability of the real data (WO vs a held-out reference HO, the untouched holdout half).

On datasets large enough (by default ≥ 500 rows) a real holdout is set aside before anonymization; on smaller ones an approximate holdout is drawn by randomly splitting the data when the metrics are computed. See the "Evaluation with a holdout reference" section in the Technical Assessment report.

Interpreting Closest Rate

The Closest Rate measures how well avatars are separated from the original data while ensuring that the original data/avatar pair is not easily distinguishable from other avatars in the dataset. This composite measure reflects both the distance between the individual and their closest avatar, as well as the distinction between the original data/avatar pair and the rest of the generated avatars.

Example

Consider a dataset with a Closest Rate of 85%. This means that for 85% of individuals, their most similar avatar is not too close to other avatars, and their original data/avatar pair is not easily distinguishable from other data points. In contrast, a low Closest Rate could indicate that a significant portion of individuals has avatars that are very similar or easily identifiable, which would increase the risk of re-identification.

Metric Specificity

The Closest Rate is particularly useful for providing a global measure of the anonymization quality in synthetic data generation. By combining the results of Distance to Closest and Closest Distances Ratio, this metric offers an overall view of the separation between the original data and their avatars, helping to assess the risk of re-identification.

Formula

Using the values of Distance To Closest ($d^{WA}_i$, $d^{HO}_i$) and Closest Distances Ratio ($r^{WA}_i$, $r^{HO}_i$), two thresholds are taken from the holdout reference, with $q = 5$ by default:

$$\tau_d = Q_{q\%}\big(d^{HO}\big) \qquad \tau_r = Q_{q\%}\big(r^{HO}\big)$$

An individual is at risk when both of its values are at or below these thresholds:

$$R = \left\{\, i \;:\; d^{WA}_i \le \tau_d \;\text{ and }\; r^{WA}_i \le \tau_r \,\right\} \qquad \text{Closest Rate} = 100 \times \left(1 - \frac{|R|}{n}\right)$$

$Q_{q\%}$ is the $q$-th percentile (midpoint interpolation) and $n$ the number of individuals for which a ratio is available.

How to Improve This Metric?

To improve the Closest Rate, it is recommended to increase the value of k, which represents the number of neighbors considered when generating the synthetic data. A higher k increases the separation between avatars and the original data, reducing the risk that avatars are too close to the actual data or easily confused with other avatars in the dataset.