Row Direct Match Protection

The Row Direct Match Protection metric evaluates a simple attack scenario by checking the number of unique rows (records) that are identical between the original data and the synthetic data. The goal is to measure how distinct the synthetic data can be from the original data, ensuring that the original records cannot be directly re-identified. We focus here on unique records, as duplicates can be considered protected with respect to the singling out criterion.

Interpreting Row direct match protection

The Row Direct Match Protection is expressed as a percentage and returns the proportion of records from the original data that are not exactly present in the synthetic data. A 100% score means that none of the original data records are present as is in the synthetic data, which is a positive indicator of privacy protection. On the other hand, low values indicate that some records from the original data are reproduced in the synthetic data, thus increasing the risk of re-identification.

Example

Imagine a dataset where the Row Direct Match Protection is 95%. This means that 95% of the records from the original data do not exactly appear in the synthetic data. In other words, only 5% of the records are exactly reproduced, which represents a reduced risk of re-identification. A low score (close to 0) would indicate that many rows from the original data are present in the synthetic data, which would compromise confidentiality.

Metric Specificity

This metric is particularly useful for verifying the reliability of the local generation model used by the anonymization method. The better the model, the fewer direct matches there should be between the original and synthetic data. Additionally, higher values for this metric are easier to achieve for datasets with a large number of columns, as the probability of replicating an entire row is reduced.

Formula

Let $U$ be the original rows that occur exactly once in the original data (duplicated rows protect each other and are left out), and $A$ the set of distinct avatar rows:

$$\text{Row Direct Match Protection} = 100 \times \left(1 - \frac{\#\{\, u \in U : u \in A \,\}}{n}\right)$$

A match requires equality on every column. $n$ is the total number of original rows.

How to Improve This Metric?

Improvement of the Row Direct Match Protection can be achieved by increasing the k parameter, which represents the number of neighbors considered during the generation of synthetic data. A higher k parameter helps increase the local modeling space for each synthetic data point, thereby increasing the likelihood of generating synthetic data that is different from existing data.