How to Compare Two Google Result Sets with Jaccard Similarity

When two Google searches return different URLs, “how similar are they?” needs a defined unit and a reproducible calculation. Jaccard similarity is a useful starting point because it compares sets without pretending that their order is identical.

It works well for questions such as:

• How much did a result set change between two dates?
• Do two country settings surface the same sources?
• Does a translated query discover a different domain set?
• How much overlap exists between page one and page two?

The calculation

For two sets, A and B:

Jaccard similarity = size of (A ∩ B) / size of (A ∪ B)

A score of 1 means the sets contain the same members. A score of 0 means they share none.

Suppose each result set contains ten unique URLs and six occur in both. The intersection is six, but the union is fourteen:

6 / 14 = 0.429

The similarity is 42.9%, not 60%. Dividing by one set's size would answer a directional recall question rather than symmetric similarity.

A small implementation

function compareSets(valuesA, valuesB) {
  const a = new Set(valuesA);
  const b = new Set(valuesB);
  const intersection = [...a].filter((value) => b.has(value));
  const onlyA = [...a].filter((value) => !b.has(value));
  const onlyB = [...b].filter((value) => !a.has(value));
  const unionSize = new Set([...a, ...b]).size;

  return {
    similarity: unionSize === 0 ? null : intersection.length / unionSize,
    intersection,
    onlyA,
    onlyB,
  };
}

Returning null for two empty sets is often safer than declaring them perfectly similar. An empty pair might represent two valid no-result responses, two failed collections, or a filtering rule that removed everything. Interpret the collection outcome before assigning a score.

Define the member of the set

The arithmetic is simple; the definition is where most comparisons go wrong.

Possible set members include:

• Top-level result-block URLs
• Every URL found recursively in child blocks
• Unique normalized page URLs
• Unique registrable domains
• URLs from only a classified result type

These produce different answers. A parent result with four sitelinks can represent one top-level URL or five recursive destinations.

State the unit in the metric name. jaccard_unique_normalized_urls is much clearer than serp_similarity.

Normalize before comparing

Fragments and known tracking parameters can make one destination look like several. Preserve every original URL, then derive a conservative comparison key.

Do not normalize by:

• Deleting all query parameters
• Lowercasing paths
• Automatically merging www and apex domains
• Treating every redirect as permanent

If URL identity is uncertain, compute both page-level and domain-level similarity. Domain similarity is more tolerant of article-path changes, while URL similarity detects exact destination turnover.

Jaccard ignores order

These two result sets have Jaccard similarity 1:

A, B, C, D
D, C, B, A

Their members are identical even though their order is reversed. That is a feature when source presence matters and a limitation when placement matters.

Pair Jaccard with at least one ordered measurement:

• Overlap at k: shared members among the first k defined positions
• Mean absolute movement: average position change for shared members
• Entered and exited sets: destinations found only in the later or earlier snapshot
• Rank-biased overlap: an option when changes near the top should count more heavily

Be careful with the word “rank.” If the source data preserves visible blocks rather than classified organic listings, call the array position a visible-block position.

Compare like with like

Control the inputs before interpreting the score:

When the purpose is to measure change over days or weeks, time is intentionally different. The other fields should remain stable.

Aggregate across a query corpus

For multiple queries, calculate a score for each query before summarizing. Pooling every URL into one large set lets common domains dominate the result.

Report the median and distribution of per-query scores, along with the number of valid pairs. Keep missing or failed pairs out of the denominator under an explicit rule.

A useful row might contain:

{
  "queryId": "q-0042",
  "unit": "unique_normalized_url",
  "setSizeA": 12,
  "setSizeB": 11,
  "intersectionSize": 8,
  "unionSize": 15,
  "jaccard": 0.5333
}

Jaccard similarity does not say whether either result set is better. It says how much their defined members overlap. Reserp supplies ordered result blocks and nested URLs that can be transformed into either URL or domain sets; the API documentation describes that response structure.