AI Answer Variability Across Runs (Jaccard)

How to use

1. Paste the answers from repeated runs of one question, separated by a line of three dashes (---).
2. Enter the brands to look for, with other spellings after a vertical bar.
3. Read the overlap matrix: each cell is the Jaccard score of two runs, from 0 (no brand in common) to 1 (the same brands).
4. See which brands appear in every run and which appear once.

What it checks and how it works

For each pair of runs the tool takes the set of brands named in each and computes the Jaccard index: the number of brands in both runs divided by the number in either. A score of 1 means the two runs named the same brands; 0 means they share none. The mean, the lowest and the highest pair summarise the whole set, up to 12 runs.

The index looks only at which brands appeared, not at their order, wording or sentiment. Two runs that name no brand at all have no defined score and are left out of the mean. A low overlap on a few runs is a reason to run more, not a finding about an assistant. The result describes the pasted runs only.