> ## Documentation Index
> Fetch the complete documentation index at: https://docs.truthfactor.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Benchmark your mobile data

> Measure connectivity, name match, and profile match on your own phone cohort with the Signals API, then compare the results to your ground truth.

Run the same three-metric mobile data-quality benchmark TruthFactor uses internally, against
**your own** phone cohort. You submit a batch of rows, poll for the verdicts, compute three
headline rates, and — if you have labels of your own — compare them to your ground truth to
understand agreement and variance.

This guide uses the three **premium** phone signals. They answer the same questions as the
standard signals but resolve more of your rows, so fewer come back unresolved — which is what
you want when you are measuring a rate. Premium signals need a premium grant on your account;
see [Signals](https://docs.truthfactor.ai/essentials/signals.md).

## What the benchmark measures

Three signals, three metrics. Each metric is a rate over the rows that returned a **definite**
verdict — rows where TruthFactor returned `trusted: null` are excluded from that metric's
denominator (see [the verdict model](https://docs.truthfactor.ai/essentials/verdicts.md)).

| Metric | Signal | Definition (over the resolved subset) |
| - | - | - |
| Connectivity | `premium_phone_connectivity` | `connected / (connected + not_connected)` |
| Name match | `premium_phone_name_match` | `match / (match + no_match)` |
| Profile match | `premium_phone_profile_match` | `true / (true + false)` |

In verdict terms, every metric is `#(trusted = true) / #(trusted is true or false)` for that
signal. A `trusted: null` row neither counts for nor against the rate — it simply did not
resolve, and is dropped from that metric only.

## Before you start

**1. An API key with the three premium signals enabled**

An admin of an active account mints the key from the dashboard; the `tf_live_…` secret is
shown once. Confirm the key is live and pointed at the right account:

```bash theme={null}
curl -sS https://api.truthfactor.ai/api/v1/whoami \
  -H "Authorization: Bearer $TF_API_KEY"
```

See [Authentication](https://docs.truthfactor.ai/essentials/authentication.md). If a signal comes back rejected for
want of a grant, your account does not yet have the premium tier — contact your account
manager.

**2. Credits**

Budget roughly **1 credit per returned verdict**. `rows × signals` is the ceiling; rows
that do not resolve a given signal cost less.

**3. Your cohort as a CSV**

One row per phone. The fields you supply decide which signals can run:

| Field | Format | Needed for |
| - | - | - |
| `phone` | E.164, e.g. `+14155550148` | **All three signals** (required) |
| `first_name`, `last_name` | plain text | **Name match** |
| `profile_url` | LinkedIn URL or slug, e.g. `linkedin.com/in/janedoe` | **Profile match** |
| `email` | address | Optional — increases profile-match coverage (see [coverage](#coverage-how-much-input-you-need)) |
| `correlation_id` | your own row id | Optional — echoed back verbatim so you can join results |

> [!WARNING]
>
> **Some phone numbers may receive an automated verification call.** As part of the check, some
> numbers receive a single automated verification call; others are verified without a call. Only
> benchmark numbers it is acceptable to call. At most **one call is placed per number**,
> regardless of how many signals you request.

## 1. Submit the batch

`POST /api/v1/signals` with up to **5000 inputs** per batch. Set `skip_cache: true` so you
measure a fresh resolution rather than a cached verdict.

```bash cURL theme={null}
curl -sS -X POST https://api.truthfactor.ai/api/v1/signals \
  -H "Authorization: Bearer $TF_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "signals": [
      "premium_phone_connectivity",
      "premium_phone_name_match",
      "premium_phone_profile_match"
    ],
    "skip_cache": true,
    "inputs": [
      {
        "correlation_id": "row-1",
        "phone": "+14155550148",
        "first_name": "Jane",
        "last_name": "Doe",
        "profile_url": "linkedin.com/in/janedoe",
        "email": "jane@example.com"
      }
    ]
  }'
```

```python Python theme={null}
import os, requests

resp = requests.post(
    "https://api.truthfactor.ai/api/v1/signals",
    headers={"Authorization": f"Bearer {os.environ['TF_API_KEY']}"},
    json={
        "signals": [
            "premium_phone_connectivity",
            "premium_phone_name_match",
            "premium_phone_profile_match",
        ],
        "skip_cache": True,
        "inputs": [
            {
                "correlation_id": "row-1",
                "phone": "+14155550148",
                "first_name": "Jane",
                "last_name": "Doe",
                "profile_url": "linkedin.com/in/janedoe",
                "email": "jane@example.com",
            }
        ],
    },
)
request_id = resp.json()["request_id"]
```

You get `202` straight away:

```json theme={null}
{
  "request_id": "rq_8efcd577456f4bd1",
  "requested_datetime": "2026-10-26T14:03:11Z",
  "status": "pending",
  "accepted": 1
}
```

> [!NOTE]
>
> The run is **asynchronous**. A large batch that places automated verification calls can take
> up to **\~an hour** to complete. Submit the whole cohort as one batch and poll gently.

## 2. Poll for results

```bash theme={null}
curl -sS https://api.truthfactor.ai/api/v1/signals/rq_8efcd577456f4bd1 \
  -H "Authorization: Bearer $TF_API_KEY"
```

`status` stays `pending` and `completed_datetime` is `null` until every item is terminal. The
batch is **done when `completed_datetime` is non-null** (`status` flips to `complete`). Each
`data[]` item carries your `correlation_id`, the echoed `input`, an item `status`, and a
`signals` map of `{trusted, confidence}` per signal:

```json theme={null}
{
  "request_id": "rq_8efcd577456f4bd1",
  "requested_datetime": "2026-10-26T14:03:11Z",
  "completed_datetime": "2026-10-26T14:41:52Z",
  "status": "complete",
  "data": [
    {
      "correlation_id": "row-1",
      "input": { "phone": "+14155550148", "first_name": "Jane", "last_name": "Doe" },
      "status": "scored",
      "signals": {
        "premium_phone_connectivity":  { "trusted": true,  "confidence": 99 },
        "premium_phone_name_match":    { "trusted": true,  "confidence": 97 },
        "premium_phone_profile_match": { "trusted": false, "confidence": 90 }
      }
    }
  ]
}
```

A signal that could not resolve for a row comes back as `{ "trusted": null, "confidence": null }`
(an unresolved item also carries an `unresolved_reason`). Exclude those from that metric's
denominator — see [the verdict model](https://docs.truthfactor.ai/essentials/verdicts.md) for why `null` is never a `false`.

> [!TIP]
>
> `correlation_id` is yours — pass whatever identifies the row on your side and it comes back
> untouched, so you never match on array position when you join results to your ground truth.

## 3. Compute the metrics

For each signal, over the rows with a definite (`true`/`false`) verdict:

* **Connectivity** = `#(trusted = true)` / `#(trusted = true or false)`
* **Name match** = `#(trusted = true)` / `#(trusted = true or false)`
* **Profile match** = `#(trusted = true)` / `#(trusted = true or false)`

To compare against **your own** ground truth, join on `correlation_id` and, per signal, build a
confusion matrix of the TruthFactor verdict against your label:

* **Agreement** — the share of rows where both sides have a definite verdict and they match.
  Report it on that definite overlap only.
* **Variance** — the difference between TruthFactor's headline rate and your own on the same
  cohort.
* Decide **up front how you treat blank or unknown ground-truth cells** — counting them as a
  negative versus excluding them changes the rate materially. Be consistent across all three
  signals and state the choice alongside the result.

## Coverage — how much input you need

A metric is only meaningful if enough rows actually resolve. **Coverage** is the share of
inputs that came back with a definite (non-null) verdict for a signal — and it varies a lot by
signal. The figures below are from a real **250-row North America benchmark run**:

| Signal | Coverage | What it means |
| - | - | - |
| `premium_phone_connectivity` | **\~95%** | Nearly every number resolves. |
| `premium_phone_name_match` | **\~92%** | Most numbers resolve a name verdict. |
| `premium_phone_profile_match` | **\~56%** | Much lower — size your input accordingly (see below). |

Profile match is the one to plan around. **Providing an `email` address on each row increases
profile-match coverage** substantially; without emails, profile match returns `null` on roughly
**44%** of rows in this cohort.

### Size the input for the resolved sample you need

Your usable sample for a metric is `rows × coverage`. Size the input so the **resolved** count —
not the submitted count — is large enough to be statistically meaningful. Because profile match
resolves at \~56% versus connectivity's \~95%, you need **roughly 2× the rows** to reach the same
resolved-sample size:

| Resolved rows you want | Connectivity (\~95%) | Name match (\~92%) | Profile match (\~56%) |
| - | - | - | - |
| 200 | \~211 inputs | \~218 inputs | \~358 inputs |
| 500 | \~527 inputs | \~544 inputs | \~893 inputs |

> [!NOTE]
>
> These rates are from a **North America** cohort and **vary by region** — coverage and latency
> differ from one region to the next. TruthFactor will publish per-region coverage rates as more
> benchmarks run. Treat the figures above as a planning guide, and measure your own cohort's
> coverage from the first run.

## Caveats

* **Name match needs a real first and last name.** A LinkedIn slug alone is not enough — enrich
  names before you submit, or name match will resolve on far fewer rows than connectivity.
* **Profile match has a thin denominator.** A verdict exists only where the profile exposes the
  data to compare against, so report it **directionally** on small cohorts and supply `email` to
  lift coverage.
* **Region changes coverage.** Coverage and latency vary from one region to the next, so
  measure your own cohort's coverage rather than assuming the North America figures.
* **Use `skip_cache: true`** for a benchmark, so you measure a fresh resolution rather than a
  cached verdict from an earlier run.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.