Data quality framework

How OPDA defines, measures, and reports data quality for PDTF claims. Replaces the binary compliance flag of the current Conformance Scheme with a richer per-dimension profile that each member firm reports against quarterly via the Accreditation Directory.

1. Overview and scope

The Data Quality (DQ) framework gives OPDA, its member firms, and the consumers and regulators who depend on PDTF claims a shared way to talk about how good a claim is, separate from how strongly attested it is. The Conformance Scheme today answers a binary question ("is this firm compliant against this release?"). The DQ framework answers a richer one: across six standard dimensions, how does a given firm's output measure up, and how confident should a downstream user be in re-using it? The framework applies to every member firm that issues, holds, or verifies PDTF claims under the Conformance Scheme, and it produces a per-firm, per-dimension DQ profile that feeds into the Accreditation Directory on a quarterly cadence.

This framework is being drafted under ADR 0001 Wave 2, which commits OPDA to filling the gap identified in DAMA-DMBOK2 knowledge area #11 (Data Quality Management). The work is owned by the Compliance & Risk Working Group per governance.md §3.C, with Technical WG providing schema hooks and per-overlay validation rules. The framework's regulatory anchor is UK GDPR Article 5(1)(d) (accuracy), which requires personal data to be "accurate and, where necessary, kept up to date" — an obligation that maps directly onto two of the six dimensions below and which member firms already discharge as data controllers in their own right.

Three things this framework is not: it is not a re-implementation of any single firm's internal DQ regime (firms keep their own); it is not a binary pass/fail rebrand (the Conformance Scheme remains for the underlying compliance check); and it is not a league table. The Accreditation Directory ADR deliberately rules out a single overall maturity number, and the DQ framework inherits that constraint: scores are published per dimension, not collapsed.

2. The six DQ dimensions

OPDA adopts the six standard DQ dimensions named in ADR 0001 Wave 2: accuracy, completeness, consistency, timeliness, uniqueness, and validity. These are the conventional dimensions discussed in DAMA-DMBOK2's knowledge area #11 and recognisable to any data office that has run a DQ programme. For each dimension below, the framework gives a plain-English definition, anchors it on the property-data domain via concrete PDTF overlay examples, describes the shape of a measurement (without specifying numeric thresholds — those are WG decisions), and lists the common failure modes seen in property-transaction data.

2.1 Accuracy

Definition. Accuracy is the degree to which a claim correctly describes the real-world state it asserts. A claim is accurate when its content matches the underlying truth — the actual title owner, the actual EPC rating, the actual lease term. DAMA-DMBOK2 treats accuracy as the dimension that ties data back to its referent in the world, distinct from validity (which only asks "is the value well-formed?").

What it means for property data. Accuracy on a title claim (OC1) means the registered proprietor, title number, and charges match HM Land Registry's current position. Accuracy on an EPC claim means the rating, expiry date, and recommendations match the GOV.UK EPC register. Accuracy on a TA6 form depends on the seller's good-faith disclosure — a different kind of "truth" but the same dimension.

How to measure. Accuracy is measured by comparing the claim against an authoritative external reference where one exists (HMLR, EPC register, Companies House, Local Land Charges). Where no external reference exists, accuracy is measured indirectly via downstream-error rates — claims that were later corrected, disputed, or contradicted by a subsequent transaction event. The shape of the measurement is a ratio of agreeing fields to checked fields, weighted by field criticality.

Common failure modes.

• Stale copies — a title claim issued against an HMLR position that has since changed.
• Transcription drift — manual re-keying of EPC numbers, postcodes, lease terms introducing typos.
• Authority confusion — using a derived data source as if it were primary (e.g. a third-party EPC aggregator instead of the EPC register).

2.2 Completeness

Definition. Completeness is the degree to which a claim populates the fields that the applicable schema and overlays require, with meaningful values rather than placeholders. DAMA-DMBOK2 distinguishes structural completeness (required fields present) from semantic completeness (fields carry information, not just non-null tokens).

What it means for property data. Completeness on a BASPI v5 pack means every NTSELAT-required material-information field is populated for the property type in question — not "Not known" for fields that have a knowable answer. Completeness on a leasehold pack means the LPE1 and TA7 overlays carry the service-charge accounts, ground-rent terms, and management-pack documents the buyer's solicitor will demand.

How to measure. Completeness has a natural numeric shape: the ratio of populated required fields to required field count, evaluated against the merged schema for the overlays the claim asserts compliance with. The framework distinguishes "populated with a meaningful value" from "populated with a not-applicable / not-known token" — the former counts, the latter does not, unless the field's schema explicitly accepts the token as terminal.

Common failure modes.

• Bulk "Not known" fills used to satisfy required-field validation without supplying real information.
• Optional-field neglect — fields that are technically optional but carry the material-information signal a buyer needs.
• Overlay-mismatch — claim asserts compliance with an overlay (e.g. NTS2) but omits fields that overlay adds on top of NTS.

2.3 Consistency

Definition. Consistency is the degree to which a claim agrees with related claims — whether within the same transaction (e.g. the title proprietor named in OC1 matches the seller named in TA6), across the same firm's claims over time (e.g. the same property described identically across two listings), or across firms (e.g. two issuers reporting the same EPC). DAMA-DMBOK2 treats consistency as a cross-cutting dimension: it can only be measured by comparing two or more claims.

What it means for property data. Consistency on a property pack means the address, UPRN, and registered owner reconcile across every overlay in the pack. Consistency across firms means that when two member firms report a claim about the same property within the same window, their reports agree on the facts both purported to observe. Consistency over time means the same firm's claim about the same property doesn't silently change between releases without an audit trail.

How to measure. Consistency is measured by sampling pairs (or groups) of claims that ought to agree and computing the rate at which they do. The shape of the measurement is a pairwise agreement ratio over a chosen reconciliation key (UPRN, title number, EPC number, transaction ID). For within-pack consistency, the reconciliation is automatic from the schema; for cross-firm consistency, it requires a published reconciliation protocol.

Common failure modes.

• Address normalisation drift — same property keyed by different normalised addresses across overlays.
• Silent revisions — a claim's content changing without an audit-trail entry, so downstream users can't see what moved.
• Overlay collision — two overlays in the same pack carrying conflicting values for an overlapping field.

2.4 Timeliness

Definition. Timeliness is the degree to which a claim reflects the current state of the world at the moment it is consumed. A claim can be perfectly accurate when issued and stale by the time a verifier checks it. DAMA-DMBOK2 distinguishes "currency" (how recently issued) from "timeliness as consumed" (how recently relative to the consumer's need) — OPDA conflates these into a single dimension because PDTF claims carry issuance dates explicitly.

What it means for property data. Timeliness on an EPC claim means the certificate is within its statutory validity window (typically ten years from issue). Timeliness on a title claim means the OC1 snapshot postdates any known charge or transfer event. Timeliness on a search result (CON29R, LLC1, CON29DW) means the search was issued within the conveyancing-industry-accepted "fresh" window — usually three to six months, though this is a moving target.

How to measure. Timeliness is measured as the time elapsed between the claim's issuance and either (a) its consumption, or (b) a fixed reference moment such as the quarterly Directory publish. The shape of the measurement is a freshness window in days, compared to a per-claim-type policy. Claims that exceed the policy window are not "wrong" — they are flagged stale and excluded from the currency calculation.

Common failure modes.

• Cached re-issuance — a firm re-issues a claim with a new VC date but stale underlying content.
• Missing observation timestamp — a claim carries an issuance date but no source-observation date, so consumers can't compute true freshness.
• Statutory expiry creep — EPCs and certain searches have hard validity windows that consumers may overlook.

2.5 Uniqueness

Definition. Uniqueness is the degree to which a claim is the sole authoritative statement about its subject within scope. DAMA-DMBOK2 treats uniqueness as the absence of duplicates, near-duplicates, and shadow records that should have been consolidated. For PDTF, the scope is usually the (firm, property, claim-type) triple.

What it means for property data. Uniqueness on a firm's listing data means there is one canonical claim per property per claim type, not three nearly-identical OC1 issuances or duplicate BASPI packs created by separate teams. Across firms, uniqueness is not enforced — multiple firms legitimately observe the same property — but within a firm, duplicates degrade trust and complicate reconciliation.

How to measure. Uniqueness is measured as one minus the duplicate ratio: the count of non-duplicate claims divided by the total claim count, over a sampling window. Duplicates are detected via a reconciliation key per claim type (typically UPRN + claim-type, narrowed by transaction context where applicable). The shape of the measurement is therefore a deduplication ratio, with the duplicate definition itself encoded per claim type.

Common failure modes.

• Shadow records — a property held in two systems within the same firm with no canonical link.
• Re-issuance loops — a firm reissues a claim for technical reasons (key rotation, schema migration) without retiring the prior issuance.
• Identifier ambiguity — the same property keyed by both UPRN and address with no resolution.

2.6 Validity

Definition. Validity is the degree to which a claim conforms to the syntactic and semantic rules of its schema and overlays. A valid claim parses, passes JSON Schema validation against the merged base + overlay schema, and respects any SHACL or business-rule constraints layered on top. DAMA-DMBOK2 calls this dimension out as the "structural correctness" check that is necessary but not sufficient for accuracy.

What it means for property data. Validity on any PDTF claim means it validates against the base pdtf-transaction.json schema and every overlay it asserts compliance with — for example, an NTS2 + TA6 claim must pass merged-schema validation including discriminator branches for yes/no fields, required-when-yes details, and enum constraints on tenure types. Validity is the dimension Technical WG owns most directly via the schema hooks called out in ADR 0001.

How to measure. Validity is measured by running the official PDTF validator (per the overlays README) against a sample of the firm's claims. The shape of the measurement is the fraction of claims that pass validation cleanly versus those that pass with warnings or fail. The validator's decisions are deterministic and reproducible, which makes validity the most automatable of the six dimensions.

Common failure modes.

• Schema-version drift — claims issued against an older overlay version still circulating after a migration window has closed.
• Discriminator-branch errors — yes/no fields where "yes" is asserted but the required follow-on details are missing.
• Enum violations — free-text values supplied where an enum is required (most often in tenure, EPC rating, planning categories).

3. Per-claim-type measurement framework

The six dimensions are framework-level; the actual measurement happens per claim type. A title claim (OC1) and a TA10 fittings-and-contents form have very different "right answers" for accuracy, completeness, and timeliness — and the evidence supporting a measurement differs too. The framework therefore requires each claim type to define its own measurement protocol, derived from the six dimensions but parameterised for the claim's subject matter, authoritative sources, and statutory constraints.

3.1 Proposed protocol shape

Each per-claim-type protocol must specify, at minimum:

• Which dimensions apply — most claims will use all six, but some are dominated by a subset (e.g. a CON29R search is mostly timeliness + validity; uniqueness is rarely a meaningful concern).
• Sampled fields per dimension — not every required field contributes equally; the protocol names the critical fields whose state drives the score.
• Authoritative reference — for accuracy, the named external source (HMLR, EPC register, Companies House, Local Land Charges, etc.) against which agreement is checked.
• Reconciliation key — for consistency and uniqueness, the field combination used to identify "the same claim" across instances.
• Freshness window — for timeliness, the days-from-observation policy.
• Evidence supporting the measurement — the artefact a firm must retain (audit log, sample export, validator output) to back the score in a spot-check.
• Sampling cadence and sample size — how often the protocol is run, against what fraction of the firm's claim population.

Protocols are version-controlled alongside the claim-type schemas, so a change to a protocol is a documented event with a release tag. Member firms whose tooling automates DQ measurement can pin to a protocol version the way they pin to a schema version today.

3.2 PDTF v3 overlays — initial dimension relevance map

The table below lists the v3 overlays present in the schemas repository (per source/03-standards/schemas/src/schemas/v3/overlays/) with a draft note on which dimensions are most relevant to each. The relevance judgements are first-pass and to be confirmed by C&R WG with Technical WG input.

The point of the table is not to lock dimensions per overlay but to make the relevance call explicit so C&R WG can argue with it. Some overlays will turn out to merit dimensions not listed here at first pass; some listed dimensions may be downgraded in practice.

4. Assurance Level × DQ relationship

OPDA's existing Assurance Levels (AL1–AL4) describe how strongly a claim is attested — they are a property of the claim itself, set by the issuer's accreditation and the source the claim is drawn from. Per governance.md §5 and ADR 0001 Wave 2:

• AL1 — self-asserted data (low trust, no issuer accreditation).
• AL2 — issued by an accredited property professional (medium trust).
• AL3 — issued by an accredited issuer with extra controls, or by a trusted proxy on behalf of an official primary authority (medium trust, additional assurance).
• AL4 — issued by the official primary authority (high trust, legal validity).

The DQ framework is orthogonal: the six dimensions describe how good a claim is at the level it's attested at. A high-AL claim with poor DQ is still a real problem (a primary authority emitting stale title data); a low-AL claim with strong DQ is genuinely useful information (a self-asserted PIQ completed thoroughly and consistently).

The proposed integration is that each Assurance Level sets a floor for the DQ score per dimension. To claim AL3 or AL4 on a given claim type, the issuer's measured DQ profile for that claim type must meet or exceed the floor. The Accreditation Directory then publishes the actual scores above the floor — so a member firm can be AL3 with a strong accuracy score and a weak timeliness score, and the Directory shows both signals.

A worked sketch (illustrative, not policy): suppose the accuracy dimension is measured on a 0–6 scale. AL1 has no floor. AL2 floors at 2. AL3 floors at 4. AL4 floors at 5. A firm with AL3 accreditation on title claims whose measured accuracy comes out at 4.7 publishes "AL3, accuracy 4.7" in the Directory — the floor is met, and the score is exposed. A firm whose accuracy drops to 3.8 either loses AL3 on that claim type, or is given a remediation window, or its AL3 claims are flagged. The remediation pathway itself is a WG decision (see §7).

5. Reporting integration with the Accreditation Directory

Per ADR 0004, the Accreditation Directory is the publication surface for member firms' DQ profiles. The integration shape:

• Cadence. Quarterly publish, per ADR 0004 §4. Firms self-attest their DQ scores by the 15th of the quarter-end month; OPDA refreshes the Directory by the 1st of the following quarter.
• Capability bundle. DQ contributes a bundle of six capabilities (one per dimension) to the Directory's schema, per ADR 0004's "Resolving the parallel-execution cross-dependency" section. Each dimension is scored on the standard Engagement / Process / Evidence axes at 1–6, with evidence requirements that escalate per ADR 0004 §6.
• Per-firm rollup. The public Directory shows per-firm section averages including a "Data Quality" section; the drill-down view shows per-dimension scores and links to the supporting VC.
• What's public. Per ADR 0004 §3, public viewers see firm-level scores; the member-firm portal (deferred tier) carries the per-claim-type detail.
• Stale flag. Per ADR 0004 §4, firms missing two quarterly submissions are flagged stale until they refresh.

Because DQ is scored per-firm but produced by measuring per-claim-type, an intermediate aggregation step is needed: per-claim-type scores roll up to per-dimension firm scores by averaging across the claim types the firm asserts compliance with. The roll-up algorithm itself is a WG decision (simple unweighted mean, volume-weighted, or claim-type-criticality-weighted are all defensible).

6. Governance, roles, and review

The DQ framework's ownership and review machinery follows the existing OPDA operational structure in governance.md §3:

• Compliance & Risk WG owns the framework end to end (per ADR 0001 §"Resolved during review" #3). That includes the dimension definitions, the per-claim-type protocols, the AL × DQ floors, the remediation pathway, and the change-management process for the framework itself. C&R already owns UK GDPR Art 5(1)(d) compliance per governance.md §3.C, so DQ ownership sits inside an existing remit rather than creating a new one.
• Technical WG provides schema hooks and per-overlay validation rules — the validity dimension is mostly mechanised through their work, and the consistency/completeness dimensions rely on schema-level decisions about required fields and discriminators. Technical WG also commits to making the official PDTF validator available as a reference implementation member firms can run against their own data.
• Domain Data Stewards (per data-stewardship) are responsible for context-specific quality within each bounded context — flagging when a claim type's measurement protocol misfires for their domain, surfacing common failure modes, and proposing protocol revisions.
• Member firms self-attest their DQ scores per quarter and retain the evidence artefacts a spot-check would request (per ADR 0004 §6).

Review cadence. The framework itself is reviewed annually by C&R WG, with interim revisions handled via the standard governance.md §6 change-management SOP. Per-claim-type protocols can be revised more frequently as overlays evolve; a protocol revision is a versioned event with backwards-compatibility considered explicitly.

Dispute escalation. If a member firm disputes a spot-check finding, or two firms disagree about a cross-firm consistency measurement, the dispute escalates through the Conformance Scheme's existing dispute pathway. C&R WG adjudicates; unresolved cases escalate to the Executive Committee per governance.md §3.A.

7. Open questions for C&R WG kick-off

These are the substantive choices the framework cannot make on the WG's behalf. They are listed as questions rather than asserted positions so the WG kick-off has explicit decision points to walk through.

1. What numeric scales does the framework use per dimension? ADR 0004 specifies a 1–6 Engagement / Process / Evidence scale for capabilities in the Directory; do the DQ dimension scores themselves use the same 1–6 scale, or a 0–100 percentage scale that maps into 1–6 for Directory publication?
2. What are the AL floors per dimension per claim type? The framework proposes that AL sets a floor for DQ; the WG must fill in the matrix. Where the matrix is sparse (e.g. uniqueness on single-instance claims), can a floor of "not applicable" be set without breaking the AL × DQ relationship?
3. What is the remediation pathway for a firm that drops below an AL floor mid-quarter? A grace period, a published warning flag, an AL downgrade for new claims while existing claims keep their AL until expiry — all of these are defensible. The WG should pick one and document the rationale.
4. How are roll-ups computed? Per-claim-type to per-dimension firm scores; per-dimension firm scores to the section average shown publicly in the Directory. Simple unweighted mean, volume-weighted, criticality-weighted, or median?
5. What is the sampling protocol for high-volume claim types? Quarterly is too coarse for firms issuing claims at industrial scale. The WG should decide whether high-volume firms publish a continuous-sampling-derived quarterly score, a rolling window, or a separate cadence entirely.

Related

• Conformance scheme — the binary compliance regime this framework extends.
• Accreditation Directory — where DQ scores are reported.
• Data security framework — sibling Wave 2 workstream owned by the same WG.
• Overlay attachments policy — attachment quality is a DQ concern.
• Data stewardship — Domain Data Stewards' role in context-specific quality.

DAMA KA rubric — Wave 1 alignment

Per ADR 0001 Wave 1, KA-tagged pages follow a consistent rubric: Purpose · Activities · Deliverables · Roles · Metrics · Maturity. Sections below may be filled incrementally by Compliance & Risk WG — content authored above already maps to several of these; this scaffold is for the gaps.