Building a Golden Set for Production RAG: How to Curate, Refresh, and Use Evaluation Data That Keeps Up With Reality

The first version of your RAG system probably looked better in a demo than it did two weeks after launch.

In the demo, the prompts were clean, the corpus was stable, and the questions were representative mostly because the team unconsciously selected examples that the system handled well. In production, the story changes fast. Users ask for things in shorthand, refer to documents by old names, combine multiple intents in one query, assume tenant-specific context, and expect answers that reflect policy updates made yesterday afternoon. Meanwhile, the corpus changes, embeddings change, chunking changes, re-rankers change, prompts change, and one innocent metadata filter bug can quietly turn a high-performing assistant into a hallucination machine for one customer segment.

That is the moment when most teams realize they do not have an evaluation problem in the abstract. They have a data problem. More specifically, they do not have a golden set that reflects reality well enough to tell them when the system is improving, when it is regressing, and which subsystem is to blame.

This article is about how to build that golden set for a production retrieval-augmented generation system, and how to keep it useful after the first burst of enthusiasm wears off. I am not talking about a static benchmark assembled once for a launch review. I mean a living evaluation asset that captures real user behavior, edge cases, tenant isolation requirements, corpus drift, and answer expectations tightly enough to support release gates and ongoing LLMOps.

The practical goal is simple: when you change retrieval, chunking, metadata filters, re-ranking, prompting, model choice, or orchestration, you should be able to answer three questions with confidence:

Did overall quality improve?
Did any critical slice regress?
If it regressed, was the problem retrieval, grounding, answer generation, policy handling, or isolation logic?

A good golden set makes those questions answerable. A bad one produces false confidence.

The failure pattern: “our evals passed, but production still broke”

A common production incident goes like this.

A team improves latency and cost by moving from a larger embedding model to a smaller one, adjusting chunk sizes, and replacing a cross-encoder re-ranker with a cheaper bi-encoder stage plus heuristic scoring. Offline results look fine on a handful of manually created test prompts. The new version ships.

At first glance, nothing is obviously wrong. Average answer quality in ad hoc checks seems stable. But within a week:

Support tickets increase for “assistant gave the wrong policy answer.”
One tenant reports seeing content phrased from another tenant’s documents, or at least answers influenced by cross-tenant retrieval leakage.
Search success for newly uploaded docs drops sharply.
Multi-hop questions that require combining two policy sections now fail more often.
The system becomes more verbose and confident even when retrieval is weak.

What happened? Usually some mix of these:

The eval set was built from synthetic prompts or sanitized examples rather than real traffic.
Retrieval was graded only by answer correctness, masking the difference between good retrieval and a lucky generator.
There were no labels for tenant boundaries or authorization-sensitive cases.
Newly added documents were underrepresented, so freshness regressions were invisible.
There was no slice for ambiguous, under-specified, or jargon-heavy user queries.
The team tracked one headline metric but not failure modes by subsystem.

The naive instinct is to “add more test questions.” That helps, but only if the new questions are curated into a golden set with the right structure and maintenance process.

The pattern: a production golden set is a living, sliced, multi-label dataset

Teams often think of a golden set as a spreadsheet of questions and ideal answers. That is not enough for production RAG.

A useful golden set has to support diagnosis, not just scoring. That means each example should be rich enough to evaluate retrieval, grounding, generation, safety/policy behavior, authorization boundaries, and freshness. It also has to support segmentation, because the average hides exactly the incidents that matter in production.

In practice, the best production golden sets have five properties.

1. They are sourced primarily from real user demand

Synthetic data can help fill gaps, but the backbone should come from real traffic, real support issues, real search failures, and real business-critical workflows.

2. They separate retrieval quality from answer quality

If a model produces a plausible answer with weak evidence, your eval framework must still treat retrieval as bad. Otherwise you will ship systems that look smart until they face a question the model cannot bluff.

3. They are sliceable by business risk and failure mode

You want to know not just the overall score but performance for:

new-document freshness
tenant-specific questions
policy-sensitive answers
metadata-filtered retrieval
long-tail terminology
multi-hop synthesis
“no answer should be given” scenarios
access-controlled content

4. They include time as a first-class dimension

A static benchmark ages quickly in RAG. Documents change, naming conventions change, user behavior changes, and the model stack changes. Freshness and drift have to be designed in.

5. They are connected to release decisions

If the dataset is not wired into regression testing, release gates, and dashboards, it becomes an artifact people admire but do not use.

Why the naive approach fails

The most common failed approaches look reasonable at first.

Naive approach 1: handcraft 50 nice-looking prompts

This is the classic demo-to-prod trap. Handcrafted prompts are usually clearer, more complete, and less noisy than production traffic. They also reflect the authors’ mental model of the system rather than the users’ behavior.

As a result, they underweight:

shorthand and ambiguity
domain-specific abbreviations
references to stale naming
multipart questions
accidental context assumptions
adversarial or malformed input
user impatience, like “just give me the exact PTO carryover rule for contractors in Germany”

These prompts are not useless. They are just insufficient as the backbone.

Naive approach 2: grade only the final answer

If the answer is approximately right, teams often call the example a pass. But in RAG systems, you need to know whether the answer was right for the right reason.

Suppose the answer model knew a common policy pattern from pretraining and produced a reasonable response despite missing the current internal policy document. If you score only answer correctness, retrieval defects remain hidden until policy changes or a tenant-specific variant appears.

This becomes especially dangerous in enterprise settings where being grounded in the current approved corpus matters more than sounding correct.

Naive approach 3: use an LLM judge without human anchoring

LLM-as-judge is useful and often necessary for scale, but teams misuse it when they replace rubric design and anchor labeling with a single model score. Judge models can be overly generous, inconsistent on edge cases, or biased toward stylistic fluency over grounding.

A judge is best used as an amplifier of a human-defined rubric, not a substitute for one.

Naive approach 4: treat the dataset as static

The set that represented production in January probably does not represent production in April. New document types appear. User adoption spreads to new workflows. Tenants onboard with different terminology. Corpus freshness matters more over time. If you are not refreshing the set, your offline metrics drift away from operational truth.

Naive approach 5: optimize for one aggregate metric

One average score gives executives comfort and gives engineers very little else. In production, the business impact lives in slices. A two-point gain overall is not a win if policy-sensitive retrieval dropped ten points for one large tenant segment.

A better approach: build a layered golden set and score it at multiple levels

The architecture that works in practice is a layered evaluation dataset plus a pipeline that computes metrics at the query, slice, and release levels.

Think of the golden set as three connected artifacts.

Canonical examples: high-confidence human-reviewed evaluation records.
Shadow traffic samples: recent production queries with weaker or partial labels, used for drift detection and candidate promotion into the canonical set.
Challenge suites: intentionally difficult or high-risk scenarios, such as tenant isolation, freshness, no-answer, and policy conflict cases.

The canonical examples are your release-gate backbone. Shadow traffic keeps the backbone honest. Challenge suites ensure the business-critical edges are never washed out by averages.

Recommended dataset schema

At minimum, each golden-set record should contain:

query_id
query_text
query_timestamp
tenant_id or tenant class
user_role or access class if relevant
task_type such as lookup, synthesis, comparison, procedural, policy, troubleshooting
criticality such as low, medium, high
expected_behavior such as answer, abstain, ask clarifying question, cite, escalate
corpus_snapshot_id or document version references
authorized_doc_ids set if access control matters
must_retrieve_doc_ids where known
acceptable_doc_ids or relevant passage set
reference_answer or answer rubric
retrieval_labels such as precision/recall at k, top-k relevance judgments
answer_labels such as correctness, completeness, groundedness, citation accuracy
policy_labels such as no-PII leakage, no cross-tenant content, safe refusal behavior
slice_tags such as freshness, ambiguous, multi-hop, acronym-heavy, metadata-filtered
source_origin such as production log, support ticket, red-team, synthetic gap-fill
review_status
last_validated_at

Not every field will be filled for every example. But if your schema cannot express retrieval expectations separately from answer expectations, it is too weak for production use.

How to source high-signal examples

The question is not “where can we get test prompts?” The question is “which sources best predict future production pain?”

Here is the sourcing order I recommend.

1. Real production queries from logs

Start with actual user queries, not cherry-picked examples. Sample across:

top-frequency queries
high-value workflows
low-success sessions
queries with user reformulations
sessions that ended in escalation or abandonment
queries on newly ingested content
long-tail low-frequency but business-critical queries

Do not just sample uniformly. If you do, the set will be dominated by easy and repetitive requests. Use stratified sampling.

A practical starting distribution for canonical curation might be:

30% high-frequency user queries
20% support/escalation-derived failures n- 15% freshness-sensitive recent-doc queries
15% access-control or tenant-boundary scenarios
10% multi-hop/synthesis tasks
10% abstain/clarify/no-answer cases

Adjust based on your business.

2. Support tickets and incident reviews

Every support issue related to “wrong answer,” “could not find document,” “used outdated policy,” or “returned another customer’s data” should be a candidate source for a permanent eval example.

Incident-driven additions usually have excellent ROI because they represent failures with proven business impact.

3. Sales engineering and customer success call notes

These teams hear where the system disappoints before the platform team does. They often surface terminology mismatches, tenant-specific jargon, and workflow assumptions absent from product telemetry.

4. Search analytics and retrieval telemetry

Look for queries with:

low clickthrough on retrieved citations
rapid reformulation chains
low overlap between retrieved and clicked docs
empty or near-empty result sets
high answer latency followed by abandonment

These are often richer than simple thumbs-down signals.

5. Synthetic generation for gap filling only

Synthetic examples are valuable when used narrowly and explicitly:

generate paraphrases for known user intents
create controlled multi-hop compositions
produce adversarial no-answer scenarios
cover rare but required compliance cases

But synthetic data should fill coverage gaps, not define reality.

How to label retrieval quality

This is where many teams cut corners. Do not.

If retrieval is a core subsystem, label it directly.

Retrieval labels to capture

For each query, define one or more of the following depending on feasibility:

Must-retrieve documents: specific documents that must appear in top-k
Relevant document set: documents considered acceptable evidence
Relevant passage spans: exact chunk/span relevance if possible
Retrieval sufficiency: whether the retrieved set contains enough evidence to answer
Rank quality: whether the strongest evidence appears near the top
Filter correctness: whether metadata/tenant/access filters behaved correctly

For production RAG, retrieval sufficiency is often more useful than strict classical IR completeness. The operational question is usually: did the system retrieve enough of the right evidence, in the authorized scope, high enough in rank, to answer correctly?

How to make retrieval labeling tractable

Full passage-level judgments are expensive. Use a tiered approach.

Tier 1: high-risk examples get deeper labels

For high-criticality cases, annotate:

required documents
required passages
allowed evidence scope
forbidden evidence scope

This is essential for policy, legal, security, and tenant isolation scenarios.

Tier 2: medium-risk examples get document-level relevance

Label top-k retrieved documents as:

relevant
partially relevant
irrelevant
unauthorized
stale/outdated

Tier 3: lower-risk examples get sufficiency labels

Annotators answer:

Would this retrieved context be sufficient for a trustworthy answer?
Is any key evidence missing?
Is there evidence that should not have been returned?

This gives you scale without requiring exhaustive annotation everywhere.

Core retrieval metrics

Use several, not one:

Recall@k for must-retrieve docs
NDCG@k or MRR where ranking matters
Context sufficiency rate
Unauthorized retrieval rate
Fresh-doc retrieval rate
Filter accuracy by tenant/access slice

If your system uses a multi-stage retriever, also capture stage-specific metrics:

candidate generation recall
re-ranker promotion success
metadata filter precision/recall

That is how you find whether the issue lives in embeddings, ANN parameters, filters, or the re-ranker.

How to label answer quality

The answer label should not just be “good/bad.” It should reflect the behavior you want in production.

Answer rubric dimensions

A practical rubric includes:

Correctness: factually correct relative to source corpus
Groundedness: supported by retrieved evidence
Completeness: covers the requested scope without major omissions
Citation accuracy: cites the right document/section if citations are shown
Appropriate abstention: says “I don’t know,” asks clarifying questions, or refuses when warranted
Policy compliance: no unsafe disclosure, no cross-tenant leakage, no prohibited advice
Usefulness: concise and actionably phrased for the workflow

For each dimension, define a 0-2 or 0-3 scale with written anchor examples. Keep rubrics concrete enough that two reviewers can usually agree.

Reference answers versus rubric-only grading

You do not always need a single exact reference answer. In enterprise RAG, many questions permit multiple acceptable phrasings. What matters is:

required facts that must be present
optional facts that improve completeness
forbidden claims
expected behavior if evidence is insufficient

This is often better represented as a structured answer spec than a prose gold answer.

Example:

Must mention effective date
Must distinguish contractors from employees
Must cite current policy section 4.2
Must not speculate about local legal exceptions
If region is unspecified, should ask a clarifying question

That structure also improves LLM-judge reliability.

Human review, LLM judges, and hybrid labeling

Pure human review is expensive. Pure LLM judgment is brittle. The production pattern is hybrid.

A workable human-in-the-loop pipeline

Human experts define rubrics and label an anchor set.
An LLM judge scores candidate examples using those rubrics.
Humans review disagreements, high-risk slices, and low-confidence cases.
Agreement metrics are tracked between humans and the judge.
The judge rubric is periodically recalibrated as system behavior changes.

Where humans should stay in the loop

Keep human review mandatory for:

high-criticality policy content
legal/compliance domains
access-control and tenant-boundary cases
new slices where rubric ambiguity is high
examples used as release blockers

Where LLM judges help most

Use them for:

broad answer-quality scoring on medium-risk examples
detecting citation mismatch
classifying likely abstain-vs-answer cases
triaging shadow traffic for promotion to the canonical set
generating rationale snippets for reviewer efficiency

Model choices for judging

Larger models tend to produce more stable semantic grading, but cost and latency matter.

A practical pattern:

Use a stronger judge model for canonical set scoring and release-gate runs.
Use a cheaper model for nightly shadow-traffic triage.
Keep a human-reviewed calibration subset to detect judge drift.

If you are comparing models, do it empirically. Judge consistency matters more than leaderboard reputation.

Covering the hard slices most teams miss

A golden set is only as good as the slices it protects.

Tenant boundaries and authorization

If your platform serves multiple customers or internal audiences with differentiated access, this needs its own challenge suite.

Examples should include:

same query across different tenants with different expected answers
same doc title existing in multiple tenant spaces
users with partial access to a corpus
retrieval where the globally most similar doc is unauthorized
answer generation where retrieved evidence includes mixed-scope context by mistake

Metrics should include both retrieval leakage and answer leakage. Sometimes unauthorized content is not surfaced verbatim but still influences the wording. That is still a production incident.

Freshness and corpus drift

Production RAG lives or dies on whether it reflects the current corpus.

Maintain a slice explicitly built from:

recently added docs
recently changed docs
deprecated docs that should no longer be preferred
renamed policies and superseded guidance

Measure:

retrieval hit rate on docs added in the last N days
preference for latest version versus obsolete version
answer correctness on post-change queries

Ambiguous and under-specified queries

A robust assistant should not always answer directly. Sometimes it should ask a clarifying question.

Examples:

“What’s the reimbursement policy?” with missing geography or employee type
“How do I reset access?” with multiple systems matching
“Can contractors get this benefit?” where the benefit differs by program

Label expected behavior carefully. Incorrect direct answering should fail.

Multi-hop synthesis

Some RAG systems can retrieve documents fine but fail when the answer requires combining multiple sources.

Include cases that require:

policy + exception document
product spec + release note
procedure + entitlement matrix
troubleshooting guide + environment-specific note

Label not just whether both docs were retrieved but whether the final answer reconciled them correctly.

No-answer and refusal scenarios

Many teams underinvest here because users prefer direct answers. But in production, an assistant that confidently answers unsupported questions is costlier than one that abstains well.

Your set should include:

answer not present in corpus
answer would require prohibited disclosure
query refers to deleted or superseded material
request is too ambiguous without clarification

Tool-augmented cases

If your RAG stack invokes tools beyond retrieval, such as SQL, CRM lookups, or permission checks, your golden set should encode expected orchestration behavior too. For example:

retrieve policy, then verify user entitlement through a tool
ask a clarification before triggering a downstream action
refuse to summarize inaccessible records even if metadata suggests existence

Refreshing the golden set so it keeps up with reality

This is where most teams fail operationally. They build a dataset, present it in a QBR, and then it slowly becomes a monument.

Treat refresh as a pipeline, not an event.

A practical refresh cadence

Daily: collect shadow traffic, failure signals, and drift indicators
Weekly: sample candidates for review, especially from new docs and recent incidents
Biweekly or monthly: promote reviewed cases into the canonical set
Quarterly: rebalance slices, retire stale examples, recalibrate rubrics

The right cadence depends on corpus churn and release frequency. High-churn knowledge environments often need weekly canonical updates.

Promotion criteria from shadow to canonical

Promote examples when they are:

representative of recurring user behavior
attached to business-critical workflows
derived from an incident or support pattern
exposing a newly observed failure mode
tied to corpus changes that are likely to recur

Do not promote every weird query. Canonical sets should stay high-signal.

Detecting stale examples

An example should be flagged for revalidation when:

its referenced documents changed
its tenant policy changed
the expected behavior changed, such as now asking a clarification question
its answer depends on a deprecated process
repeated judge/human disagreement suggests rubric ambiguity

Store explicit document-version references or corpus snapshot IDs so you know which examples are affected by corpus updates.

Size guidance

Teams often ask how big the golden set should be. There is no universal answer, but a practical production baseline is:

200-500 canonical examples for initial release gating
1,000+ shadow samples rotating through nightly analysis
50-100 challenge-suite examples for each critical risk area such as tenant isolation or freshness

You want enough volume to detect slice regressions without making review impossible.

Wiring the golden set into release gates and LLMOps

A golden set creates value only when it changes behavior.

Evaluation layers in CI/CD

A robust production setup usually has three layers.

Layer 1: fast pre-merge checks

Run a smaller smoke subset on each PR for obvious regressions in:

retrieval filter behavior
top-k retrieval on must-have docs
abstain behavior for key no-answer cases
citation formatting and schema validity

This should be fast enough to run routinely.

Layer 2: pre-release regression suite

Before deployment, run the full canonical set plus challenge suites. Compare against the current production baseline.

Gate on:

overall non-inferiority or target improvement
no regressions above threshold on critical slices
zero tolerance for certain safety or tenant leakage failures

Layer 3: post-release shadow evals

After deployment, score fresh traffic samples and compare trends. This catches issues that your curated set missed.

Example release policy

A practical release gate could be:

Overall answer-quality score must improve by >= 1 point or remain within 0.5 points of baseline
Retrieval sufficiency must not drop more than 1 point overall
Freshness slice must not regress at all
Tenant leakage rate must remain zero
Policy-critical slice correctness must improve or remain unchanged
Any regression >3 points in a high-criticality slice blocks release pending review

The exact thresholds depend on metric variance and business tolerance.

Dashboarding and ownership

Assign owners by subsystem:

retrieval owner: recall, sufficiency, unauthorized retrieval, freshness
generation owner: groundedness, completeness, abstention quality
platform owner: latency, cost, reliability, permission enforcement
product owner: workflow usefulness, coverage of key journeys

Your dashboard should show:

current vs baseline by slice
distribution, not just average
top regressions with example drill-down
corpus freshness indicators
judge-human agreement over time

Architecture patterns that support maintainable eval data

The eval pipeline becomes much easier when the RAG architecture emits the right traces.

Instrumentation you should capture per request

For each production query, log:

normalized query text
tenant/user scope
retrieval candidates with scores
post-filter candidate set
re-ranker output
chunks passed into generation
cited chunks/docs in answer
model and prompt versions
tool invocations and outputs
final answer and any refusal/clarification path
user feedback if present

Without traceability, labeling becomes guesswork and regression diagnosis is slow.

Corpus versioning

You need some way to refer to the document state used for evaluation. This can be:

immutable document version IDs
corpus snapshot IDs
per-document content hashes

If you do not version the corpus, you cannot cleanly separate model regressions from data changes.

Recommended storage model

A practical setup:

canonical golden set in a versioned relational table or dataset registry
raw traces in an observability store or warehouse
labeled retrieval artifacts in a searchable annotation store
promotion workflow managed through review queues

Do not keep the whole thing in a manually edited spreadsheet beyond the earliest stage.

Cost and latency tradeoffs in evaluation design

Evaluation itself has a cost profile, and you should design for it.

Human review cost

Human annotation is expensive but often the highest-leverage spend because it creates durable assets and catches policy nuance.

Reduce waste by focusing humans on:

high-risk slices
anchor examples for judge calibration
disagreement review
incident-derived cases

LLM judge cost

Judge cost can become significant if you run full semantic grading on every nightly sample.

Control cost with:

tiered evaluation, using cheap heuristics first
small-model triage, large-model confirmation
partial rescoring only for changed components or affected slices
caching judge outputs for unchanged examples

Retrieval eval latency

If you are testing many retriever configurations, retrieval-only evals are often much cheaper and faster than full answer-generation evals. Use that to narrow candidates before running full-stack tests.

A practical experimentation flow:

Run retrieval metrics on 5-10 retriever configurations.
Keep top 2-3 candidates by recall/sufficiency and latency.
Run full generation evals only on finalists.

Model comparison strategy

When comparing answer models in RAG, remember:

Better models may mask retrieval weakness, which looks good until grounding matters.
Smaller models with stronger retrieval and better prompt constraints sometimes outperform larger models on enterprise trust metrics.
Cross-encoder re-rankers often improve quality but may blow latency budgets.
Larger context windows can reduce retrieval misses in some cases, but often increase cost and may introduce distraction from irrelevant chunks.

This is why the golden set needs separate retrieval and answer labels. Otherwise you will optimize the wrong layer.

An implementation blueprint

Here is a concrete way to stand this up over 6-8 weeks.

Phase 1: define schema and capture traces

Create the eval record schema
Ensure request tracing captures retrieval, filtering, context, answer, and versions
Identify business-critical slices and assign owners
Define answer and retrieval rubrics with examples

Phase 2: build the initial canonical set

Sample 300-500 production-derived queries stratified by slice
Add all major incident-derived examples from the last quarter
Create challenge suites for tenant isolation, freshness, abstain, and multi-hop
Human-label a calibration subset deeply

Phase 3: add hybrid grading

Implement retrieval metric computation from labels
Add LLM-judge scoring for answer dimensions
Measure human-judge agreement
Tune prompts/rubrics until disagreement on anchor examples is acceptable

Phase 4: wire into release gates

Add a fast smoke suite to PR checks
Add a full regression suite pre-release
Set slice-specific blocking thresholds
Publish dashboards with drill-down examples

Phase 5: operationalize refresh

Build daily shadow sampling from live traffic
Add automatic candidate selection from support incidents and drift alerts
Run weekly review queues
Version canonical set updates and note why examples were added or retired

Common mistakes to avoid

A few failure modes show up repeatedly.

Overweighting easy queries

If half your set is simple FAQ lookup, the benchmark will look stable while complex workflows degrade.

Mixing tenant scopes in labels

For access-controlled systems, “relevant” is not enough. You need “relevant and authorized.”

Letting reference answers become outdated

In fast-changing corpora, stale labels silently poison the eval.

Treating all failures as equivalent

A minor phrasing issue is not in the same class as cross-tenant leakage or outdated policy guidance. Encode severity.

Ignoring abstention quality

Many incidents happen because systems answered when they should have asked a question or refused.

No owner for dataset health

If no one owns refresh, rubric updates, and promotion policy, the golden set decays quickly.

What “good” looks like after a few months

You know the process is working when:

every major incident yields one or more new canonical examples
release discussions reference slice metrics, not just overall score
retrieval and answer regressions are separable in dashboards
newly uploaded or updated documents are visibly represented in evals
tenant and authorization challenge suites are treated as hard gates
shadow-traffic drift regularly produces useful additions to the set
teams trust the benchmark because it predicts production outcomes reasonably well

That last point matters most. The value of a golden set is not that it is academically elegant. The value is that it becomes operationally predictive.

Final takeaways

If you are running production RAG, your golden set is not a one-time benchmark. It is a living control surface for quality.

Build it from real user demand, not mostly synthetic prompts. Label retrieval separately from answer generation so you can diagnose failures. Cover slices that map to business risk: freshness, tenant isolation, authorization, ambiguity, no-answer behavior, and multi-hop synthesis. Refresh it continuously as the corpus and user behavior change. And most importantly, wire it into release gates so it can actually prevent regressions from shipping.

The teams that get this right stop arguing about whether the assistant “feels better.” They can point to concrete evidence: this retriever improved fresh-doc recall, this prompt reduced unsupported answers on ambiguous questions, this model swap saved 35% cost without hurting policy-critical slices, and this release is blocked because tenant-boundary retrieval regressed.

That is what mature RAG operations look like.

Not perfect answers every time. But a disciplined system for measuring the failures that matter, adapting the evaluation data to reality, and making quality visible before customers do.