Methodology · Version 2.0 · 2026-10-11
How we research U.S. certification and licensing exams
Abstract
Information about smaller U.S. certification exams is spread across candidate handbooks, fee schedules, exam blueprints, federal regulations, labor statistics and scattered vendor pages. Most of it has never been gathered into one place. We built a research pipeline that pairs frontier large language models (with live web search and page retrieval) with fixed research briefs, a ranked list of trusted sources, required fields per guide and automated validation. With it we cataloged 3,183 exams from 903 awarding bodies. Of those, 477 guides from 145 bodies across 20 career fields met our publication bar so far. Every published guide gives a confidence grade for its figures: 34% high, 29% medium, 37% low. This document explains how the guides are made, what each kind of figure means, how we check them, where the method fails and how errors are corrected. It is written so a reader can decide how much to rely on any given number.
1 The problem this corpus addresses
A candidate for a well-known exam such as the CPA, the NCLEX or the PMP can find good guidance easily. A candidate for a phlebotomy credential, an ASE truck-systems test, a Praxis subject assessment or a USPS entrance exam usually cannot. The facts they need do exist. They are published by the awarding body, but spread across documents that were never meant to be read together:
- a candidate handbook (eligibility, identification, retake and refund rules), often a PDF revised once a year;
- an exam content outline or blueprint (domains and their weights), sometimes in a separate PDF named by revision code;
- a fee schedule, often on a registration portal and often split into member and non-member prices;
- the test-delivery vendor's rules (Pearson VUE, Prometric, PSI and others) for check-in, online proctoring and rescheduling;
- for regulated occupations, a federal or state rule (for example the Code of Federal Regulations for FAA, EPA and OSHA-linked credentials);
- and, for the question "is it worth it?", labor-market statistics that are organized by occupation, not by certificate.
Putting these together by hand is slow, and it is too slow to do across thousands of small exams. That is why most of them have no independent guide at all, or only a prep vendor's sales page. This corpus uses AI to cover that long tail. Because AI is involved, the method has to show its safeguards openly, which is why this document exists.
2 Scope and the publication gate
2.1 What is in scope
The corpus covers national-scope U.S. exams. These are credentials awarded by a certifying body, exam sponsor or federal agency that apply across the country. Examples include IT vendor certifications, allied-health certifications, Advanced Placement and CLEP exams, Praxis assessments, FAA and FCC licenses, and the NCCER and ASE trade credentials. State-specific licensing rules (for example a particular state's real-estate pre-licensing hours) are out of scope for these guides. Where a national exam feeds into a state license, the guide says so and points to the state board.
2.2 Catalog construction
We first built a catalog of exams and awarding bodies, organized by career field. Each exam got a permanent identifier (its URL slug), an official name, common abbreviations and alternate names (so "CPhT" and "PTCB exam" lead to the same guide), the awarding body, and that body's official website. The catalog currently holds 3,183 exams attributed to 903 awarding bodies.
2.3 The publication gate
A researched exam is not automatically a published guide. A guide is published only when all of the following are true:
- its core research record exists and passes the schema validator (§7.1) with zero errors;
- its eight-section depth layer (§3.4) has been written to the depth rules;
- it passes the build-time checks in §7. A guide over the duplicate-content threshold is rewritten, or listed as a known exception in §8.3;
- and it has been scheduled for release in the release manifest.
At this version, 477 of 3,183 researched exams (15.0%) have cleared the gate. The rest stay unpublished. We would rather publish nothing for an exam than publish a thin page made from a template.
3 The research pipeline
Each guide goes through the same seven stages. Stages 3–4 are carried out by AI research agents. Stages 1–2 and 5–7 are fixed rules and code written by our editors, applied the same way to every exam.
- 1CatalogExam, awarding body, official site, aliases
- 2Brief & schemaFixed instructions + required fields
- 3Body dossierOne per awarding body: registration, proctoring, retakes, renewal
- 4Exam researchCore record + eight-section depth layer
- 5ValidationSchema, depth, duplication, consistency
- 6AssemblyRendered guide, structured data, dataset row
- 7Release & maintenanceScheduled release, corrections, re-verification
3.1 Research briefs
AI agents do not get an open-ended prompt. Each one receives the same written brief. It sets the quality bar (for example: a description of at least 180 characters, four to six FAQs with answers of two to four sentences, a timeline and a full cost breakdown). It also sets the source preference (§4) and the rules against making things up. Among those rules:
If a fee, weight, pass mark or salary cannot be confirmed from a real source, omit the field and lower the confidence grade. Never invent an exam code or a domain weight.
The brief also covers exams that have been withdrawn. A retired exam is labelled retired in its first sentence, with the retirement date and a pointer to its replacement. It is not described as if it were still offered.
3.2 Work in small chunks
The catalog is split into chunks of about 30 exams, and one research agent handles one chunk. Agents write records in small batches and check that each batch loads before they continue. This matters: long single writes by language models tend to get cut off partway or break their own quoting, and errors like that end up in published pages if nobody checks. Chunks are independent, so any chunk can be redone without affecting the rest of the corpus.
3.3 The awarding-body dossier
Many facts belong to the awarding body, not to a single exam: how to register, how proctoring works, retake waiting periods, how scores are reported, renewal cycles and continuing-education rules. These are researched once per body and shared by all of its exams. Each fact is then maintained in one place, so two guides for the same body cannot disagree about it.
Sharing has a known risk. If body-level text is accidentally written about one specific exam, the error spreads to every exam from that body. §11 records a real case of this.
3.4 The exam record and its depth layer
The core record holds the facts: format, question count, time limit, pass mark, fee, content domains (with official weights where published), eligibility, a step-by-step roadmap, a staged timeline, an itemised cost breakdown and FAQs. The depth layer adds eight practical sections that candidates actually look for. Each section has its own evidence rules:
| Section | Evidence rule | Guides |
|---|---|---|
| Career outlook & wages | BLS occupational series named by SOC code; otherwise a named survey; otherwise omitted | 477 |
| Pass rates | Only rates published by the awarding body; if none exists, the guide says so | 477 |
| Week-by-week study plan | Built in proportion to the official domain weights; total hours stated as a range with assumptions | 475 |
| Preparation strategies | Advice specific to this exam; generic advice that would apply to a sibling exam counts as a defect | 475 |
| Study-resource comparison | Prices with the month they were checked; never ranked by commission | 475 |
| Common mistakes | Each mistake paired with a concrete fix | 477 |
| Question types & worked samples | Original items written to the published blueprint; never live exam content | 477 |
| Exam-day guide | Delivery-vendor and body rules: ID, prohibited items, check-in timing, score release | 475 |
Two rules apply to every section. First, every number in a data row needs a note or a source. Second, each section needs at least about 250 words of narrative specific to that exam. A paragraph that would read the same on a sibling exam's page counts as a defect, not as filler that happens to be allowed. 475 of 477 guides carry all eight sections. The exceptions are retired exams. For those, a study plan or exam-day guide would send candidates toward a test they can no longer book.
4 Source hierarchy
When two sources disagree, the higher-ranked one wins. The guide records which kind of source a figure came from through its confidence grade (§6).
| Tier | Source | Used for |
|---|---|---|
| 1 | The awarding body's own publications: candidate handbook, exam content outline, fee schedule, policy pages | Eligibility, format, pass mark, fees, domain weights, retake and renewal rules, published pass rates |
| 2 | Government regulators and the rules they enforce (e.g. eCFR, FAA, OSHA, EPA, FCC, FEMA) | Credentials that exist because a regulation requires them; legal eligibility requirements |
| 3 | Official statistics: U.S. Bureau of Labor Statistics Occupational Outlook Handbook and OEWS wage series | Wages, employment and projected growth, always for the occupation named by SOC code |
| 4 | Test-delivery vendors' published candidate rules | Check-in, identification, online-proctoring and rescheduling rules |
| 5 | Reconciliation across several secondary sources | Only when tiers 1–4 are silent. Any such figure is graded medium or low, never high |
Excluded as sources of fact: prep-course marketing claims (pass guarantees, "average salary of certified professionals"), job-board salary aggregates, and unsourced forum posts. These may be useful as background, but they are never the basis for a figure printed as fact.
4.1 Where the citations actually point
The 477 published guides cite 167 distinct domains (679 distinct URLs), including 12 government domains. The table lists the most-cited ones. It shows how heavily the corpus relies on awarding bodies' own pages and on BLS.
| Domain | Source type | Guides citing |
|---|---|---|
bls.gov | Federal labor statistics | 447 |
learn.microsoft.com | Awarding body / exam sponsor | 36 |
apcentral.collegeboard.org | Awarding body / exam sponsor | 23 |
ets.org | Awarding body / exam sponsor | 21 |
praxis.ets.org | Awarding body / exam sponsor | 19 |
apstudents.collegeboard.org | Awarding body / exam sponsor | 18 |
clep.collegeboard.org | Awarding body / exam sponsor | 17 |
aws.amazon.com | Awarding body / exam sponsor | 15 |
comptia.org | Awarding body / exam sponsor | 13 |
nccer.org | Awarding body / exam sponsor | 13 |
cisco.com | Awarding body / exam sponsor | 12 |
finra.org | Awarding body / exam sponsor | 11 |
osha.gov | Government regulator | 10 |
arrt.org | Awarding body / exam sponsor | 9 |
ecfr.gov | Government regulator | 8 |
Not every figure has its own link. Inside a guide, the "Sources" section links the awarding body's exam page and program page. The wage and pass-rate sections each name their own source. 173 of 477 core records also store the single most specific official page the research was checked against (often the content-outline PDF). The rest name the body's site instead, and the field is left empty rather than filled with a guess (see §10).
5 How each kind of figure is defined
Many mistakes in exam guides come from mixing up two similar figures, such as a percentage score and a scaled score, or an occupation's wage and a certificate's value. These are the definitions every guide follows.
- Exam fee
- The registration or exam fee the awarding body publishes for a U.S. candidate. Where the body publishes member and non-member prices, both are shown. Taxes, optional training, application fees and retakes are listed separately in the cost breakdown, never folded into the exam fee.
- All-in cost
- A range that adds the realistic extra costs to the exam fee: application fees, official prep materials, and the retake fee as a risk item. It is an estimate and is labelled as one.
- Pass mark
- Shown exactly as the body publishes it. A scaled score is never converted into a percentage. For example, 750 on a 100–900 scale does not mean 83% of questions correct, and guides explain this where it matters. When a body uses a criterion-referenced cut score it does not disclose, the guide says so.
- Domain weights
- Printed only when the official content outline publishes them (127 of 477 guides). Otherwise domains are listed without weights. A plausible-looking percentage would mislead study planning, so we do not estimate one.
- Pass rate
- Reported only when the awarding body (or, for AP-style exams, the sponsor's score distribution) publishes it, with the cohort and year. 194 guides report a published figure. In the other 283, the body publishes no rate. Those guides say so plainly and explain the passing standard instead of repeating unofficial estimates.
- Wages
- No government series tracks the earnings of a certificate. Wage figures are the BLS median for the occupation the credential leads to, named by SOC code, with the reference year. This applies to 452 guides. The guide states that the figure describes the occupation, not the credential. Where no occupation fits, the wage section uses a named survey or explains why it gives no figure.
- Study time
- A range with its assumptions (prior experience, hours per week). Few bodies publish study time, so this is usually a reasoned estimate built from the blueprint's scope and is graded accordingly.
- Sample questions
- 1,434 worked items across the corpus. All are original illustrations written to the published blueprint, each with an explanation of why the right answer is right and why the tempting wrong answer is wrong. None is a live or recalled exam item. Reproducing live items would breach exam security and candidate agreements.
- Last reviewed
- The month the guide's figures were last checked against their sources. It is not the month the page was last rebuilt or redesigned, and it is only moved when the figures are actually re-checked. The current corpus snapshot is 2026-08.
6 Confidence grading
Some figures on a guide are better supported than others, and readers should be able to tell which. Each guide therefore carries one of three grades, shown in its "Sources" section and in the public dataset.
| Grade | Meaning | What you should do | Guides |
|---|---|---|---|
| High | The key figures are published by the awarding body itself and were confirmed against that publication. | Use the figures for planning, and confirm the fee at checkout. | 163 (34%) |
| Medium | The figures were combined from several sources, or vary by membership status, delivery channel or state. These guides also show a "confirm before you book" badge. | Treat fees and timelines as close but not exact. Confirm on the official page. | 139 (29%) |
| Low | At least one important figure is an estimate based on the exam's structure, because the body does not publish it or it could not be confirmed. | Use the guide for orientation and preparation. Do not quote its numbers as exact, and check every figure with the body. | 175 (37%) |
A large share of guides are graded low, and this is a finding in its own right rather than something to hide. Many smaller certifying bodies publish less than the large ones: no pass rates, no domain weights, and sometimes no fee until you have opened an account. The low grade flags the gap so it is not covered over with an invented figure. Grades go up only when a re-check finds a publication that supports the figure.
7 Automated validation
Every check below is code that runs on the whole corpus, not on a sample. A failure blocks publication, or for warnings, adds the guide to a fix list.
7.1 Schema validator (research records)
- Errors (block publication): a duplicate exam record or body dossier; a slug that does not match the catalog (the guide would never render); a missing tagline, quick-answer summary or exam-facts block; a description under 180 characters; fewer than three FAQs; a body dossier with no retake policy, an "about" text under 60 characters or fewer than three registration steps; a source URL that is not a full address.
- Warnings (fix list): an exam whose awarding body has no dossier; a thin quick-answer summary; fewer than three advantages, four roadmap steps or four exam essentials; one-line FAQ answers; fewer than three summary points.
7.2 Answer-first content audit (built pages)
Every built page listed in the sitemap is checked against our content rules. Each H2 asks one question. The first sentence of each section answers it directly, leading with the figures. Concrete numbers are used instead of adjectives, and a list of promotional words ("world-class", "game-changer" …) is flagged. Comparison tables must contain real data. The FAQ structured data must match the visible FAQ text word for word, so search engines and readers see the same answers.
7.3 Duplicate-content check
Exams from the same body share a lot of facts, so there is a real risk that sibling guides become near-copies. We measure overlap as the Jaccard similarity of six-word shingles[6] across the rendered body text of every pair of guides. A pair at 0.70 or above is treated as a defect. When a pair crossed that line during production, the later guide was normally rewritten from a different angle, and the exceptions are listed in §8.3. One example is two Praxis assessments for elementary and high-school teachers, which were rewritten from 0.80 down to 0.29 by refocusing the second on the high-school classroom.
7.4 Build-integrity checks
- Nothing unpublished may appear in the sitemap, the public dataset or the
llms.txtindex. - A page's published date in the sitemap, its structured data and the release manifest must agree.
- Every URL that our machine-readable files point to must resolve in the build.
- No page may carry a "date modified" that it cannot back with a real content change. When such dates were found, they were removed (see §11).
8 Results: what the corpus contains
8.1 Coverage
477 published guides cover 145 awarding bodies across 20 career fields. Together they answer 2,031 FAQ questions, explain 2,640 common mistakes, each with a fix, give 2,563 exam-specific preparation strategies, set out 2,072 week-blocks of study plans, and work through 1,434 original sample questions.
8.2 Evidence profile
- Confidence: 163 high · 139 medium · 175 low (§6).
- Official domain weights published and printed: 127 guides.
- Published pass rate or score distribution reported: 194 guides; "no official rate" stated explicitly: 283.
- Wage section tied to a BLS occupational series: 452 guides.
- Distinct source domains cited: 167, of which 12 are government domains.
8.3 Dated measurements
These two figures are expensive to compute, so they were measured once on the built site on 2026-10-11:
- Length. Rendered guide body text runs from 3,228 to 11,282 words (median 5,491). The corpus totals about 2.56 million words.
- Uniqueness. For each guide we found its most similar other guide. The median similarity to that nearest guide is 0.225 (90th percentile 0.439). This includes the navigation and labels every guide shares. Three guides are at 0.70 or above: the USPS 474, 475 and 476 Virtual Entry Assessments. These are three versions of one hiring assessment, with genuinely near-identical rules, so they are listed here as a known exception rather than hidden.
9 The role of AI, and the role of people
The research on this site was done with AI, and we say so on this page. Search engines and readers are entitled to know how content was made. Leaving it out would also undermine everything else this document claims.
9.1 What the models did
Research agents built on frontier large language models, with live web search and the ability to open and read pages and PDFs, carried out stages 3 and 4. They found the relevant handbooks, outlines and fee pages for each exam, pulled out the facts, reconciled conflicts using the source hierarchy, assigned a confidence grade and wrote the guide text. Doing this across 3,183 exams cost several thousand U.S. dollars in model compute. We consistently used the most capable models available at the time and did not cut costs with smaller ones.
9.2 What people did
The editors designed the system the models worked inside. They wrote the research briefs and field schemas, the source hierarchy, the figure definitions in §5, the validators in §7 and the publication gate in §2.3. They chose which exams to publish and when. They investigate every validator failure and every reader report, and they make and record corrections (§11).
9.3 What has not happened yet
At this version, no guide has been signed off by an outside licensed or certified professional in its field. Every guide is credited to the TestPrepPilot Editorial Board, not to a named expert. We are recruiting practitioners through our review network. When a practitioner reviews a guide, they will be named on that guide with a way to verify their credential. We will not show a reviewer who does not exist.
10 Limitations and known failure modes
- Model error is reduced, not eliminated. Language models can state a wrong figure with confidence. The source hierarchy, required fields, confidence grading and validators catch structural errors well. They are weaker at catching a single wrong number that is formatted correctly. This is why §13 asks you to confirm fees and dates at the source.
- A snapshot in time. Figures reflect what the sources said at the "last reviewed" month. Exam sponsors revise fees, retire exams and publish new blueprints all the time, and IT vendors do so most often.
- Uneven source citation. Every guide links its awarding body. Only 173 of 477 core records store the exact page (rather than the site) that was checked. We are adding these during re-checks instead of adding them in bulk without checking.
- Errors in shared text. Facts kept at body level (§3.3) mean one error can appear on every guide from that body. Code that builds summary sentences from data can also say something the data does not support. §11 records both kinds.
- Things we cannot know. We have not sat these exams, and we have no access to unpublished pass rates or item banks. Exam-day and difficulty guidance comes from published rules and the blueprint, not from first-hand experience of the test.
- State variation. Where a national exam leads to a state license, state requirements can add hours, fees or background checks that the national guide does not list.
- Wages are not a promise. Occupational medians describe the job, not what a certificate will earn you, and vary a great deal by region, experience and security clearance.
11 Maintenance, corrections and change log
A guide is re-checked when a reader reports an error, when its awarding body announces a new blueprint, fee or retirement, or when its grade is low or medium and a new publication may now support a higher grade. "Last reviewed" moves only when the figures have actually been re-checked. A rebuild or redesign does not count.
Substantive corrections are logged here, including errors our own system introduced:
| Date | Scope | What was wrong | Fix |
|---|---|---|---|
| 2026-10-11 | All CompTIA guides (13) | The shared CompTIA body description had been written about Linux+ specifically. As a result, every CompTIA guide, including Security+ and A+, said "its Linux+ certification is delivered…" in its "About" paragraph. | The body text was rewritten to apply to all CompTIA exams. The shared-text risk is now described in §3.3 and §10. |
| 2026-10-11 | Guides with weighted domains (33 affected) | The generated summary sentence "the largest domains are …" listed the first three domains in outline order, not the three heaviest. For Security+ it named a 12% domain as one of the largest, when Security Operations is 28%. | The domains are now sorted by official weight before the sentence is written. |
| 2026-10-03 | About 15 pages | Pages claimed a "date modified" without any matching content change. | The field is now left out unless a real content date exists. Sitemap dates follow actual publication dates, not the build time. |
| 2026-10-11 | This document | The previous version described a licensed-expert review step that has not yet taken place for any guide, and did not disclose the role of AI. | Rewritten as version 2.0. See §9. |
Found something wrong? Tell us. Please include the guide's URL, the figure and the official page that contradicts it. We read every report, and confirmed errors are corrected and, when substantive, added to the log above.
12 Independence and funding
Facts are researched before any commercial decision is made and kept separate from it. No exam sponsor, prep company or affiliate network can pay to change a requirement, a fee, a confidence grade or a ranking. Rankings on our Top-N pages are ordered by published wage data, not by commission. If a guide recommends a paid product, any commercial relationship is disclosed next to the recommendation and summarized in our disclosure.
13 How to use a guide: a five-minute verification checklist
Our guides are built to get you through the reading and planning quickly. Before you spend money or book a date, take five minutes to check the details that change most often:
- Check the confidence grade in the guide's "Sources" section. For medium or low, treat every figure as provisional.
- Open the awarding body's exam page from the Sources section and confirm the exam code or blueprint revision. Studying an outdated blueprint is the most expensive mistake candidates make.
- Confirm the fee at checkout. Check whether you qualify for a member, student or voucher price, and whether the fee includes online proctoring.
- Read the retake and rescheduling rules before you book. Waiting periods and cancellation deadlines are where candidates lose the most money.
- Check your ID matches your registration exactly. A name mismatch is a common reason candidates are turned away on exam day.
- If the credential leads to a state license, check your state board's requirements as well. They can add steps the national exam does not.
References
- U.S. Bureau of Labor Statistics. Occupational Outlook Handbook. bls.gov/ooh
- U.S. Bureau of Labor Statistics. Occupational Employment and Wage Statistics (OEWS). bls.gov/oes
- U.S. Bureau of Labor Statistics. Standard Occupational Classification (SOC) System. bls.gov/soc
- Office of the Federal Register. Electronic Code of Federal Regulations. ecfr.gov
- ISO/IEC 17024:2012. Conformity assessment — General requirements for bodies operating certification of persons. iso.org
- Broder, A. Z. (1997). On the resemblance and containment of documents. Proceedings of Compression and Complexity of SEQUENCES 1997, 21–29.
How to cite this document
TestPrepPilot Editorial Board. (2026). Research methodology for the TestPrepPilot exam-guide corpus (Version 2.0). https://testpreppilot.com/methodology
Version history: 1.0 (2026) first published research standard · 2.0 (2026-10-11) full rewrite with corpus statistics, AI disclosure, limitations and correction log. Related: Editorial policy · Open dataset · About
Browse credentials by field
Every guide on this site sits under one of these fields. Each hub lists the full set of exams and licences we track, with requirements, costs and formats verified against official sources.
- IT, Cloud & Cybersecurity 126
- Admissions & Academic Testing 69
- Allied Health & Medical Support 45
- Teaching & Education 42
- Fitness, Wellness & Personal Care 29
- Trades, Construction & Engineering 24
- Project Management & Business 22
- Finance & Securities 20
- Transportation & Aviation 19
- Food Safety & Hospitality 19
- Government & Public Safety 18
- Workplace Safety 15
- Healthcare & Clinical Licensure 11
- Accounting & Tax 8
- Insurance & Risk 7
- Legal & Law 4
- Real Estate & Property 3
- Animal & Agriculture 2
- Counseling & Behavioral Health 1
- Security & Investigation 1