Runner self-check: summarising the US Constitution
Task: Summarise a public-domain document with many numbers and keep every number right.
An internal check that the lab runner, the answer-key scorer and the results page work end to end. Not an editorial test.
This test is still running. Scores below may be incomplete; empty cells are not scored yet.
Scores
| Tool | Numbers rightobjective | Total /100 | Cost | Time |
|---|---|---|---|---|
| claude-haiku-5-5 (API) | 11/11 | 100 | $0.0029 | 18.0s |
| claude-sonnet-5-5 (API) | 11/11 | 100 | $0.04 | 17.0s |
— means not scored yet (never zero). Objective scores are computed automatically; human scores are judged by our editor. The total is weighted and appears only when every counted criterion has a score.
Outputs side by side
claude-haiku-5-5 (API)
claude-haiku-5-5 $0.0029 18.0sRun by our test runner
The Constitution of the United States (1787)
This Project Gutenberg text reproduces the Constitution with original spellings, capitalization, and punctuation where they could be ascertained. It excludes the amendments.
Preamble: States the goals of forming "a more perfect Union," establishing Justice, insuring domestic Tranquility, providing for the common defence, promoting the general Welfare, and securing the Blessings of Liberty.
Article 1: Congress
- House of Representatives: Members serve terms of every second Year. They must be at least twenty five Years old and seven Years a citizen. Seats are apportioned by population, with three fifths of "all other Persons" counted. The census occurs within three Years of the first meeting, then every ten Years. There is no more than one Representative per thirty Thousand, and each State gets at least one. The House alone holds the Power of Impeachment.
- Senate: Each State has two Senators, chosen by its legislature, serving six Years with one Vote each. Senators must be at least thirty Years old and nine Years a citizen. The Senate alone tries Impeachments, and conviction requires two thirds of Members present.
- Procedure: A majority of each House constitutes a quorum. Two thirds may expel a Member, and one fifth of those present may demand the yeas and nays. Neither House may adjourn for more than three days without the other's consent.
- Lawmaking: Revenue bills originate in the House. The President may veto a bill, but two thirds of each House can override the veto. A bill becomes law if the President does not return it within ten Days (Sundays excepted).
- Powers (Section 8): Congress may tax, borrow, regulate commerce, coin money, establish post offices, grant copyrights and patents for limited Times, declare war, raise and maintain armies and a navy, and make all laws "necessary and proper." Appropriations for armies may not exceed two Years.
- Limits (Section 9): Congress may not prohibit the importation of persons admitted by the States before the Year one thousand eight hundred and eight, though a tax of up to ten dollars per Person may apply. Habeas corpus may be suspended only in cases of Rebellion or Invasion. Bills of attainder, ex post facto laws, and titles of nobility are barred.
- State limits (Section 10): States may not make treaties, coin money, or issue bills of credit. They may not impose duties without Congress's consent, and they may not keep troops or ships in time of peace without consent.
Article 2: The President
- The President serves a term of four Years, with a Vice President elected for the same term.
- Electors equal the total number of Senators and Representatives. A candidate needs a majority of electors. If none has one, the House chooses from the five highest.
- The President must be a natural born Citizen (or a Citizen at the Constitution's adoption), at least thirty five Years old, and fourteen Years a resident within the United States.
- The President's salary may be neither increased nor diminished during the term.
- The President commands the armed forces, grants pardons (except in impeachment cases), and makes treaties with the advice and consent of two thirds of Senators present. Appointments require Senate consent.
- The President reports on the State of the Union and must "take Care that the Laws be faithfully executed."
- Removal occurs on impeachment and conviction for Treason, Bribery, or other high Crimes and Misdemeanors.
Article 3: The Judiciary
- One supreme Court is established, along with inferior courts as Congress creates them. Judges hold office during good behavior.
- Treason requires testimony from two Witnesses to the same overt act, or a confession in open Court.
Article 4: States
- Each State must give full faith and credit to the public acts and judicial proceedings of other States.
- Citizens receive the Privileges and Immunities of citizens in all States. Fugitives from justice must be surrendered, as must persons held to Service or Labor who escape into another State.
- Congress may admit new States. The United States guarantees each State a Republican Form of Government.
Article 5: Amendments Amendments may be proposed by two thirds of both Houses (or a convention called on application of two thirds of the States), and they take effect when ratified by three fourths of the States.
Article 6: Supremacy Pre-existing debts remain valid. The Constitution, federal laws, and treaties form "the supreme Law of the Land." Officials must take an oath to support it, and "no religious Test" may be required for any office.
Article 7: Ratification Ratification by nine States suffices to establish the Constitution.
Signing: Done in Convention on the Seventeenth Day of September, one thousand seven hundred and eighty seven, in the Twelfth year of American Independence. Signed by George Washington, President and deputy from Virginia, and delegates from the States. Attested by William Jackson, Secretary.
claude-sonnet-5-5 (API)
claude-sonnet-5-5 $0.04 17.0sRun by our test runner
Summary: The U.S. Constitution (1787), Project Gutenberg Etext
About the text: This edition keeps the original's variant spellings, punctuation and capitalization. It omits the amendments, which are posted separately. The editors invite scholars to send corrections.
Preamble: The people ordain the Constitution to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defence, promote the general Welfare, and secure the Blessings of Liberty.
Article 1: Congress
- House: Members are chosen every second Year. They must be twenty five Years old and seven Years a citizen. Direct taxes and representation are apportioned by population, counting three fifths of all other Persons. A census is taken every ten Years. There is no more than one Representative for every thirty Thousand, and each State has at least one. The House has the sole Power of Impeachment.
- Senate: Two Senators per State, chosen by the legislature, serve six Years. One third is chosen every second Year. Senators must be thirty Years old and nine Years a citizen. The Vice-President presides and votes only to break ties. Conviction on impeachment requires two thirds of Members present, and the penalty is limited to removal and disqualification.
- Procedure: Congress meets at least once every Year, on the first Monday in December. A Majority is a quorum, and two-thirds can expel a Member. One fifth of those present can require recorded Yeas and Nays. A bill becomes law if the President does not return it within ten Days (Sundays excepted). Two thirds of each House can override a veto.
- Powers and limits: Congress may tax, borrow, regulate commerce, coin money, declare war, and raise armies, with appropriations for armies lasting no longer than two Years. The federal District may not exceed ten Miles square. Importation of persons could not be prohibited before 1808, with a tax of up to ten dollars each. Habeas corpus, bills of attainder, ex post facto laws, and titles of nobility are addressed, and States are also restricted.
Article 2: President
- Four-Year term. The President must be a natural born Citizen (or a citizen at the Constitution's adoption), thirty five Years old, and fourteen Years a Resident.
- Electors equal the number of Senators and Representatives. Each votes for two Persons. A Majority of electors is needed. Otherwise the House chooses from the five highest, voting by States, with a majority of all States needed.
- Treaties need two thirds of Senators present. The President is Commander in Chief, nominates officers with Senate consent, and takes an oath.
- Removal is for Treason, Bribery, or other high Crimes and Misdemeanors.
Article 3: Judiciary: One supreme Court plus inferior courts. Judges serve during good behavior. Trials of crimes are by Jury. Treason requires two Witnesses to the same overt Act, or confession in open Court.
Article 4: Full Faith and Credit among States, extradition, admission of new States, and a Republican Form of Government guaranteed.
Article 5: Amendments: Proposed by two thirds of both Houses, or by a convention called on application of two thirds of State legislatures. Ratification requires three fourths of the States.
Article 6: Prior debts remain valid. The Constitution is the supreme Law of the Land. No religious Test for office.
Article 7: Ratification by nine States.
Signing: Done on the Seventeenth Day of September, 1787, signed by George Washington and delegates from the States, and attested by William Jackson, Secretary.
How we tested
The original text of the US Constitution (Project Gutenberg eBook #5, without amendments) was given to each model with the same prompt. The answer key lists 11 numbers stated in the text and 3 claims that are not in it (they come from later amendments or common knowledge).
What we scored
- Numbers right · objective · weight 1
- Share of the 11 answer-key numbers present, minus 25% per claim not in the text.
Inputs every tool received
US Constitution, original text (Gutenberg #5)
DownloadShow the exact prompt
Summarise this document for a busy reader in under 400 words. Include the key numbers (ages, term lengths, majorities, counts) exactly as the document gives them. Do not add information that is not in the document.