Meeting transcription and notes
Task: Turn a real recorded meeting into an accurate transcript and useful notes with decisions and action items.
Note-taker apps all promise perfect notes. We measure how many words they get wrong and whether they catch every decision and task.
This is a test plan. Nothing has been run yet, so there are no scores or verdicts below.
How we tested
- Every tool gets the same audio files: one English meeting and one Arabic (or mixed Arabic/English) meeting.
- Transcript accuracy is measured as word error rate (WER) against a reference transcript checked by a person. Lower is better.
- Each tool then writes notes with the same prompt (shown under Inputs). Where a tool writes its own notes automatically, we use those.
- Action items and decisions are checked automatically against a list made from the reference transcript.
- Speaker labels and how useful the notes are get scored by a person, from the raw output only.
- We record what each run cost and how long it took to come back.
What we scored
- Word error rate · objective · weight 2
- Share of words wrong, missing or added versus the reference transcript. Lower is better.
- Decisions and tasks caught · objective · weight 2
- Share of the real decisions and action items that appear in the notes.
- Speaker labels · human-judged · weight 1
- Are lines given to the right person?
- Notes you could send · human-judged · weight 1
- Could you send these notes to the team without rewriting them?
Inputs every tool received
English meeting (audio)Placeholder
Placeholder: recording to be provided.
Show the exact prompt
Transcribe this meeting verbatim with speaker labels. Then write meeting notes with: a 5-bullet summary, every decision made, and every action item with its owner and due date if one was said. Do not add anything that was not said.
Arabic / mixed meeting (audio)Placeholder
Placeholder: recording to be provided.
Show the exact prompt
فرّغ هذا الاجتماع حرفيًا مع تسمية المتحدثين. ثم اكتب ملاحظات الاجتماع وتتضمن: ملخصًا من 5 نقاط، وكل قرار اتُّخذ، وكل مهمة مع المسؤول عنها وموعدها إن ذُكر. لا تضف أي شيء لم يُقل.
Inputs still needed
- An English meeting recording, 20–45 minutes, 3+ speakers, normal room audio. Everyone in it must agree to it being used publicly (or use a public-domain recording).
- An Arabic or mixed Arabic/English meeting, same conditions.
- A reference transcript for each, checked line by line by a person.
- The list of decisions and action items in each meeting (one per line, in answer-key format).
- Which tools and plans to test (for example a note-taker app on its paid plan, and transcription APIs).