AI coding help for a non-coder building a small app
Task: Someone who has never coded builds a small working app by following one AI tool's instructions.
Demos make app-building look instant. We sit a real non-coder in front of each tool with the same small app idea and count what it takes to get something that works.
This is a test plan. Nothing has been run yet, so there are no scores or verdicts below.
How we tested
- One written app brief and an acceptance checklist (what "working" means), the same for every tool.
- A tester with no coding experience uses one tool at a time, starting fresh, and only does what the tool tells them.
- We stop at a working app or after 90 minutes.
- Recorded per tool: did it work, number of messages to get there, how many times the tester was stuck, every checklist item that passes, cost, and total time.
- A person also scores how clear the explanations were.
What we scored
- Checklist items passing · human-judged · weight 3
- Out of the acceptance checklist (enter the number passing).
- Messages to working · human-judged · weight 1
- How many messages the tester sent. Lower is better (cap 50).
- Times stuck · human-judged · weight 1
- Times the tester couldn't follow an instruction. Lower is better (cap 10).
- Explanations a beginner understands · human-judged · weight 1
- Were the steps explained in plain words?
Inputs every tool received
App brief and checklistPlaceholder
Placeholder: brief and checklist to be provided.
Inputs still needed
- The app brief (for example: "a one-page app to log my shop's daily expenses by category and show the monthly total, saved on my computer").
- The acceptance checklist: 5–8 things the app must do.
- A tester who has never written code, and the tools and plans to test.