Can AI Create a Math Test?

I put ChatGPT, Claude, & Gemini to the test

One of the most common questions I get from educators is: "Can AI really help me with [insert seemingly impossible task]?"

Last week, a high school math teacher put AI to the test - literally!

His request:

1. Could AI generate a Chapter 1 test for his Precalculus Honors students using their textbook, complete with an answer key?

2. It had to be a 60-minute test, 60% multiple-choice, 40% free-response, with a 10% bonus question.

Result? 👍 AI NAILED IT - but it wasn't a one-click wonder.

Here’s the truth: You can’t just chuck a textbook at ChatGPT and expect a perfectly formatted, rigorous exam to magically appear (at least, not yet!). It takes a bit of finesse.

Here's our AI-powered test creation process:

1. Crafting the Perfect Prompt: We used the Prompt Maker GPT in ChatGPT to write a detailed, specific prompt, and I even ran it through Google AI Studio for extra optimization.

2. Model Showdown: We fed the prompt and a single PDF page of the textbook to ChatGPT, Claude, and Gemini to see which performed best. Claude emerged as the champion for this round!

3. Iterate and Conquer: We worked section by section, refining the prompt and testing different approaches until the teacher was happy with the results.

4. Teacher Takes the Reins: Over the weekend, he experimented further, generated the full test, and documented his process for future use.

Next up on our AI adventure:

• Generating figures and charts to accompany questions.

• Exploring AI-powered grading solutions.

• Experimenting with more complex problem types.

Could he have created the test manually? Sure. But sometimes, the journey of discovery with AI is just as valuable as the destination.

Now, over to you! How are you using generative AI in math education? What strategies have you found successful (or not)?

Here’s the prompt we used!

Prompt:

Act like an experienced math test creator with over 15 years of experience in designing tests for high school students. Your expertise includes

creating a wide variety of math problems, from basic arithmetic to advanced calculus and statistics, tailored to assess different levels of understanding and problem- solving skills.

Objective:

You will generate a comprehensive math test based on the content provided in a PDF document. The test should cover key concepts, skills, and problem types relevant to the material in the document. The test must be suitable for the intended grade or difficulty level described in the document, or, if not specified, be appropriate for a high school sophomore.

Instructions:

1. Review the Document:

 Extract key mathematical topics, concepts, formulas, and examples from the provided PDF.

 Identify the level of difficulty and the intended audience (e.g., high school, college-level).

 Note any specific instructions, focus areas, or learning objectives mentioned in the document.

2. Generate Test Sections:

 Divide the test into relevant sections based on the topics identified. Each section should represent a key concept or skill area (e.g., algebra, geometry, calculus).

 For each section, include a brief description of what it covers.

3. Create Questions:

 For each section, create 5-7 questions that vary in difficulty (easy, medium, hard).

 Ensure a mix of question types: multiple choice, short answer, calculations, proofs, and word problems.

 Questions should assess a range of skills, including basic understanding, application, and problem-solving.

4. Include Detailed Instructions:

 Provide clear instructions for each section and question type.

 Indicate if calculators or other tools are permitted.

5. Provide Solutions and Scoring Guide:

 Include detailed solutions and explanations for each question.

 Provide a scoring guide or rubric, indicating how points are allocated for each question.

6. Check for Alignment and Clarity:

 Ensure that all questions align with the topics and difficulty level intended by the document.

 Verify that the language and formatting are clear and professional.

7. Final Review:

 Review the entire test for completeness, accuracy, and alignment with the document’s content.

 Make any necessary adjustments to ensure the test is challenging yet fair, and covers the content comprehensively.

 Take a deep breath and work on this problem step-by-step.