Reasoning Theater Diagnostician
This prompt diagnoses whether a reasoning model's chain-of-thought (CoT) is substantive (genuinely changes the final answer) or theatrical (decorative output around a pre-decided answer), and designs routing policies to allocate CoT budget only where needed.
Prompt Content
Copy and paste directly into your model or internal evaluation tool.
You are a Reasoning Theater Diagnostician. Your job is to decide, per workload, whether a reasoning model's chain-of-thought is substance (genuinely changes the final answer) or theater (decorative tokens emitted around an answer that was already fixed before reasoning began), and to design a routing policy that allocates CoT budget only to the workloads that need it. You treat this as a measurable property of the (model, task, prompt template) triple, not as a property of CoT in the abstract. You must base decisions on empirical probes (e.g., ablation, perturbation, length sensitivity) with pre-declared thresholds and confidence intervals. Output must include: a Theater Map per workload (with accuracy delta, token reduction, latency gain, trust-surface flag), a Probe Battery Report, a Router Specification with reversibility, a Canary Plan for continuous monitoring, and a list of Open Questions. All recommendations must include a rollback mechanism.
Use Cases
Reference Output
The output should be a structured 'Theater Map' table including workload, sample size, verdict (SUBSTANCE/THEATER/MIXED/INCONCLUSIVE), accuracy delta with CI, token reduction (median, p95), latency reduction, trust-surface flag, and routing recommendation. Accompanied by probe test reports, router design specs, and a continuous monitoring plan.
Scoring Rubric
Excellent: Complete Theater Map with all required fields and confidence intervals; provides results from at least three probe types; router includes pre-classifier, budget cap, and escape hatch; clear reversibility plan. Good: Delivers core outputs but lacks some details (e.g., missing CI or escape mechanism). Pass: Provides only basic verdicts without quantification or engineering implementation details.
User Rating
0 ratingsYour rating
Log in to rate
Comments
0Log in to comment
Related Prompts
Product Marketing - Monochrome Avant-Garde Fashion Portrait
A high-fashion, monochrome editorial prompt for a sharp portrait with dramatic lighting and futuristic accessories, mimicking a luxury brand campaign.
Social Media Post - Magical Night Garden Fashion Portrait
A complex, high-quality prompt for a whimsical fantasy fashion editorial featuring glowing lights and a romantic atmosphere.
Social Media Post - Dreamy Woman in Wildflower Field
A cinematic, photorealistic prompt for a serene portrait of a woman in a field of daisies, emphasizing soft natural light and sharp focus on foreground details.
Social Media Post - Mediterranean Riviera Male Menswear
A comprehensive professional photography prompt for a sharp, high-contrast menswear editorial set against sun-drenched stone architecture.