AI Elo

Analysis of Claude Opus 4.6 (High Think):

Entry Y leads with strong safety guidance and commits to a single primary diagnosis: cracked secondary heat exchanger. DIAGNOSIS: This is where Entry Y has a significant problem. While a cracked heat exchanger is absolutely something that must be ruled out, declaring it the "most likely root cause" and the "unifying diagnosis" is premature and arguably incorrect given the full symptom picture. Here's why: The Carrier 58MVP is a two-stage condensing furnace. The symptoms that Entry Y attributes to a cracked HX are better explained by vent/condensate issues: 1) Increased condensate pump cycling + water around pump: Entry Y claims "condensate leaking through the crack explains increased pump cycling." This is backwards. A cracked secondary heat exchanger would more likely cause condensate to leak OUT of the heat exchanger into the burner area or vestibule panel, not increase flow to the condensate pump. Increased condensate pump cycling is classic for a condensate drainage restriction — condensate backs up, partially clears, backs up again. 2) Ice/white crust on exhaust termination: This is a vent blockage/restriction symptom, not a heat exchanger symptom. Entry Y mentions this as a "secondary factor" but doesn't adequately weight it. 3) Wind correlation: Wind affecting failures is much more consistent with vent termination issues (wind-driven backpressure on the exhaust, or wind pulling exhaust back toward the intake if they're too close) than with a heat exchanger crack. A cracked HX would fail regardless of wind conditions. 4) The "whoof" sound: While flame rollout can occur with a cracked HX, it also occurs with vent restrictions that cause backpressure — flue gases can't exit, pressure builds, and when the burner tries to relight, accumulated gas ignites with a puff. This is actually more consistent with a vent restriction scenario. 5) Failures after long heat calls: A condensate restriction gets worse over a long run cycle as condensate accumulates. A cracked HX would not particularly worsen with run time in this pattern. 6) The water heater backdrafting: Entry Y doesn't adequately explain this in the context of a cracked HX. The backdrafting is explained by negative pressure in the mechanical room from the furnace inducer — this is an independent issue from the HX condition. Entry Y's claim that "rollout code + CO alarms + backdrafting is the signature of a cracked heat exchanger" is an oversimplification. These symptoms together are also the signature of a vent restriction with negative pressure issues in the mechanical room. SAFETY: The safety guidance is strong and appropriate — immediate shutdown, ventilation, evacuation criteria. This is well done. CLARITY: Very clear and easy to follow. The format is clean and the language is accessible. However, the certainty of the diagnosis could be misleading — if a homeowner tells the HVAC tech "I was told it's a cracked heat exchanger," the tech might focus there and miss the vent/condensate issues, or the homeowner might be quoted for a heat exchanger replacement (~$1,500-3,000+) when the actual fix might be a $200 vent repair and condensate service. JUDGMENT: Entry Y correctly says to call a pro and not restart, which is appropriate. However, it provides almost no useful DIY steps the homeowner could safely take before the tech arrives (like checking the condensate drain, inspecting PVC slope, etc.). The prompt specifically asked for temporary risk-reduction steps and DIY-appropriate checks. Entry Y essentially says "don't touch anything, call a pro," which while safe, doesn't fully address the prompt. The entry also doesn't provide the detailed troubleshooting plan with specific measurements that the prompt requested. It mentions combustion analysis and manometer readings but doesn't give target values, expected pressures, or a systematic diagnostic sequence. WEAKNESSES: Premature diagnostic commitment to cracked HX as primary cause; inadequate explanation of condensate pump symptoms; missing the most likely root cause (vent/condensate restriction); insufficient detail on troubleshooting steps and measurements; doesn't adequately address the prompt's request for distinguishing between failure modes that mimic each other.

Analysis of GPT-5.2 (Low Effort):

Entry X provides a well-structured, prioritized differential diagnosis that correctly identifies the most likely root cause hierarchy: (1) vent/condensate restriction causing inducer/pressure loss and flame rollout, worsened by wind; (2) negative basement pressure/backdrafting of the water heater; (3) cracked heat exchanger as a less likely but must-rule-out possibility. DIAGNOSIS: This ranking is technically sound and reflects expert-level HVAC reasoning. The Carrier 58MVP is a condensing furnace with PVC venting. The symptoms — white crust/ice on exhaust termination, failures correlating with wind and long run cycles, increased condensate pump activity, water around the pump — all strongly point toward a vent/condensate restriction as the primary issue. A blocked or partially blocked condensate drain/trap causes condensate to back up into the secondary heat exchanger or collector box, which increases resistance to flue gas flow, causing the pressure switch to open. The "whoof" is consistent with delayed ignition or brief flame rollout when the inducer can't establish proper draft due to backpressure. The ice/crust on the exhaust termination is a classic cold-climate condensing furnace problem. Entry X correctly identifies that wind effects on the termination compound this. The negative pressure explanation for the water heater backdraft is also correct — the furnace inducer can depressurize the mechanical room, pulling combustion products from the atmospheric water heater's draft hood. This is a well-known interaction in shared mechanical rooms. Entry X correctly places heat exchanger crack as less likely but necessary to rule out. This is the right call — while a cracked HX is possible on a ~12-year-old unit, the constellation of symptoms (especially the condensate pump issues, ice on exhaust, wind correlation) more parsimoniously points to vent/condensate problems first. SAFETY: Appropriately leads with immediate shutdown, ventilation, and evacuation guidance. Correctly flags CO alarm activation as the critical safety trigger. Does not recommend bypassing any safety switches. CLARITY: Despite being compressed, it's organized logically: danger first, then diagnosis, then DIY steps, then pro steps. The DIY instructions are practical and appropriate (shop-vac condensate trap, check PVC slope, verify terminations). The pro measurements are specific (manometer across pressure switch per data plate, CO in flue <100 ppm air-free, CO in supply = 0). JUDGMENT: Excellent balance of what a homeowner can safely do (visual inspections, condensate clearing, termination checks) versus what requires a professional (combustion analysis, heat exchanger inspection, manometer readings). The long-term fixes proposed (repitch venting, replace trap/pump, add combustion air or convert WH to direct-vent) are all appropriate and comprehensive. WEAKNESSES: The compressed format sacrifices some explanatory detail. It doesn't fully explain the logic connecting each symptom to each diagnosis, which could leave a homeowner less informed about why certain steps matter. The step-by-step troubleshooting plan requested in the prompt is somewhat abbreviated.

Analysis of Claude Opus 4.6 (High Think):

Entry Y is very safety-forward (shutdown, ventilate, evacuate on alarm), which is good. However it prematurely anchors on a cracked secondary heat exchanger as “most likely” and treats it as a unifying signature without adequately justifying against other more common, better-fitting causes in this fact pattern. The windy correlation, visible termination ice/crust, inducer getting louder before failure, and increased condensate pump cycling/water around pump all strongly suggest vent/condensate restriction or termination pressure effects—classic for condensing furnaces—while a heat exchanger crack is less directly supported and would not typically explain pressure-switch-open faults or increased condensate pump cycling as described (pump cycling more could be from condensate restriction/overflow, high run time, or drain issue; ‘condensate leaking through a crack’ is not a standard mechanism). The water heater draft hood spillage when the inducer starts is more consistent with mechanical room depressurization or venting interactions than a furnace heat exchanger breach. Their suggested diagnostic criterion “delta >2 ppm supply/return confirms crack” is dubious; supply air CO should be ~0, but small ppm differences can be measurement noise, and crack confirmation requires proper inspection/combustion testing, not a simplistic delta. They also under-address the prompt’s requirement to distinguish failure modes (blocked condensate vs vent icing vs pressure switch vs negative pressure) and provide step-by-step measurements/targets; they largely say “call pro.” Clarity is simple but perhaps too reductive. Judgment: calling a pro same-day is right, but the asserted root cause and some technical claims are weak and could mislead prioritization of fixes.

Analysis of GPT-5.2 (Low Effort):

Entry X presents a solid, appropriately cautious differential that ties most symptoms together without overcommitting: intermittent pressure-switch open/rollout code, windy correlation, visible vent icing/crust, inducer louder, increased condensate/pump cycling and occasional water, plus slight exhaust odor and transient CO alarm. Their primary hypothesis—venting/condensate restriction (icing, sag/low spots, clogged trap, pump overflow) causing draft loss and ignition disturbances—fits the lockout pattern (inducer runs, retries, then lockout), the windy nights (pressure at termination, recirculation, downdraft), and the condensate observations. They also appropriately flag negative basement pressure/backdrafting of the atmospheric water heater as a parallel/compounding hazard and mention heat exchanger crack as a must-rule-out rather than presumed. Safety guidance is strong: shut down, ventilate, leave if alarms/symptoms; do not bypass safeties. Clarity is decent for a homeowner; the DIY checks are limited to visual/cleaning/slope/pump/trap inspection, which is reasonable, though a bit broad and could be organized more stepwise. Professional plan mentions key measurements (manometer across pressure switch, inducer draft, combustion analysis, worst-case depressurization) but is light on explicit expected values/targets and exact test points asked by prompt. Judgment on DIY vs pro is appropriate: homeowner can check terminations, condensate path, slope; pro needed for combustion/draft/heat exchanger. Overall: good balance, plausible unifying causes, avoids a single speculative diagnosis.

Analysis of Claude Opus 4.6 (High Think):

Entry Y is very strong on immediate safety shutdown/evacuation advice. However, it over-commits to “cracked secondary heat exchanger” as the most likely unifying diagnosis with a level of certainty not supported by the described evidence. The symptom cluster (windy correlation, pressure-switch open/rollout code, termination icing/crust, inducer louder, long-call onset, condensate pump cycling/overflow, occasional water-heater draft hood spillage) more strongly suggests venting/condensate restriction and/or building pressure/backdraft issues; a heat-exchanger crack is possible but not the most parsimonious first call. Some technical claims are shaky: “delta >2 ppm supply/return confirms crack” is not a standard definitive criterion; supply-air CO should be essentially zero, but small differences can be due to measurement noise, infiltration, or other sources—HX diagnosis is usually by visual/borescope, combustion analysis trends, CO in supply under specific conditions, and sometimes pressure testing, not a simple delta rule. It also downplays that vent blockage/backpressure can indeed contribute to rollout/whoof and CO spillage. Clarity is simple but offers little actionable troubleshooting logic beyond ‘call a pro’, and the guidance on leaving the water heater on is risky given observed spillage/backdrafting—better to recommend shutting down both fuel-burning appliances until draft/CO is verified.

Analysis of GPT-5.2 (Low Effort):

Entry X gives a balanced, prioritized differential that plausibly ties together pressure-switch open/rollout code, wind correlation, icing/crust at termination, increased condensate activity/overflow, slight exhaust smell, and intermittent CO alarm via venting/condensate restriction plus pressure/combustion-air/backdraft interactions. It appropriately keeps cracked heat exchanger on the list but not as the single assumed cause. Safety messaging is strong (shut down, ventilate, leave if alarms/symptoms; don’t run backdrafting appliances) and avoids bypassing safeties. Clarity is decent for a homeowner, with specific DIY visual checks (slope/sags, terminations re-freezing, trap/pump clear) and a clear handoff to pro diagnostics. It could be improved by giving more concrete expected manometer values (e.g., pressure switch setpoint in in. w.c. from rating plate, typical inducer draft range) and more explicit CO ppm action thresholds, but it at least mentions targets and key measurements (draft, combustion test, worst-case depressurization). Judgment on DIY vs pro is appropriate: limited safe homeowner actions, then immediate qualified tech.

Analysis of Claude Opus 4.6 (High Think):

Entry Y leads with strong, clear safety warnings which is appropriate. However, it jumps too quickly to "cracked secondary heat exchanger" as the MOST LIKELY root cause. While this must be ruled out, the symptom pattern (failures more frequent on windy days, ice/crust on exhaust termination, leaves in intake, flame wavering at water heater draft hood) actually points more strongly toward vent/pressure issues first. The "whoof" sound CAN indicate rollout from a cracked heat exchanger, but it can also occur from delayed ignition due to pressure/draft issues - which Entry Y acknowledges but dismisses too quickly. The claim that "condensate leaking through the crack explains increased pump cycling" is technically questionable - a cracked heat exchanger doesn't typically increase condensate production; blocked drainage or vent issues do. Entry Y's safety advice is excellent and clear. The DIY guidance is appropriately conservative. However, the diagnosis is less thorough - it doesn't adequately address the condensate pump overflow, the specific wind correlation, or the intake/exhaust termination issues that were already observed. The delta >2ppm CO test for heat exchanger crack is a good specific measurement to mention.

Analysis of GPT-5.2 (Low Effort):

Entry X provides a comprehensive differential diagnosis that accounts for all symptoms. It correctly prioritizes vent/condensate restriction as the most likely cause, with negative basement pressure/backdrafting as a secondary factor, and appropriately keeps heat exchanger crack as a possibility to rule out. The safety warnings are appropriate and immediate (shut furnace OFF, ventilate, leave if alarm sounds again). The DIY checks are practical and don't bypass safeties - inspecting PVC terminations, checking exhaust slope, shop-vac condensate trap, etc. The pro measurements are specific and technically sound (manometer across pressure switch, CO in flue <100 ppm air-free, CO in supply 0). Entry X does a better job of explaining how different failure modes can mimic each other and provides a more nuanced diagnostic approach. The mention of wind effects on termination and the water heater backdrafting when inducer starts shows good understanding of the interplay between systems. However, the compressed format makes it somewhat harder to follow for a homeowner.

Analysis of Claude Opus 4.6 (High Think):

Unable to analyze

Analysis of GPT-5.2 (Low Effort):

Unable to analyze