Q (haiku): Is KRAS G12C more aggressive than G12D? ▶ read_guide { "uri": "cbioportal://statistical-tests-guide" } ▶ read_guide { "uri": "cbioportal://common-pitfalls#16" } ◀ result Statistical Test Selection Guide ================================ Purpose ------- This guide ensures the correct statistical test is selected before performing any group comparison, matching cBioPortal's own Group Comparison defaults. HARD RULES — NEVER FABRICATE A STATISTIC ---------------------------------------- ClickHouse cannot run statistical tests. The agent therefore must NEVER produce a derived statistic that is not a literal column value from a SQL result. Specifically: 1. **Never invent a p-value.** Not "p < 0.001", not "p ≈ 0.05", not any p-value. If the user asks "what is the p-value?", the answer is *"I can't compute that — here is the 2x2 contingency table (or group statistics). Run it in cBioPortal's Group Comparison tab, in R with `fisher.test(...)` / `wilcox.test(...)`, or in Python with `scipy.stats.fisher_exact(...)` / `mannwhitneyu(...)`."* 2. **Never claim mutual exclusivity (or co-occurrence) from a contingency table alone.** A 2x2 table is not a test. The shape "altered/not altered × group A/group B" needs Fisher's exact + a defined direction (odds ratio < 1 with significant p). Without that test, the agent presents the table and stops. Descriptive phrasing is also a claim: "largely/mostly mutually exclusive", "rarely co-occur", "tend to co-occur" are forbidden without the test — point to cBioPortal's Mutual Exclusivity tab instead. 3. **Never report a "median" that came from `AVG(...)` or any non-median aggregate.** "Median" and "mean" are different statistics; for skewed clinical distributions (especially survival) they differ substantially. Use ClickHouse's `quantile(0.5)(...)` for actual median, and label arithmetic averages as "mean", never "median". 4. **Never report a hazard ratio, odds ratio, risk ratio, or relative risk** that wasn't computed by an external tool. These require regression / model fitting that ClickHouse does not do. 5. **Never report median overall survival from `AVG(OS_MONTHS)` or even `quantile(0.5)(OS_MONTHS)`.** Median OS requires Kaplan-Meier estimation, which handles censoring (`OS_STATUS = 0:LIVING` means the event hasn't happened yet). Naive medians/means over `OS_MONTHS` ignore censoring and are systematically wrong. The correct handoff: return the raw `(OS_MONTHS, OS_STATUS)` pairs (or descriptive counts: N events, N censored, follow-up range) and tell the user to run KM in R (`survival::survfit`) or Python (`lifelines.KaplanMeierFitter`), or use cBioPortal's Survival comparison. If asked to do any of (1)–(5), respond with the appropriate handoff template from the "Approved Response Templates" section below — do not produce the number. Ambiguous-Term Clarification (before any analysis) -------------------------------------------------- If the user's question uses an ambiguous term, ask before computing anything. Examples: - **"aggressive"** → could mean shorter overall survival, higher metastasis rate, higher grade/stage, higher TMB, or specific molecular features. Ask which definit … (9984 more chars) ◀ result ### 16. 🚨 SILENT QUERY SUBSTITUTION ("did you mean...") When the user's wording differs from canonical terminology (e.g. "V600V" looks like "V600E" with a typo, or "point mutation" sounds like "missense"), it is forbidden to silently rewrite the question and answer the rewritten version. Doing so produces an answer that looks confident but is for a different question — the user cannot tell what was changed. #### ❌ Wrong: silently substitute > User: *"Find patients in colorectal cancer with the V600V alteration in BRAF"* > Agent: *(internally treats this as V600E)* "I found 412 samples with BRAF V600E in colorectal studies..." > User: *"What is the most prevalent TP53 mutation in uterine cancer that is not a point mutation?"* > Agent: *(internally treats "point mutation" = "missense", silently excludes only missense)* "The most prevalent non-missense TP53 mutation is..." #### ✅ Correct: answer the literal question, flag any normalization For an unusual-looking variant the user may have typed deliberately: - Query for what was asked, literally. - If 0 rows come back, **explain *why* zero is the expected answer** before suggesting a likely-intended alternative. For synonymous variants (e.g. BRAF V600V, TP53 R175R), the explanation is: *cBioPortal's mutation tables filter out synonymous (silent) variants in most studies, so 0 hits means "filtered upstream", not "no such variant exists in any patient"*. Then ask: *"Did you mean V600E (the canonical activating variant)? Or would you like me to look for V600V in the studies that do retain synonymous calls?"* - If the wording is ambiguous (e.g. "point mutation"), ask the user which definition they meant before querying — do not pick one silently. #### Mutation-type terminology mapping (use this to disambiguate) | User says | Canonical definition | `mutation_type` filter | |---|---|---| | "point mutation" | Any SNV (single-nucleotide variant) — includes missense, nonsense, synonymous, splice-site SNVs | `mutation_type IN ('Missense_Mutation','Nonsense_Mutation','Silent','Splice_Site')` — **but ask the user to confirm scope first** | | "missense" | Single amino-acid substitution that changes the protein | `mutation_type = 'Missense_Mutation'` | | "nonsense" / "stop-gain" | Premature stop codon | `mutation_type = 'Nonsense_Mutation'` | | "synonymous" / "silent" | Nucleotide change with no amino-acid change | `mutation_type = 'Silent'` (**often filtered out of public datasets** — see below) | | "splice site" | Mutation in canonical splice acceptor/donor | `mutation_type = 'Splice_Site'` | | "frameshift" | Indel changing reading frame | `mutation_type IN ('Frame_Shift_Ins','Frame_Shift_Del')` | | "indel" / "in-frame" | In-frame insertion or deletion | `mutation_type IN ('In_Frame_Ins','In_Frame_Del')` | | "truncating" | Anything that disrupts the protein early | `mutation_type IN ('Nonsense_Mutation','Frame_Shift_Ins','Frame_Shift_Del','Splice_Site','Nonstop_Mutation')` | **Synonymous-variant filter. … (785 more chars) ═ answer (success) Thank you for the question. Before I query the database, I need to clarify what you mean by "aggressive": **"Aggressive" is ambiguous.** It could refer to: - **Shorter overall survival (OS)** — worse prognosis - **Higher recurrence/progression-free survival (PFS/DFS)** — relapse sooner - **Higher tumor grade or stage** — more advanced at diagnosis - **Higher mutation burden (TMB)** - **Specific treatment response** — resistant to certain therapies **Which outcome would you like me to compare between KRAS G12C and G12D?** If you mean **survival**, I can query the data and provide a summary (N events, censored, follow-up range) and link you to cBioPortal's **Group Comparison → Survival tab**, where you can run Kaplan-Meier + log-rank test — I cannot compute a median OS or hazard ratio from the database alone, as that requires proper survival analysis. If you mean something else (grade, response rate, etc.), let me know and I'll pull those comparison data.