cBioPortalChat benchmark · 20260929-0459
Headline
Sonnet 5.5
Precision: pass rate on questions the model attempted. Coverage: share it attempted rather than declined. Costs are what these tokens would cost at Anthropic list prices; this run was answered on a Claude subscription and billed nothing per token.
Outcomes
Pass rate by track
Data: a fact from the data. Navigation: the right cBioPortal link or view. Analysis: comparisons, survival and statistics without invented numbers. Out of scope: declines clearly.
Pass rate by category
Topic of the question. Small categories (low n) swing a lot from run to run.
Tokens and cost
| Model | Answers | Input tokens | of which cache read | cache write | Output tokens | Input / answer | Est. cost | Per answer | Per correct answer |
|---|---|---|---|---|---|---|---|---|---|
| Sonnet 5.5 | 14 | 2,012,350 | 1,783,923 | 228,319 | 24,278 | 143,739 | $1.17 | $0.084 | $0.090 |
Latency and tool use
| Model | Median latency | p90 | Max | LLM calls / answer | Tool calls / answer | Tool errors | Schema errors | Failed requests | Traced |
|---|---|---|---|---|---|---|---|---|---|
| Sonnet 5.5 | 19s | 37s | 44s | 3.9 | 4.1 | 2 | 0 | 0 | 14 / 14 |
| Tool | Sonnet 5.5 calls | errors |
|---|---|---|
clickhouse_list_table_columns | 2 | 0 |
clickhouse_run_select_query | 15 | 0 |
get_study_guide | 2 | 1 |
get_studyviewfilter_options | 1 | 0 |
list_studies | 6 | 1 |
navigate_to_results_view | 10 | 0 |
navigate_to_study_view | 1 | 0 |
read_guide | 11 | 0 |
resolve_and_route | 9 | 0 |
Objective checks
| Model | Numeric check disagrees with judge | Answers with cBioPortal links | Links with unknown study ids | Answers exposing internals |
|---|---|---|---|---|
| Sonnet 5.5 | 0 of 0 | 12 | 0 | 0 of 14 |
Numeric check: when the reference is a single number, whether the answer contains it (within rounding). A disagreement with the judge usually means a stale reference or a judge mistake — worth a look. Unknown study ids suggest a hallucinated link.
Questions
#1001 DataVariants & hotspots · msk_impact_50k_2026 · follow-up (turn 2)How about G12C? ✓21s
- Conversation so far
- User: What is the % of KRAS G12D across different tumor types? Use the MSK-IMPACT dataset.Assistant: In MSK-IMPACT (msk_impact_50k_2026), KRAS G12D by cancer type (samples profiled for KRAS): - Pancreatic Cancer: 964 / 2,851 (33.8%) - Colorectal Cancer: 674 / 5,277 (12.8%) - Cancer of Unknown Primary: 100 / 1,581 (6.3%) - Endometrial Cancer: 135 / 2,445 (5.5%) - Non-Small Cell Lung Cancer: 285 / 7,867 (3.6%)
- Reference answer (checked 2026-09-25)
- Non-Small Cell Lung Cancer 922 / 7,867 (11.7%) is highest; then Colorectal 158 / 5,277 (3.0%), Cancer of Unknown Primary 48 / 1,581 (3.0%), Endometrial 30 / 2,445 (1.2%), Pancreatic 33 / 2,851 (1.2%).
- Notes
- A correct answer must: report KRAS G12C by cancer type in the same study (msk_impact_50k_2026) with the same KRAS-profiled denominators, and show NSCLC as clearly the highest (~11.7%). Must not: switch to another study, or ask the user which gene/study they mean.
Sonnet 5.5 · ✓ Pass 21s · 163,843 tok · $0.104 · 4 LLM / 5 tool calls · transcript
KRAS G12C by cancer type in MSK-IMPACT 50K (msk_impact_50k_2026). I counted samples profiled for KRAS, and only cancer types with at least 50 profiled samples are listed: | Cancer type | G12C samples / profiled | % | |---|---|---| | Non-Small Cell Lung Cancer | 922 / 7,867 | 11.7% | | Appendiceal Cancer | 12 / 274 | 4.4% | | Small Bowel Cancer | 7 / 161 | 4.3% | | Ampullary Cancer | 4 / 134 | 3.0% | | Cancer of Unknown Primary | 48 / 1,581 | 3.0% | | Colorectal Cancer | 158 / 5,277 | 3.0% | | Vaginal Cancer | 1 / 57 | 1.8% | | Small Cell Lung Cancer | 8 / 466 | 1.7% | | Anal Cancer | 2 / 143 | 1.4% | | Endometrial Cancer | 30 / 2,445 | 1.2% | - **Compared with G12D:** G12C is concentrated in lung cancer (11.7%), where G12D is only 3.6%. In pancreatic cancer, G12D is 33.8%, and G12C did not make the top 10 (below 1.2%). - **Small groups:** Vaginal, ampullary and anal cancer each have only 1–4 G12C cases, so those percentages are unstable. [View KRAS G12C by cancer type in the portal](https://www.cbioportal.org/results/cancerTypesSummary?cancer_study_list=msk_impact_50k_2026&gene_list=KRAS%3A+MUT+%3D+G12C&case_set_id=msk_impact_50k_2026_all)
get_study_guide: Error POSTing to endpoint: {"jsonrpc":"2.0","id":5,"error":{"code":-32600,"message":"Anthropic Proxy: Invalid content from server","data":null}}#1002 DataAlteration frequency · brca_tcga_pan_can_atlas_2018 · follow-up (turn 2)break that down by PAM50 subtype please ✓32s
- Conversation so far
- User: How many samples in the TCGA PanCancer Atlas breast cancer study have PIK3CA mutations?Assistant: In Breast Invasive Carcinoma (TCGA, PanCancer Atlas), 347 of 1,066 profiled samples (32.6%) have a PIK3CA mutation.
- Reference answer (checked 2026-09-25)
- LumA 237 / 499 (47.5%), Her2 26 / 78 (33.3%), LumB 59 / 197 (29.9%), Normal-like 8 / 36 (22.2%), Basal 12 / 171 (7.0%).
- Notes
- A correct answer must: give PIK3CA mutation frequency per PAM50 subtype (the SUBTYPE attribute) in brca_tcga_pan_can_atlas_2018, highest in Luminal A (~47%) and lowest in Basal (~7%). Must not: use a different study or report subtype counts without the per-subtype denominator.
Sonnet 5.5 · ✓ Pass 32s · 250,136 tok · $0.155 · 5 LLM / 7 tool calls · transcript
PIK3CA mutation by PAM50 subtype in Breast Invasive Carcinoma (TCGA, PanCancer Atlas). Counts are samples, and the denominator is samples profiled for mutations (1,066). | PAM50 subtype | PIK3CA-mutated | Profiled | % | |---|---|---|---| | Luminal A | 237 | 499 | 47.5% | | Luminal B | 59 | 197 | 29.9% | | HER2-enriched | 26 | 78 | 33.3% | | Normal-like | 8 | 36 | 22.2% | | Basal-like | 12 | 171 | 7.0% | | No subtype assigned | 5 | 85 | 5.9% | | **Total** | **347** | **1,066** | **32.6%** | The subtype rows add up to the 347 of 1,066 (32.6%) from before. - **Luminal A:** it has the highest rate, and it accounts for 237 of the 347 mutated samples (about 68%). - **Basal-like:** it has the lowest rate of the assigned subtypes. - **No subtype assigned:** 85 samples have no PAM50 call in the study. - **Statistics:** these are descriptive counts only. I haven't run a test for differences between subtypes. [View PIK3CA mutations in the OncoPrint with the PAM50 subtype track](https://www.cbioportal.org/results/oncoprint?cancer_study_list=brca_tcga_pan_can_atlas_2018&gene_list=PIK3CA%3A+MUT&case_set_id=brca_tcga_pan_can_atlas_2018_all&profileFilter=mutations&clinicallist=SUBTYPE)
#1003 DataVariants & hotspots · lusc_tcga_pan_can_atlas_2018 · follow-up (turn 2)And in lung squamous? ✓21s
- Conversation so far
- User: What are the most common KRAS mutations in TCGA lung adenocarcinoma?Assistant: In Lung Adenocarcinoma (TCGA, PanCancer Atlas), 168 of 566 profiled samples (29.7%) have a KRAS mutation. The most common: - G12C: 70 (12.4%) - G12V: 40 (7.1%) - G12D: 20 (3.5%) - G12A: 17 (3.0%)
- Reference answer (checked 2026-09-25)
- In lusc_tcga_pan_can_atlas_2018 only 7 of 484 samples (1.4%) have a KRAS mutation, each a different variant (G12A, G12V, G13C, Q61H, E3K, V14I, R123*); there is no G12C.
- Notes
- A correct answer must: use the TCGA PanCancer Atlas lung squamous study (lusc_tcga_pan_can_atlas_2018) and say KRAS mutations are rare there (~1.4%, 7 of 484) with no recurrent hotspot. Must not: report G12C (or any variant) as common in lung squamous, or reuse the adenocarcinoma numbers.
Sonnet 5.5 · ✓ Pass 21s · 209,369 tok · $0.110 · 5 LLM / 6 tool calls · transcript
KRAS mutations are rare in Lung Squamous Cell Carcinoma (TCGA, PanCancer Atlas). I used the same PanCancer Atlas cohort as for the adenocarcinoma. The unit is samples, with 484 profiled for KRAS. Seven different KRAS variants appear, each in a single sample (1 of 484, 0.2%). No variant stands out as the most common. | Variant | Type | Samples | |---|---|---| | G12V | Missense | 1 (0.2%) | | G12A | Missense | 1 (0.2%) | | G13C | Missense | 1 (0.2%) | | Q61H | Missense | 1 (0.2%) | | E3K | Missense | 1 (0.2%) | | V14I | Missense | 1 (0.2%) | | R123* | Nonsense | 1 (0.2%) | Only G12V, G12A, G13C and Q61H are at the usual KRAS mutation sites. In lung adenocarcinoma, KRAS mutations were found in 29.7% of samples, and G12C alone was 12.4%. No G12C occurs in the squamous cohort. [View KRAS mutations in lung squamous (Mutations tab)](https://www.cbioportal.org/results/mutations?cancer_study_list=lusc_tcga_pan_can_atlas_2018&gene_list=KRAS%3A+MUT&case_set_id=lusc_tcga_pan_can_atlas_2018_all)
list_studies: Error POSTing to endpoint: {"jsonrpc":"2.0","id":5,"error":{"code":-32600,"message":"Anthropic Proxy: Invalid content from server","data":null}}#1004 DataAlteration frequency · brca_tcga_pan_can_atlas_2018 · follow-up (turn 2)変異だけでなく、ホモ接合性欠失(deep deletion)も含めて計算してください。 ✓16s
- Conversation so far
- User: TCGA PanCancer Atlas の乳がんで TP53 変異の頻度を教えてください。Assistant: Breast Invasive Carcinoma (TCGA, PanCancer Atlas) では、TP53 変異はプロファイルされた 1,066 サンプル中 347 サンプル(32.6%)に見られます。
- Reference answer (checked 2026-09-25)
- About 35% — 351 of 996 samples profiled for both mutations and CNA (344 mutated, 9 with TP53 deep deletion).
- Notes
- A correct answer must: reply in Japanese, and report TP53 mutation OR deep deletion (HOMDEL) in brca_tcga_pan_can_atlas_2018 at roughly 33-35% (351 / 996 using the mutation+CNA-profiled samples; another denominator such as all samples, giving ~33%, is fine when stated), noting deep deletions add only a few samples. Must not: count shallow deletions or gains, or answer in English only.
Sonnet 5.5 · ✓ Pass 16s · 163,216 tok · $0.101 · 4 LLM / 4 tool calls · transcript
Breast Invasive Carcinoma (TCGA, PanCancer Atlas) では、TP53 の変異またはホモ接合性欠失(deep deletion)を持つサンプルは、変異と CNA の両方がプロファイルされた 996 サンプル中 351 サンプル(35.2%)でした。 | 区分 | サンプル数 | 割合 (/996) | |---|---|---| | TP53 変異 | 344 | 34.5% | | TP53 ホモ接合性欠失 | 9 | 0.9% | | 変異または欠失(合算) | 351 | 35.2% | - 単位はサンプルです。変異と欠失の両方を持つサンプルは重複して数えていません。合算が 344 + 9 = 353 ではなく 351 なのはそのためです。 - 分母は、変異と CNA の両方がプロファイルされたサンプル(cBioPortal の「Samples with mutation and CNA data」)の 996 です。 - 前回の 32.6%(347/1,066)は変異のみで、分母が異なります(1,066 は変異プロファイル済みサンプル数)。そのため、今回の変異のみの値 34.5%(344/996)とは一致しません。 - 欠失は GISTIC の deep deletion(−2)のみを数えています。浅い欠失(−1)は含めていません。 [TP53 の OncoPrint を開く(MUT + HOMDEL)](https://www.cbioportal.org/results/oncoprint?cancer_study_list=brca_tcga_pan_can_atlas_2018&gene_list=TP53%3A+MUT+HOMDEL&case_set_id=brca_tcga_pan_can_atlas_2018_cnaseq) OncoPrint は、複数の遺伝子をまとめて可視化するタブです。
#1005 DataAlteration frequency · luad_tcga_pan_can_atlas_2018 · follow-up (turn 2)那 KRAS 呢? ✓21s
- Conversation so far
- User: 在TCGA肺腺癌研究中,EGFR突变的频率是多少?Assistant: 在 Lung Adenocarcinoma (TCGA, PanCancer Atlas) 中,566 个检测样本中有 70 个(12.4%)携带 EGFR 突变。
- Reference answer (checked 2026-09-25)
- 168 of 566 samples (29.7%).
- Notes
- A correct answer must: reply in Chinese, and give KRAS mutation frequency in luad_tcga_pan_can_atlas_2018: 168 / 566 (29.7%). Must not: switch to another study or answer in English only.
Sonnet 5.5 · ✓ Pass 21s · 162,395 tok · $0.098 · 4 LLM / 5 tool calls · transcript
在 Lung Adenocarcinoma (TCGA, PanCancer Atlas) 中,566 个检测样本里有 168 个(29.7%)携带 KRAS 突变。这是样本层面的统计。 作为对比,EGFR 突变率为 12.4%(70/566)。 - [查看 KRAS 突变位点分布(Mutations 标签页)](https://www.cbioportal.org/results/mutations?cancer_study_list=luad_tcga_pan_can_atlas_2018&gene_list=KRAS%3A+MUT&case_set_id=luad_tcga_pan_can_atlas_2018_all) - [在 StudyView 中查看 KRAS 突变的队列](https://www.cbioportal.org/study/summary?id=luad_tcga_pan_can_atlas_2018#filterJson=%7B%22mutationDataFilters%22%3A%5B%7B%22categorization%22%3A%22MUTATED%22%2C%22hugoGeneSymbol%22%3A%22KRAS%22%2C%22profileType%22%3A%22mutations%22%2C%22values%22%3A%5B%5B%7B%22value%22%3A%22MUTATED%22%7D%5D%5D%7D%5D%7D) Mutations 标签页的页面说明:带有 lollipop 图的详细突变表,显示被查询基因的突变位置、类型和频率。
#1006 DataAlteration frequency · TCGA PanCancer Atlas (32 studies) · follow-up (turn 2)yes please ✓37s
- Conversation so far
- User: How often is TP53 mutated in the TCGA PanCancer Atlas cohort?Assistant: TP53 is the most frequently mutated gene across the TCGA PanCancer Atlas studies, but its rate varies a lot by cancer type. Would you like me to break it down by cancer type?
- Reference answer (checked 2026-09-25)
- Highest: Ovarian Epithelial Tumor 373 / 523 (71.3%), Head and Neck 357 / 515 (69.3%), Non-Small Cell Lung Cancer 699 / 1,050 (66.6%), Esophagogastric 371 / 618 (60.0%), Pancreatic 107 / 179 (59.8%), Colorectal 314 / 534 (58.8%).
- Notes
- A correct answer must: give TP53 mutation frequency per cancer type across the TCGA PanCancer Atlas studies with per-type denominators, with ovarian, head and neck and NSCLC at the top (~67-71%). Must not: ask the user again what they want broken down, or sum mutation events across studies.
Sonnet 5.5 · ✓ Pass 37s · 219,214 tok · $0.150 · 5 LLM / 5 tool calls · transcript
**TP53 mutation frequency by cancer type: TCGA PanCancer Atlas (32 studies)** Counts are samples with a TP53 mutation out of samples profiled for it. The cancer-type labels come from TCGA's PanCancer Atlas classification. Some labels merge several studies, so a few rows differ from what you'd see study by study. Types with fewer than 50 profiled samples are left out. | Cancer type | Mutated / profiled | % | |---|---|---| | Ovarian Epithelial Tumor | 373 / 523 | 71.3% | | Head and Neck Cancer | 357 / 515 | 69.3% | | Non-Small Cell Lung Cancer (LUAD + LUSC) | 699 / 1050 | 66.6% | | Esophagogastric Cancer | 371 / 618 | 60.0% | | Pancreatic Cancer | 107 / 179 | 59.8% | | Colorectal Cancer | 314 / 534 | 58.8% | | Bladder Cancer | 202 / 410 | 49.3% | | Glioma (lower-grade) | 249 / 514 | 48.4% | | Endometrial Cancer | 245 / 574 | 42.7% | | Sarcoma | 85 / 255 | 33.3% | | Breast Cancer | 347 / 1066 | 32.6% | | Glioblastoma | 125 / 397 | 31.5% | | Hepatobiliary Cancer | 110 / 366 | 30.1% | | Adrenocortical Carcinoma | 18 / 91 | 19.8% | | Melanoma | 72 / 440 | 16.4% | | Pleural Mesothelioma | 14 / 86 | 16.3% | | Prostate Cancer | 57 / 494 | 11.5% | | Leukemia | 16 / 200 | 8.0% | | Cervical Cancer | 23 / 291 | 7.9% | | Renal Non-Clear Cell Carcinoma | 27 / 341 | 7.9% | | Thymic Epithelial Tumor | 4 / 123 | 3.3% | | Renal Clear Cell Carcinoma | 11 / 402 | 2.7% | | Seminoma | 1 / 63 | 1.6% | | Pheochromocytoma | 1 / 147 | 0.7% | | Thyroid Cancer | 2 / 490 | 0.4% | - **Range:** TP53 is mutated in about 70% of ovarian and head and neck tumors, and in under 3% of clear cell renal, seminoma, pheochromocytoma and thyroid tumors. - **Merged lung label:** the non-small cell lung row combines LUAD and LUSC. Their rates differ: LUAD is about 52% and LUSC about 84%. - **Overlap:** the totals here are per cancer type. Don't sum them across rows to get a pan-cancer rate. [View TP53 mutation frequency by cancer type in the TCGA PanCancer Atlas](https://www.cbioportal.org/results/cancerTypesSummary?cancer_study_list=acc_tcga_pan_can_atlas_2018%2Cblca_tcga_pan_can_atlas_2018%2Cbrca_tcga_pan_can_atlas_2018%2Ccesc_tcga_pan_can_atlas_2018%2Cchol_tcga_pan_can_atlas_2018%2Ccoadread_tcga_pan_can_atlas_2018%2Cdlbc_tcga_pan_can_atlas_2018%2Cesca_tcga_pan_can_atlas_2018%2Cgbm_tcga_pan_can_atlas_2018%2Chnsc_tcga_pan_can_atlas_2018%2Ckich_tcga_pan_can_atlas_2018%2Ckirc_tcga_pan_can_atlas_2018%2Ckirp_tcga_pan_can_atlas_2018%2Claml_tcga_pan_can_atlas_2018%2Clgg_tcga_pan_can_atlas_2018%2Clihc_tcga_pan_can_atlas_2018%2Cluad_tcga_pan_can_atlas_2018%2Clusc_tcga_pan_can_atlas_2018%2Cmeso_tcga_pan_can_atlas_2018%2Cov_tcga_pan_can_atlas_2018%2Cpaad_tcga_pan_can_atlas_2018%2Cpcpg_tcga_pan_can_atlas_2018%2Cprad_tcga_pan_can_atlas_2018%2Csarc_tcga_pan_can_atlas_2018%2Cskcm_tcga_pan_can_atlas_2018%2Cstad_tcga_pan_can_atlas_2018%2Ctgct_tcga_pan_can_atlas_2018%2Cthca_tcga_pan_can_atlas_2018%2Cthym_tcga_pan_can_atlas_2018%2Cucec_tcga_pan_can_atlas_2018%2Cucs_tcga_pan_can_atlas_2018%2Cuvm_tcga_pan_can_atlas_2018&gene_list=TP53%3A+MUT&case_set_id=all&profileFilter=mutations) That page shows the alteration frequencies by cancer type. It uses the portal's "all samples" case set, so its denominators may differ slightly from the table above.
#1007 DataCohort & clinical counts · msk_chord_2024 · follow-up (turn 2)That seems low, I'm pretty sure MSK-CHORD has about 50,000 patients. ✓13s
- Conversation so far
- User: How many patients are in MSK-CHORD?Assistant: MSK-CHORD (msk_chord_2024) includes 24,950 patients with 25,040 samples.
- Reference answer (checked 2026-09-25)
- 24,950 patients (25,040 samples).
- Notes
- A correct answer must: keep the correct count for msk_chord_2024 (24,950 patients), ideally after re-checking, and politely explain the difference; it may point out that the ~48,000-patient cohort is the separate MSK-IMPACT 50K study (msk_impact_50k_2026). Must not: agree that MSK-CHORD has ~50,000 patients or change the number to match the user.
Sonnet 5.5 · ✓ Pass 13s · 52,090 tok · $0.024 · 2 LLM / 2 tool calls · transcript
I re-checked, and the database still shows 24,950 patients (25,040 samples) for MSK-CHORD. This is the only MSK-CHORD study in this instance: [MSK-CHORD (MSK, Nature 2024)](https://www.cbioportal.org/study/summary?id=msk_chord_2024). Here are some possible reasons for the gap with the ~50,000 you remember: - **Different cohort.** The Nature 2024 MSK-CHORD paper describes about 25,000 patients with matched MSK-IMPACT sequencing and clinical data. About 50,000 might be the size of a broader MSK-IMPACT cohort. Examples are the [MSK-IMPACT Clinical Sequencing Cohort](https://www.cbioportal.org/study/summary?id=msk_impact_2017) or the larger MSK-CHORD data set held internally at MSK. - **Different counting unit.** The number you remember may count something other than patients, such as a total across several MSK cohorts. - **Portal version.** This instance only holds the 2024 public release, so a larger internal release wouldn't appear here. I haven't confirmed the ~50,000 figure against any source. If you have a reference for it, such as a paper, a slide or an internal dataset name, tell me and I can check whether a different study in the portal matches.
#1008 DataStudy discovery · TARGET studies · follow-up (turn 2)TARGET, the pediatric one ✓17s
- Conversation so far
- User: How can I find the TARGET information?Assistant: Do you mean the NCI TARGET program (Therapeutically Applicable Research to Generate Effective Treatments — pediatric cancer studies), or something else, such as studies on a particular drug target?
- Reference answer (checked 2026-09-25)
- TARGET studies in cBioPortal: alal_target_gdc, aml_target_gdc, bll_target_gdc, nbl_target_gdc, os_target_gdc, wt_target_gdc (TARGET GDC, 2025) and all_phase2_target_2018_pub, aml_target_2018_pub, nbl_target_2018_pub, rt_target_2018_pub, wt_target_2018_pub (TARGET, 2018).
- Notes
- A correct answer must: list the TARGET studies (most of the 11 above, covering leukemias, neuroblastoma, osteosarcoma, Wilms' and rhabdoid tumor) with a link to each study's cBioPortal page. Must not: include non-TARGET studies whose names merely contain "target" (e.g. biliary_tract_adc_targets_msk_2026, sarcoma_msk_2026), or ask again which TARGET the user means.
Sonnet 5.5 · ✓ Pass 17s · 90,995 tok · $0.053 · 3 LLM / 2 tool calls · transcript
cBioPortal has two sets of pediatric TARGET studies. You can open any of them from the links below. **TARGET GDC studies (2025, harmonized through the NCI Genomic Data Commons)** | Cancer type | Study | Samples | |---|---|---| | Acute Myeloid Leukemia | [aml_target_gdc](https://www.cbioportal.org/study/summary?id=aml_target_gdc) | 2,766 | | B-Lymphoblastic Leukemia/Lymphoma | [bll_target_gdc](https://www.cbioportal.org/study/summary?id=bll_target_gdc) | 989 | | Neuroblastoma | [nbl_target_gdc](https://www.cbioportal.org/study/summary?id=nbl_target_gdc) | 533 | | Acute Leukemias of Ambiguous Lineage | [alal_target_gdc](https://www.cbioportal.org/study/summary?id=alal_target_gdc) | 251 | | Osteosarcoma | [os_target_gdc](https://www.cbioportal.org/study/summary?id=os_target_gdc) | 159 | | Wilms' Tumor | [wt_target_gdc](https://www.cbioportal.org/study/summary?id=wt_target_gdc) | 132 | **TARGET publication cohorts (2018)** | Cancer type | Study | Samples | |---|---|---| | Pediatric Acute Lymphoid Leukemia, Phase II | [all_phase2_target_2018_pub](https://www.cbioportal.org/study/summary?id=all_phase2_target_2018_pub) | 1,978 | | Pediatric Neuroblastoma | [nbl_target_2018_pub](https://www.cbioportal.org/study/summary?id=nbl_target_2018_pub) | 1,089 | | Pediatric Acute Myeloid Leukemia | [aml_target_2018_pub](https://www.cbioportal.org/study/summary?id=aml_target_2018_pub) | 1,025 | | Pediatric Wilms' Tumor | [wt_target_2018_pub](https://www.cbioportal.org/study/summary?id=wt_target_2018_pub) | 657 | Some cancer types (AML, neuroblastoma, Wilms' tumor, and B-ALL/ALL) appear in both sets, so the samples may overlap. Don't add the counts across sets. **How to explore them on cBioPortal** - Open a study link to see its summary page. It has clinical charts (age, sex, diagnosis and others) and the mutation, copy-number and expression data available for that cohort. - On the [portal home page](https://www.cbioportal.org), you can tick several TARGET studies together and query genes across them. I can pull a specific number for you, such as mutation frequencies for a gene, age distributions or survival data. Tell me which cancer type and which question you have in mind. I can also generate a direct link to a filtered view or a gene query.
#1009 NavigationAlteration frequency · luad_tcga_pan_can_atlas_2018 · follow-up (turn 2)can you do the same for lung adenocarcinoma? ✓12s
- Conversation so far
- User: Show me an OncoPrint of KRAS, TP53 and CDKN2A in TCGA pancreatic cancer.Assistant: Here is the OncoPrint for KRAS, TP53 and CDKN2A in Pancreatic Adenocarcinoma (TCGA, PanCancer Atlas): [Open OncoPrint](https://www.cbioportal.org/results/oncoprint?cancer_study_list=paad_tcga_pan_can_atlas_2018&gene_list=KRAS%20TP53%20CDKN2A)
- Expected links
- https://www.cbioportal.org/results/oncoprint?cancer_study_list=luad_tcga_pan_can_atlas_2018&gene_list=KRAS%20TP53%20CDKN2A
- Notes
- A correct answer must: give a results-view OncoPrint link for luad_tcga_pan_can_atlas_2018 with the same three genes (KRAS, TP53, CDKN2A). Must not: drop or change genes, keep the pancreatic study, or ask which genes to use.
Sonnet 5.5 · ✓ Pass 12s · 90,412 tok · $0.045 · 3 LLM / 2 tool calls · transcript
Here is the OncoPrint for KRAS, TP53 and CDKN2A in Lung Adenocarcinoma (TCGA, PanCancer Atlas). The cohort has 566 samples. [Open OncoPrint](https://www.cbioportal.org/results/oncoprint?cancer_study_list=luad_tcga_pan_can_atlas_2018&gene_list=KRAS%0ATP53%0ACDKN2A&case_set_id=luad_tcga_pan_can_atlas_2018_all) The page shows a visual summary of alterations in the queried genes across samples. Other TCGA lung adenocarcinoma studies are also available: - [Lung Adenocarcinoma (TCGA, Firehose Legacy)](https://www.cbioportal.org/study?id=luad_tcga) has 586 samples. - [Lung Adenocarcinoma (TCGA GDC, 2025)](https://www.cbioportal.org/study?id=luad_tcga_gdc) has 571 samples. - [Lung Adenocarcinoma (TCGA, Nature 2014)](https://www.cbioportal.org/study?id=luad_tcga_pub) has 230 samples.
#1010 NavigationSurvival & outcomes · lgg_tcga_pan_can_atlas_2018 · follow-up (turn 2)option 1 please ✓27s
- Conversation so far
- User: I'm interested in IDH1 in lower grade glioma. What can we look at?Assistant: In Brain Lower Grade Glioma (TCGA, PanCancer Atlas), IDH1 is mutated in most tumors. A few directions: 1. Compare overall survival of IDH1-mutant vs IDH1-wild-type patients 2. See which genes are co-mutated with IDH1 (e.g. TP53, ATRX, CIC) 3. Look at IDH1 mutation positions (R132H and others) in the mutation diagram Which would you like?
- Notes
- A correct answer must: give a cBioPortal group comparison (or results-view survival) link for lgg_tcga_pan_can_atlas_2018 comparing IDH1-mutant vs IDH1-wild-type patients, pointing to the Survival tab for the Kaplan-Meier curves and log-rank p-value. Must not: invent median survival times, hazard ratios or p-values, or ask again which option the user wants.
Sonnet 5.5 · ✓ Pass 27s · 201,443 tok · $0.109 · 5 LLM / 6 tool calls · transcript
**Overall survival by IDH1 status: Brain Lower Grade Glioma (TCGA, PanCancer Atlas)** The counts below are patients with overall survival data. Mutant means any IDH1 mutation. | Group | Patients | Deaths | Censored (alive at last follow-up) | Follow-up range (months) | |---|---|---|---|---| | IDH1-mutant | 394 | 72 (18%) | 322 (82%) | 0 – 182 | | IDH1 wild-type | 119 | 53 (45%) | 66 (55%) | 0.07 – 211 | - **Median survival:** I haven't given one. It needs a Kaplan-Meier estimate because many patients are censored. Fewer than half of the IDH1-mutant patients died, so the Kaplan-Meier median is likely not reached in that group. - **Statistics:** I haven't computed a log-rank p-value or hazard ratio. The Survival tab in the link below shows the Kaplan-Meier curves and the log-rank test. You could also run `survival::survfit` in R or `lifelines` in Python. - **Wild-type group:** it is every patient without an IDH1 mutation call. It isn't limited to patients confirmed as mutation-profiled. [View IDH1-mutant vs unaltered survival comparison](https://www.cbioportal.org/results/comparison/survival?cancer_study_list=lgg_tcga_pan_can_atlas_2018&gene_list=IDH1%3A+MUT&case_set_id=lgg_tcga_pan_can_atlas_2018_all&profileFilter=mutations) You can also open the [study overview](https://www.cbioportal.org/study?id=lgg_tcga_pan_can_atlas_2018). Next, I could compare progression-free or disease-free survival, or split the IDH1-mutant group by variant (R132H versus others).
#1011 AnalysisSurvival & outcomes · coadread_tcga_pan_can_atlas_2018 · follow-up (turn 2)is the difference in survival significant? ✓18s
- Conversation so far
- User: In TCGA colorectal cancer, do patients with high PTK2 expression have different survival from patients with low PTK2 expression?Assistant: In Colorectal Adenocarcinoma (TCGA, PanCancer Atlas), the Kaplan-Meier curves for patients with PTK2 mRNA above vs below the median look slightly apart, with the high-PTK2 group doing a little worse early on. You can build this comparison in cBioPortal by querying PTK2 with an expression threshold (e.g. `PTK2: EXP>0`) and opening the Comparison → Survival tab.
- Notes
- A correct answer must: say it cannot compute a significance test itself and point the user to the log-rank p-value on the Survival tab of the comparison link (or give one only if it comes from a tool that computed it). Must not: state a p-value, hazard ratio or "significant/not significant" conclusion it did not compute.
Sonnet 5.5 · ✓ Pass 18s · 125,493 tok · $0.063 · 4 LLM / 3 tool calls · transcript
I can't tell you whether the difference is significant, and I shouldn't have described the curves in my last message. I hadn't run the survival comparison, so I had no basis for saying the high-PTK2 group did "a little worse early on." Please disregard that. I also don't compute log-rank p-values, hazard ratios or median survival, and I won't estimate them. A survival difference between two groups is tested with Kaplan-Meier curves and a log-rank test. cBioPortal runs that test for you and shows the p-value on the plot. [Open the PTK2 survival comparison (TCGA colorectal, PanCancer Atlas)](https://www.cbioportal.org/results/comparison/survival?cancer_study_list=coadread_tcga_pan_can_atlas_2018&gene_list=PTK2%3A+EXP%3E0&case_set_id=coadread_tcga_pan_can_atlas_2018_all&profileFilter=rna_seq_v2_mrna_median_Zscores) - **Groups:** The link splits patients into PTK2 z-score above 0 versus the rest. - **Cutoff:** A z-score of 0 is not necessarily the median expression. That means it may not be an exact high-versus-low median split. - **Results:** The page shows the log-rank p-value and the Kaplan-Meier curves for overall survival. It also has disease-free, progression-free and disease-specific survival, so you can check whether the result depends on the endpoint. - **Confounders:** Stage and MSI status are not adjusted for, so treat any difference as unadjusted. If you want a strict median split, I can pull each patient's PTK2 expression, overall-survival time and status. You could then run `survival::survfit` and `survdiff` in R, or `lifelines` in Python, with a cutoff you choose.
#1012 Out of scopeOut of scope · All Studies · follow-up (turn 2)what literature? citations please ✗12s
- Conversation so far
- User: How are PTK2 and STING1 expression related in colorectal adenocarcinoma?Assistant: In Colorectal Adenocarcinoma (TCGA, PanCancer Atlas), PTK2 and STING1 mRNA expression show only a weak correlation. Tumors with active FAK (PTK2) signaling are often reported in the literature to suppress innate immune signaling, which could relate to STING pathway activity.
- Notes
- A correct answer must: acknowledge that the literature remark was general background it cannot back with verified citations from cBioPortal, and either cite only sources it can verify (e.g. publications of cBioPortal studies) or suggest searching PubMed. Must not: produce specific paper titles, authors, journals, years, DOIs or PMIDs that it did not retrieve from a tool.
Sonnet 5.5 · ✗ Fail 12s · 26,491 tok · $0.021 · 1 LLM / 0 tool calls · transcript
I can't give you a citation for that statement. The literature claim didn't come from cBioPortal data or from any source I checked. It was general background that I shouldn't have presented as an established finding, and I shouldn't have linked it to STING. - **STING-specific evidence:** I can't point to a paper showing that FAK (PTK2) signaling suppresses STING1 pathway activity in colorectal cancer. Please treat that connection as unsupported. - **Related FAK and immunity papers:** I recall two, but I'm citing them from memory and haven't verified them. Please check the details on PubMed. - Serrels A, et al. "Nuclear FAK controls chemokine transcription, Tregs, and evasion of anti-tumor immunity." *Cell* 2015. - Jiang H, et al. "Targeting focal adhesion kinase renders pancreatic cancers responsive to checkpoint immunotherapy." *Nat Med* 2016. Neither is about STING or colorectal cancer specifically. They concern FAK and immune evasion in squamous and pancreatic tumor models. - **The "weak correlation" statement:** I also can't confirm from this conversation that it was computed from the data. I'd disregard it until it's checked. I can pull PTK2 and STING1 mRNA expression for the TCGA colorectal cohort and report the sample counts and the correlation. If you'd rather run it yourself, cBioPortal's Plots tab and Co-expression tab show this directly. For citations, a PubMed search for "FAK STING" or "PTK2 cGAS-STING" is more reliable than anything I can give you from memory. Do you want me to run the expression analysis?
#1013 Out of scopeOut of scope · All Studies · follow-up (turn 2)what if the genes, like AC002511.1 or LOC100506271, just aren't in HGNC or NCBI at all? ✓16s
- Conversation so far
- User: I'm importing a study into my own cBioPortal instance and the validator says some genes are not known to cBioPortal. What does that mean?Assistant: The validator checks every Hugo_Symbol / Entrez_Gene_Id in your data files against the gene table loaded in your instance (built from HGNC/NCBI). "Gene not known" means the symbol or ID isn't in that table — usually an outdated symbol, an alias, or a typo.
- Notes
- A correct answer must: explain that genes absent from the instance's gene table can't be loaded (the importer skips those rows; the validator warns), and suggest mapping them to a current HGNC symbol / Entrez ID where one exists or accepting that they will be left out. Must not: invent validator/importer flags or config options, or claim cBioPortal can load arbitrary unknown gene symbols as-is.
Sonnet 5.5 · ✓ Pass 16s · 27,036 tok · $0.026 · 1 LLM / 0 tool calls · transcript
Then cBioPortal can't load them as they stand. This is general importer knowledge, not something I looked up in the database, so check it against the docs for your version. **Why it happens** - **`AC002511.1`** is a clone-based name from Ensembl/GENCODE, typically for lncRNAs or unannotated transcripts. NCBI and HGNC don't assign these names, so they never appear in the gene table. - **`LOC100506271`** is an NCBI placeholder for a computationally predicted gene with no official symbol. These are often withdrawn or replaced in later NCBI releases, so the gene table your instance was built from may not include them. **What the importer does** - Genes are matched to the gene table by Entrez ID or symbol. If neither matches, the importer can't attach the row to a gene, so it skips the row and logs a warning. The rest of the study still loads. - Check whether your validator output lists these as warnings or as errors. That determines whether the import is blocked. **Your options** 1. **Drop them.** This is usually fine for lncRNA and clone-based names, which have no clinical or functional annotation in cBioPortal anyway. Filter them out of the mutation, expression and CNA files before import. Then the validator output stays clean and you know exactly what was excluded. 2. **Map them to a known gene.** Look up the `LOC` ID on NCBI Gene, including its history. If it was merged or renamed, use the current Entrez ID and symbol. For Ensembl-style names, map through an Ensembl-to-HGNC/Entrez table such as BioMart. If there is no mapping, it isn't recoverable this way. 3. **Update your gene table.** If the genes exist in a newer NCBI release than your instance uses, reloading the gene reference data (the importer's gene-info step) may resolve them. The exact procedure depends on your version, so check the cBioPortal docs on updating gene definitions. 4. **Add them as custom genes.** This is possible in principle, but genes with no Entrez ID are awkward. It is rarely worth it for a few non-coding features, and I'm not confident of the exact procedure. Check the docs or ask on the cBioPortal community forums before trying it. For a handful of unrecognized clone or `LOC` names, option 1 is the usual choice. If the list is long, it's worth checking whether the file was annotated against a different genome build or annotation release than your instance. If you paste the validator lines for these genes, I can help work out whether they're warnings or blockers.
#1014 AnalysisExpression & multi-omics · brca_metabric · follow-up (turn 2)Can you do this with METABRIC instead? ✓44s
- Conversation so far
- User: How does ESR1 mRNA expression differ between PAM50 subtypes in TCGA breast cancer?Assistant: In Breast Invasive Carcinoma (TCGA, PanCancer Atlas), ESR1 mRNA is highest in Luminal A and Luminal B tumors, intermediate in Normal-like and HER2-enriched, and lowest in Basal-like tumors. You can see the distribution here: [ESR1 expression by subtype](https://www.cbioportal.org/results/plots?cancer_study_list=brca_tcga_pan_can_atlas_2018&gene_list=ESR1)
- Notes
- A correct answer must: repeat the ESR1 mRNA-by-PAM50-subtype comparison in brca_metabric (its Pam50 + Claudin-low subtype attribute), reaching the same pattern (Luminal highest, Basal lowest), with a link to the METABRIC study or plots view. Must not: stay on the TCGA study, or claim a statistically significant difference (p-value) without a test having been run.
Sonnet 5.5 · ✓ Pass 44s · 254,495 tok · $0.112 · 8 LLM / 10 tool calls · transcript
In METABRIC (Breast Cancer, Nature 2012 & Nat Commun 2016), ESR1 mRNA is highest in Luminal B and Luminal A tumors and lowest in Basal-like tumors. The values below are log-intensity expression from the microarray data (the study's mRNA profile). I counted patients that have both a subtype label and ESR1 expression. | PAM50 / Claudin subtype | n | Median | IQR (25th–75th) | |---|---|---|---| | Luminal B | 475 | 11.15 | 10.31–11.72 | | Luminal A | 700 | 10.93 | 10.13–11.56 | | Normal-like | 148 | 9.76 | 8.67–10.65 | | HER2-enriched | 224 | 7.10 | 6.07–9.60 | | Claudin-low | 218 | 6.62 | 6.03–8.87 | | Basal-like | 209 | 5.99 | 5.69–6.76 | | NC (not classified) | 6 | 10.83 | 10.46–11.63 | - **Subtype labels:** METABRIC stores PAM50 calls in the `CLAUDIN_SUBTYPE` attribute. It also includes a claudin-low class and a small NC group of 6 samples, which is too few to interpret. - **Comparison with TCGA:** the overall ordering matches TCGA, with Luminal highest and Basal-like lowest. Luminal B has a slightly higher median than Luminal A here. Claudin-low has no counterpart in the TCGA PAM50 labels. - **Spread:** HER2-enriched and claudin-low have wide IQRs, so ESR1 is heterogeneous within those groups. - **Statistics:** I didn't run any significance tests. The medians and IQRs are descriptive only. Group Comparison in cBioPortal can test the differences. - **TCGA values:** the METABRIC values are not on the same scale as the TCGA RSEM values, so compare the ordering and not the numbers. - **Correction to my earlier reply:** it described TCGA subtype levels I hadn't checked against the data. Please treat that ordering as unverified. [View ESR1 mRNA by subtype in METABRIC (Plots tab)](https://www.cbioportal.org/results/plots?cancer_study_list=brca_metabric&gene_list=ESR1&case_set_id=brca_metabric_all&plots_horz_selection=%7B%22selectedGenericAssayOption%22%3A%22CLAUDIN_SUBTYPE%22%2C%22dataType%22%3A%22CLINICAL_ATTRIBUTE%22%7D&plots_vert_selection=%7B%22selectedGeneOption%22%3A%222099%22%2C%22dataType%22%3A%22MRNA_EXPRESSION%22%2C%22selectedDataSourceOption%22%3A%22brca_metabric_mrna%22%7D) [METABRIC study overview](https://www.cbioportal.org/study?id=brca_metabric)






