Q (haiku): How many patients have an EGFR amplification in the TCGA Lung Adenocarcinoma study?

▶ read_guide
{
 "uri": "cbioportal://gene-expression-guide"
}

▶ resolve_and_route
{
 "studyKeywords": [
  "TCGA",
  "lung",
  "adenocarcinoma"
 ]
}

◀ result
# Gene Expression Analysis Guide

This guide covers continuous-value genomic data: gene **expression**, **copy number** values, **methylation**, and related profile types. Mutation/CNA/SV *frequency* analysis lives in `cbioportal://mutation-frequency-guide`.

## Where this data lives

Continuous per-sample-per-gene values are stored in `genetic_alteration_derived`:

| Column | Description |
|---|---|
| `sample_unique_id` | `<study>_<sample_stable_id>` |
| `cancer_study_identifier` | study scope |
| `hugo_gene_symbol` | gene |
| `profile_type` | which assay/normalization (see below) |
| `alteration_value` | the actual value — stored as Nullable(String); cast with `toFloat64OrNull` |

`alteration_value` is a string because the same column hosts many different value scales. The `''` and `'NA'` sentinels mean "missing"; always filter them out and use `toFloat64OrNull(alteration_value) IS NOT NULL` for downstream math.

## Discovering profile types for a study

Different studies expose different profile types depending on what assays were run and how the data was normalized. Always check what a specific study supports before picking one:

```sql
SELECT DISTINCT profile_type
FROM genetic_alteration_derived
WHERE cancer_study_identifier = 'brca_metabric'
ORDER BY profile_type;
```

Common values across the public portal:

| Family | Profile types |
|---|---|
| mRNA expression | `rna_seq_v2_mrna`, `rna_seq_v2_mrna_median_Zscores`, `rna_seq_v2_mrna_median_all_sample_Zscores` (TCGA PanCancer Atlas), `mrna`, `mrna_median_Zscores`, `mrna_seq_v2_rsem`, `mrna_seq_v2_rsem_Zscores`, `mrna_seq_cpm`, `mrna_seq_fpkm`, `mrna_U133`, `mrna_outliers` |
| Copy number (continuous) | `cna`, `linear_CNA`, `log2CNA`, `cna_consensus`, `cna_rae`, `gistic` |
| Methylation | `methylation_hm27`, `methylation_hm450`, `methylation_epic`, `methylation_promoters_rrbs` |
| miRNA | `mirna`, `mirna_median_Zscores` |
| Protein | `protein_quantification`, `protein_level`, `RPPA` |

**Z-score vs raw choice.** When the user asks "is X correlated with Y", either works for Spearman (rank-based) — Pearson would care. Default to the non-Z-score variant if both exist, and call out which one in the response.

## Canonical recipe — Spearman correlation between two genes

```sql
SELECT *
FROM gene_pair_coexpression(
    study        = 'brca_metabric',
    gene_a       = 'TP53',
    gene_b       = 'MYC',
    profile_type = 'mrna'
);
```

Returns one row: `(gene_a, gene_b, profile_type, spearman_correlation, num_samples)`.

- `spearman_correlation` in [−1, 1]; `NULL` when fewer than 3 valid paired samples.
- Mirrors cbioportal-backend's `ClickhouseCoExpressionMapper.getCoExpressions`, simplified to a pair lookup (the backend computes one ref gene vs ALL other genes for the coexpression page; here the agent asks about a specific pair).

### Verified examples

| Study | gene_a | gene_b | profile_type | spearman | n |
|---|---|---|---|---|---|
| `brca_metabric` | TP53 | MYC | `mrna` | 0.118 | 1980 |
|  … (4931 more chars)

◀ result
{"success":true,"message":"Found 4 matching studies. Pick the best match (prefer TCGA → prefer PanCancer Atlas), use its metadata to call the appropriate navigation tool(s) and generate URLs immediately. Present other studies as alternatives.","data":{"totalCount":4,"studiesWithMetadata":[{"studyId":"luad_tcga","name":"Lung Adenocarcinoma (TCGA, Firehose Legacy)","sampleCount":586,"studyViewUrl":"https://www.cbioportal.org/study?id=luad_tcga","metadata":{"clinicalAttributeIds":["AGE","AJCC_CLINICAL_TUMOR_STAGE","AJCC_METASTASIS_CLINICAL_CM","AJCC_METASTASIS_PATHOLOGIC_PM","AJCC_NODES_CLINICAL_CN","AJCC_NODES_CLINICAL_CT","AJCC_NODES_PATHOLOGIC_PN","AJCC_PATHOLOGIC_TUMOR_STAGE","AJCC_STAGING_EDITION","AJCC_TUMOR_PATHOLOGIC_PT","ALK_ANALYSIS_TYPE","ALK_TRANSLOCATION_STATUS","ALK_TRANSLOCATION_VARIANT","CANCER_TYPE","CANCER_TYPE_DETAILED","CARBON_MONOXIDE_DIFFUSION_DLCO","DAYS_TO_COLLECTION","DAYS_TO_INITIAL_PATHOLOGIC_DIAGNOSIS","DAYS_TO_PATIENT_PROGRESSION_FREE","DAYS_TO_SPECIMEN_COLLECTION","DAYS_TO_TUMOR_PROGRESSION","DFS_MONTHS","DFS_STATUS","DISEASE_CODE","ECOG_SCORE","ETHNICITY","EXTRANODAL_INVOLVEMENT","FEV1_FVC_RATIO_POSTBRONCHOLIATOR","FEV1_FVC_RATIO_PREBRONCHOLIATOR","FEV1_PERCENT_REF_POSTBRONCHOLIATOR","FEV1_PERCENT_REF_PREBRONCHOLIATOR","FORM_COMPLETION_DATE","FRACTION_GENOME_ALTERED","HISTOLOGICAL_DIAGNOSIS","HISTORY_IMMUNOLOGICAL_DISEASE","HISTORY_IMMUNOLOGICAL_DISEASE_OTHER","HISTORY_NEOADJUVANT_TRTYN","HISTORY_OTHER_MALIGNANCY","HISTORY_RELEVANT_INFECTIOUS_DX","HIV_STATUS","ICD_10","ICD_O_3_HISTOLOGY","ICD_O_3_SITE","INFORMED_CONSENT_VERIFIED","INITIAL_PATHOLOGIC_DX_YEAR","IS_FFPE","KARNOFSKY_PERFORMANCE_SCORE","KRAS_GENE_ANALYSIS_INDICATOR","KRAS_MUTATION","KRAS_MUTATION_IDENTIFIED_TYPE","LATERALITY","LOCATION_LUNG_PARENCHYMA","LONGEST_DIMENSION","METHOD_OF_INITIAL_SAMPLE_PROCUREMENT","METHOD_OF_INITIAL_SAMPLE_PROCUREMENT_OTHER","METHOD_OF_SAMPLE_PROCUREMENT","MUTATION_COUNT","MUTATION_STATUS","MUTATION_TYPE","NEW_TUMOR_EVENT_AFTER_INITIAL_TREATMENT","NUMBER_OF_LYMPHNODES_POSITIVE_BY_IHC","OCT_EMBEDDED","ONCOTREE_CODE","OS_MONTHS","OS_STATUS","OTHER_METHOD_OF_SAMPLE_PROCUREMENT","OTHER_PATIENT_ID","OTHER_SAMPLE_ID","PATHOLOGY_REPORT_FILE_NAME","PATHOLOGY_REPORT_UUID","PERFORMANCE_STATUS_TIMING","PHARMACEUTICAL_TX_ADJUVANT","POS_LYMPH_NODE_LOCATION","PRIMARY_SITE_OTHER","PRIMARY_SITE_PATIENT","PROJECT_CODE","PROSPECTIVE_COLLECTION","PULMONARY_FUNCTION_TEST_INDICATOR","RACE","RADIATION_TREATMENT_ADJUVANT","RESIDUAL_TUMOR","RETROSPECTIVE_COLLECTION","SAMPLE_COUNT","SAMPLE_INITIAL_WEIGHT","SAMPLE_TYPE","SAMPLE_TYPE_ID","SEX","SHORTEST_DIMENSION","SMOKING_PACK_YEARS","SMOKING_YEAR_STARTED","SMOKING_YEAR_STOPPED","SOMATIC_STATUS","SPECIMEN_CURRENT_WEIGHT","SPECIMEN_FREEZING_METHOD","SPECIMEN_SECOND_LONGEST_DIMENSION","STAGE_OTHER","TARGETED_MOLECULAR_THERAPY","TIME_BETWEEN_CLAMPING_AND_FREEZING","TIME_BETWEEN_EXCISION_AND_FREEZING","TISSUE_SOURCE_SITE","TMB_NONSYNONYMOUS","TOBACCO_SMOKING_HISTORY_INDICATOR","TREATMENT_OUTCOME_FIRST_COURS … (6466 more chars)

▶ read_guide
{
 "uri": "cbioportal://mutation-frequency-guide"
}

▶ clickhouse_list_table_columns
{
 "table": "genetic_alteration_derived"
}

◀ result
# Mutation Frequency Analysis Guide

## IMPORTANT: Reporting Mutation Frequencies
- **ALWAYS report frequencies as percentages**, not raw counts: `frequency = (altered_samples / total_profiled_samples) × 100`
- For quick frequency lookups, **prefer the TCGA Pan-Cancer Atlas study first**, then offer to expand to other studies
- When reporting across multiple studies, show **ranges** (e.g., "TP53 is mutated in 30–60% of samples") rather than a single average
- **NEVER** sum mutation events across studies to compute an aggregate frequency — this can exceed 100% due to double-counting
- Warn users that samples may overlap across cohorts (e.g., MSK studies may share patients)
- **Choose and state the counting unit**: use patient-level frequencies for prevalence/rate questions unless the user explicitly asks for samples; use sample-level frequencies when the user asks about samples.
- **For "across cancer types" questions**, jump to the [Cross-Cancer-Type Mutation Frequency](#cross-cancer-type-mutation-frequency) section below — there is one correct recipe and several common wrong ones.

## Counting Unit: Samples vs Patients

Before answering any mutation count or frequency question, decide whether the unit is samples or patients and state that choice in the answer.

| User wording | Counting unit |
|--------------|---------------|
| "prevalence", "rate", "fraction of patients", "patients with", "how common is" | Patient-level: `COUNT(DISTINCT patient_unique_id)` |
| "samples", "specimens", "biopsies", sample-level cohort composition | Sample-level: `COUNT(DISTINCT sample_unique_id)` |
| Ambiguous | Ask, or default to patient-level for prevalence/rate language and say so |

### Cross-study sample-count caveat

When an answer touches more than one study and reports a sample count, prepend a one-line caveat:

> Sample IDs are unique within cBioPortal study prefixes, not guaranteed biological-sample identifiers across studies; overlapping cohorts can count the same patient/sample more than once.

Prefer one of these safer approaches:

- Use a shipped `cancer_study_query_preferences` cohort such as `pan_cancer_tcga` or `all_studies_non_redundant`.
- Restrict to one named study.
- Aggregate by `patient_unique_id` when the biological question is patient prevalence.

## STOP rule: a frequency above 100% means your query is wrong

If your query returns a frequency over 100%, **do not try to debug or explain the data inconsistency to the user**. The cause is always one of these query bugs:

- Summing mutation events instead of `COUNT(DISTINCT sample_unique_id)` for the numerator
- Using a study-wide sample count as the denominator instead of the gene-specific profiled count
- Cross-study aggregation where the same biological sample appears under multiple `sample_unique_id` values (e.g., MSK-IMPACT and MSK-CHORD share patients)
- **Joining the profiled CTE through `gene_panel` / `gene_panel_list` without a WES branch.** `gene_panel_id = 'WES'` is *not* a row in … (36467 more chars)

◀ result
{"columns":[{"name":"sample_unique_id","type":"String"},{"name":"cancer_study_identifier","type":"LowCardinality(String)"},{"name":"hugo_gene_symbol","type":"String"},{"name":"profile_type","type":"LowCardinality(String)"},{"name":"alteration_value","type":"Nullable(String)"}]}

▶ clickhouse_run_select_query
{
 "query": "
SELECT *
FROM gene_alteration_frequency_by_cancer_type(
    preference = 'pan_cancer_tcga',
    gene       = 'EGFR',
    alteration = 'amplification'
)
WHERE cancer_type LIKE '%adenocarcinoma%' AND cancer_type LIKE '%Lung%'
ORDER BY frequency_pct DESC;
",
 "max_rows": 20
}

◀ result
{"rows":[]}

▶ clickhouse_run_select_query
{
 "query": "
SELECT *
FROM gene_alteration_frequency_in_study(
    study = 'luad_tcga_pan_can_atlas_2018',
    gene  = 'EGFR',
    alteration = 'amplification'
);
",
 "max_rows": 20
}

◀ result
{"error_message":"Query execution failed: HTTPDriver for https://dl96orhu96.us-east-1.aws.clickhouse.cloud:8443 received ClickHouse error code 46\n Code: 46. DB::Exception: Unknown table function gene_alteration_frequency_in_study. (UNKNOWN_FUNCTION) (version 26.4.1.2359 (official build))\n"}

▶ clickhouse_run_select_query
{
 "query": "
SELECT *
FROM top_cna_genes_in_study(
    study = 'luad_tcga_pan_can_atlas_2018',
    top_n = 30
)
WHERE hugo_gene_symbol = 'EGFR' AND cna_type = 'AMP';
",
 "max_rows": 20
}

◀ result
{"rows":[]}

▶ clickhouse_run_select_query
{
 "query": "
SELECT DISTINCT profile_type
FROM genetic_alteration_derived
WHERE cancer_study_identifier = 'luad_tcga_pan_can_atlas_2018'
ORDER BY profile_type;
",
 "max_rows": 20
}

◀ result
{"rows":[{"profile_type":"gistic"},{"profile_type":"log2CNA"},{"profile_type":"rna_seq_v2_mrna"},{"profile_type":"rna_seq_v2_mrna_median_Zscores"},{"profile_type":"rna_seq_v2_mrna_median_all_sample_Zscores"},{"profile_type":"rna_seq_v2_mrna_median_all_sample_ref_normal_Zscores"},{"profile_type":"rppa"},{"profile_type":"rppa_Zscores"}]}

▶ clickhouse_run_select_query
{
 "query": "
WITH egfr_amp AS (
    SELECT DISTINCT sample_unique_id
    FROM genetic_alteration_derived
    WHERE cancer_study_identifier = 'luad_tcga_pan_can_atlas_2018'
      AND hugo_gene_symbol = 'EGFR'
      AND profile_type = 'gistic'
      AND alteration_value = '2'
),
patient_mapping AS (
    SELECT DISTINCT 
           sample_unique_id,
           patient_unique_id
    FROM clinical_data_derived
    WHERE cancer_study_identifier = 'luad_tcga_pan_can_atlas_2018'
)
SELECT COUNT(DISTINCT pm.patient_unique_id) AS patients_with_egfr_amplification,
       COUNT(DISTINCT ea.sample_unique_id) AS samples_with_egfr_amplification
FROM egfr_amp ea
LEFT JOIN patient_mapping pm USING (sample_unique_id);
",
 "max_rows": 10
}

◀ result
{"rows":[{"patients_with_egfr_amplification":26,"samples_with_egfr_amplification":26}]}

▶ clickhouse_run_select_query
{
 "query": "
WITH egfr_profiled_samples AS (
    SELECT DISTINCT sample_unique_id
    FROM genetic_alteration_derived
    WHERE cancer_study_identifier = 'luad_tcga_pan_can_atlas_2018'
      AND hugo_gene_symbol = 'EGFR'
      AND profile_type = 'gistic'
      AND alteration_value NOT IN ('', 'NA')
),
patient_mapping AS (
    SELECT DISTINCT 
           sample_unique_id,
           patient_unique_id
    FROM clinical_data_derived
    WHERE cancer_study_identifier = 'luad_tcga_pan_can_atlas_2018'
)
SELECT COUNT(DISTINCT pm.patient_unique_id) AS patients_profiled_for_egfr,
       COUNT(DISTINCT es.sample_unique_id) AS samples_profiled_for_egfr
FROM egfr_profiled_samples es
LEFT JOIN patient_mapping pm USING (sample_unique_id);
",
 "max_rows": 10
}

◀ result
{"rows":[{"patients_profiled_for_egfr":511,"samples_profiled_for_egfr":511}]}

▶ navigate_to_results_view
{
 "studyIds": [
  "luad_tcga_pan_can_atlas_2018"
 ],
 "genes": [
  "EGFR"
 ],
 "tab": "oncoprint"
}

◀ result
{"success":true,"message":"Navigating to https://www.cbioportal.org/results/oncoprint?cancer_study_list=luad_tcga_pan_can_atlas_2018&gene_list=EGFR&case_set_id=luad_tcga_pan_can_atlas_2018_all","url":"https://www.cbioportal.org/results/oncoprint?cancer_study_list=luad_tcga_pan_can_atlas_2018&gene_list=EGFR&case_set_id=luad_tcga_pan_can_atlas_2018_all","data":{"studyIds":["luad_tcga_pan_can_atlas_2018"],"studies":[{"studyId":"luad_tcga_pan_can_atlas_2018","name":"Lung Adenocarcinoma (TCGA, PanCancer Atlas)","sampleCount":566}],"genes":["EGFR"],"caseSetId":"luad_tcga_pan_can_atlas_2018_all","pageDescription":"Visual summary of alterations in queried genes across samples."}}

═ answer (success)
## Summary

**26 patients** have EGFR amplification in the TCGA Lung Adenocarcinoma (PanCancer Atlas) study, out of **511 patients profiled for EGFR** — a frequency of **5.1%**.

This analysis uses GISTIC copy-number data from the study's `gistic` profile, which captures discrete copy-number alterations (amplifications are coded as value `2`). The 26 patients represent distinct individuals (there are 26 samples with EGFR amplification, and each maps to one patient in this cohort).

**[View EGFR alterations in the OncoPrint](https://www.cbioportal.org/results/oncoprint?cancer_study_list=luad_tcga_pan_can_atlas_2018&gene_list=EGFR&case_set_id=luad_tcga_pan_can_atlas_2018_all)** — to see the full landscape of EGFR alterations including mutations, copy-number amplifications, and other changes across the cohort.