Literature workflows
Can RNA identify disrupted mitochondrial machinery in an unseen target?
Read the task instructions and deliverables
You are studying whether RNA measurements can identify which mitochondrial machinery was disrupted in K562 cells, a human leukemia cell line, six days after CRISPR interference reduced target-gene activity.
The files in documents contain all available cells targeting genes whose symbols begin MRPL, MRPS or ATP5, plus PAM16, DNAJC19, TFAM, HSPA9, TOMM40, TOMM20, PMPCB, LONP1 and HSPE1, and all non-targeting controls. Other targeted genes were left out. No cells were downsampled, and all 8,563 measured genes remain. counts.h5 contains the integer UMI counts in dataset counts: rows follow cells.csv.gz and columns follow genes.csv.gz. A UMI is a count of an observed RNA molecule.
In cells.csv.gz, cell_barcode identifies the cell; gene and gene_id identify its intended target; gene_transcript identifies its construct; transcript identifies the targeted promoter/transcript; sgID_AB records the guide identities; gem_group identifies its sequencing group. UMI_count is the total UMI count, core_adjusted_UMI_count is the sequencing-depth-adjusted total, core_scale_factor is the adjustment divisor, and mitopercent is the fraction of RNA from mitochondrial genes on a 0-to-1 scale. source_row is the zero-based row in the deposited matrix. In genes.csv.gz, gene_id and gene_name identify the measured gene; chr, start, end, strand and length describe its genomic location; class is its annotation-version label. All identifiers and row orders come from the deposited data.
Use cells with core_adjusted_UMI_count at least 3,000 and mitopercent below 0.20. A target construct qualifies if its target appears in genes.csv.gz, it has at least 30 such cells, its target has a mean of at least 0.1 raw UMI per qualifying non-targeting cell, and its mean target count is at most half that control mean. These means refer to raw counts over all qualifying cells in the stated group. A target gene qualifies if at least one of its constructs qualifies; its analysis cells are those from qualifying constructs. Include all qualifying non-targeting cells as controls.
Return eligible_targets as the sorted set of qualifying target gene symbols, excluding non-targeting controls. Define the class labels as mtLSU for MRPL genes, mtSSU for MRPS genes, ATP_synthase for ATP5 genes, proteostasis for the nine named genes, and control for non-targeting constructs. Which classes and how many target genes or control constructs are actually represented after eligibility checks?
Estimate how accurately the 13 mitochondrial protein-coding transcripts predict the class for cells whose target gene was absent from training. They are MT-ND1, MT-ND2, MT-CO1, MT-CO2, MT-ATP8, MT-ATP6, MT-CO3, MT-ND3, MT-ND4L, MT-ND4, MT-ND5, MT-ND6 and MT-CYB. For controls, the prediction target is a construct absent from training. Report this estimate as mitochondrial_accuracy on a 0-to-1 scale, rounded to four decimal places. Its population is all cells passing the stated rules, with equal weight for each represented class and equal weight for each target gene within a class; control constructs are the control units. Within each unit, weight cells equally. Use an adequately fitted predictor and assess whether the estimate is stable under reasonable analysis choices.
In reasoning.md, write 4–8 bullets that answer these questions using observations from these files and the inferences they support: What evidence supports the eligible population and its class sizes? What do the construct and cell records imply about the experimental units supporting the prediction estimate? What do comparisons with controls show about differences between sequencing groups and how they affect the analysis? How does prediction from the mitochondrial transcripts compare with prediction from variable nuclear transcripts with mean control expression above 0.5 raw UMI per cell? What sensitivity checks and limits affect the conclusion about a new target gene?
Deliver exactly answer.json and reasoning.md.
Return EXACTLY, as the contents of answer.json:
{"eligible_targets": ["<identifier>", "..."], "mitochondrial_accuracy": <float, 4 decimal places>}How was it graded?
Deterministic checks compare the submitted answer fields with the reference values and tolerances. An LLM judge assesses the remaining reasoning criteria and cannot override the deterministic results.
One point per passed criterion, divided by the total number of criteria. Unresolved judgments produce a score range. Critical failures are reported separately and do not add weight. No overall pass threshold is specified.
LLM judge: gpt-6-astra
Task files
Choose an evidence file.
Grading criteria
Identify all eligible perturbation targets
Earns credit
Identify the qualifying target genes, excluding non-targeting controls. Submit eligible_targets as a JSON list of strings. After alias normalization and removal of optional members, match the reference set exactly. Reference set (65 members): DNAJC19, HSPA9, LONP1, MRPL1, MRPL10, MRPL13, MRPL14, MRPL15, MRPL16, MRPL17, MRPL18, MRPL19, MRPL2, MRPL20, MRPL22, MRPL24, MRPL27, MRPL28, MRPL3, MRPL32, MRPL33, MRPL35, MRPL36, MRPL37, MRPL38, MRPL39, MRPL4, MRPL41, MRPL43, MRPL44, MRPL45, MRPL49, MRPL50, MRPL51, MRPL54, MRPL55, MRPL9, MRPS10, MRPS11, MRPS12, MRPS14, MRPS16, MRPS18A, MRPS18B, MRPS2, MRPS21, MRPS22, MRPS23, MRPS24, MRPS25, MRPS26, MRPS27, MRPS28, MRPS31, MRPS34, MRPS35, MRPS5, MRPS6, MRPS7, MRPS9, PAM16, PMPCB, TFAM, TOMM20, TOMM40.
Does not earn credit
No credit for a missing or malformed answer, a field that is not a list of strings, or set overlap below the required threshold. Optional members do not count as errors.
Accepted alternatives
Order does not affect the set comparison. Repeated members are counted once by the scorer. Letter case is ignored. Leading and trailing whitespace is ignored. Accepted aliases: ENSG00000205981 → DNAJC19; ENSG00000113013 → HSPA9; ENSG00000196365 → LONP1; ENSG00000169288 → MRPL1; ENSG00000159111 → MRPL10; ENSG00000172172 → MRPL13; ENSG00000180992 → MRPL14; ENSG00000137547 → MRPL15; ENSG00000166902 → MRPL16; ENSG00000158042 → MRPL17; ENSG00000112110 → MRPL18; ENSG00000115364 → MRPL19; ENSG00000112651 → MRPL2; ENSG00000242485 → MRPL20; ENSG00000082515 → MRPL22; ENSG00000143314 → MRPL24; ENSG00000108826 → MRPL27; ENSG00000086504 → MRPL28; ENSG00000114686 → MRPL3; ENSG00000106591 → MRPL32; ENSG00000243147 → MRPL33; ENSG00000132313 → MRPL35; ENSG00000171421 → MRPL36; ENSG00000116221 → MRPL37; ENSG00000204316 → MRPL38; ENSG00000154719 → MRPL39; ENSG00000105364 → MRPL4; ENSG00000182154 → MRPL41; ENSG00000055950 → MRPL43; ENSG00000135900 → MRPL44; ENSG00000278845 → MRPL45; ENSG00000149792 → MRPL49; ENSG00000136897 → MRPL50; ENSG00000111639 → MRPL51; ENSG00000183617 → MRPL54; ENSG00000162910 → MRPL55; ENSG00000143436 → MRPL9; ENSG00000048544 → MRPS10; ENSG00000181991 → MRPS11; ENSG00000128626 → MRPS12; ENSG00000120333 → MRPS14; ENSG00000182180 → MRPS16; ENSG00000096080 → MRPS18A; ENSG00000204568 → MRPS18B; ENSG00000122140 → MRPS2; ENSG00000266472 → MRPS21; ENSG00000175110 → MRPS22; ENSG00000181610 → MRPS23; ENSG00000062582 → MRPS24; ENSG00000131368 → MRPS25; ENSG00000125901 → MRPS26; ENSG00000113048 → MRPS27; ENSG00000147586 → MRPS28; ENSG00000102738 → MRPS31; ENSG00000074071 → MRPS34; ENSG00000061794 → MRPS35; ENSG00000144029 → MRPS5; ENSG00000243927 → MRPS6; ENSG00000125445 → MRPS7; ENSG00000135972 → MRPS9; ENSG00000217930 → PAM16; ENSG00000105819 → PMPCB; ENSG00000108064 → TFAM; ENSG00000173726 → TOMM20; ENSG00000130204 → TOMM40.
Input evidence
Reference evidence
derivation · Eligibility loop and prediction loop; answer_key.json beside task.json; reference/measurements.json.
Recomputes the private reference on the exact packet.
paper · XML paragraphs 112-114 and 143; Figure S8F caption, paragraph 201.
Documents quality rules, control-based scaling and the published comparison. Its day-8 scores are not the answer to this day-6 task.
answer-key · eligible_targets
Accepted values and the script threshold.
gates · Complete reference file.
Day-6 packet computation supporting the observation and inference; this is not a copied day-8 result.
Recorded result / GPT 6 astra
Passed. eligible_targets: matched 65 of 65 reference members. No missing or extra members.
Measure classification accuracy on held-out perturbation targets
Earns credit
Report held-out-target class accuracy using the 13 mitochondrial protein-coding transcripts, on a 0–1 scale, with equal weight per class, target gene within class, and cell within unit. In the submitted JSON, mitochondrial_accuracy must be approximately 0.7263, with an absolute tolerance of ±0.025 (inclusive). The displayed reference is rounded for readability; the original contract retains the exact value.
Does not earn credit
No credit for a missing field, a value that is not a finite JSON number, or a result outside the allowed tolerance.
Accepted alternatives
The scorer adds 1e-12 to the tolerance for floating-point comparison.
Input evidence
Reference evidence
derivation · Eligibility loop and prediction loop; answer_key.json beside task.json; reference/measurements.json.
Recomputes the private reference on the exact packet.
paper · XML paragraphs 112-114 and 143; Figure S8F caption, paragraph 201.
Documents quality rules, control-based scaling and the published comparison. Its day-8 scores are not the answer to this day-6 task.
answer-key · mitochondrial_accuracy
Accepted values and the script threshold.
measurements · Complete reference file.
Day-6 packet computation supporting the observation and inference; this is not a copied day-8 result.
Recorded result / GPT 6 astra
Failed. Submitted mitochondrial_accuracy = 0.7651; reference ≈ 0.7263. Absolute difference ≈ 0.0388; allowed difference ±0.025. Displayed numbers are rounded; the verdict is the original recorded result.
Connect eligibility evidence to the analyzed population
Earns credit
The eligible-target conclusion in C-001 is correct. The submission connects raw target counts to their matching measured-gene identifiers and non-targeting control counts, checks construct cell counts, and explains how these checks determine both the analyzed cells and the represented class sizes. It gives a concrete exclusion and distinguishes qualifying genes from qualifying constructs.
Does not earn credit
C-001 is wrong or missing, or the submission only lists a population, uses the wrong experiment, skips a required eligibility check, or gives no measured observation and its consequence.
Accepted alternatives
Accepted examples: 65 target genes (34 mtLSU, 23 mtSSU, 8 proteostasis), 97 control constructs; no ATP5 targets. MRPL53 is absent from the measured-gene table, so its expression gate cannot be established; MRPL21 has 14 cells, MRPL34 27, HSPE1 29. PMPCB P1 has 503 cells with residual expression about 0.195; P2 has 50 cells with residual about 0.839, so only P1 contributes.
Prerequisites
C-001
Input evidence
Reference evidence
derivation · Eligibility loop and prediction loop; answer_key.json beside task.json; reference/measurements.json.
Recomputes the private reference on the exact packet.
paper · XML paragraphs 112-114 and 143; Figure S8F caption, paragraph 201.
Documents quality rules, control-based scaling and the published comparison. Its day-8 scores are not the answer to this day-6 task.
gates · Complete reference file.
Day-6 packet computation supporting the observation and inference; this is not a copied day-8 result.
measurements · Complete reference file.
Day-6 packet computation supporting the observation and inference; this is not a copied day-8 result.
Recorded result / GPT 6 astra
Bullets 1–2 correctly connect gene-ID matching, raw target/control means and construct cell counts to eligibility. Independent calculations confirm 65 target genes, 97 control constructs and 19,455 analysis cells. The exclusions and separate treatment of PMPCB P1 and P2 are correct.
Use the records to support transfer to an unseen target
Earns credit
C-001, C-002 and the population conclusion in C-003 are correct and supported by the submitted analysis. All prediction cells meet the stated eligibility rules. The submission connects repeated cells and construct identities to the target-gene units required by this prediction question, shows that evaluated units were absent from training, and explains the equal-class and equal-unit weighting using actual counts or diagnostic checks.
Does not earn credit
Any prerequisite is wrong or missing, or wrong cells are used even if the reported numbers match; evaluated target genes occur in training; controls are treated as one held-out gene; or the submission names a split method without showing how the observed records support it.
Accepted alternatives
Accepted evidence includes zero overlap of target genes between train and test in every evaluation split; 97 separate control units; separate handling of PMPCB constructs; and class recalls near 0.927, 0.797, 0.707 and 0.474 whose equal average is about 0.726. Equivalent leakage-free evaluations with correct weighting are accepted.
Prerequisites
C-001, C-002, C-003
Input evidence
Reference evidence
derivation · Eligibility loop and prediction loop; answer_key.json beside task.json; reference/measurements.json.
Recomputes the private reference on the exact packet.
paper · XML paragraphs 112-114 and 143; Figure S8F caption, paragraph 201.
Documents quality rules, control-based scaling and the published comparison. Its day-8 scores are not the answer to this day-6 task.
measurements · Complete reference file.
Day-6 packet computation supporting the observation and inference; this is not a copied day-8 result.
Recorded result / GPT 6 astra
The required prerequisite C-002 fails. Although the reasoning describes withholding whole target genes or control constructs and equal-class/equal-unit weighting, the criterion explicitly requires C-002 to pass.
Connect control differences to expression comparability
Earns credit
The submission measures a difference between sequencing groups using the qualifying control cells, explains why this difference can affect expression comparisons, and reports how it accounted for that difference or checked that it did not explain the predictive signal. Both the observation and the diagnostic result must use the packet controls.
Does not earn credit
The submission gives generic normalization advice, compares unmatched target populations as if they were controls, or states a method without a measured control comparison and its implication.
Accepted alternatives
Accepted example: control mean UMI_count varies from about 8925 in gem_group 47 to 18456 in group 12; 48 groups contain 116–281 controls each. Library scaling followed by within-group control centering and scaling makes their baseline expression comparable. Other measured, justified adjustments or explicit sensitivity checks can pass.
Input evidence
Reference evidence
derivation · Eligibility loop and prediction loop; answer_key.json beside task.json; reference/measurements.json.
Recomputes the private reference on the exact packet.
paper · XML paragraphs 112-114 and 143; Figure S8F caption, paragraph 201.
Documents quality rules, control-based scaling and the published comparison. Its day-8 scores are not the answer to this day-6 task.
diagnostics · Complete reference file.
Day-6 packet computation supporting the observation and inference; this is not a copied day-8 result.
sensitivity · Complete reference file.
Day-6 packet computation supporting the observation and inference; this is not a copied day-8 result.
Recorded result / GPT 6 astra
Bullet 4 measures sequencing-group differences in qualifying controls and explains their effect on expression comparisons. Independent calculations confirm the UMI ranges, depth-adjustment CV reduction and mitochondrial variance fractions. It describes training-control centering and reports corrected versus uncorrected accuracy, supporting this standalone criterion.
Compare mitochondrial and nuclear specificity on the same population
Earns credit
C-001, C-002 and the population conclusion in C-003 are correct and supported by the submitted analysis. All prediction cells meet the stated eligibility rules. Using the same eligible target units, class weighting and unseen-target evaluation for both feature families, the submission gives measured prediction results and connects class-level errors to whether the signal distinguishes different disrupted machinery rather than just separating controls from perturbed cells. Nuclear feature selection must not use held-out class outcomes.
Does not earn credit
Any prerequisite is wrong or missing, or wrong cells are used even if the reported numbers match; only a recalled headline or feature count is given; the two results use different populations or incompatible weighting; or a control-versus-perturbed comparison is presented as complex prediction.
Accepted alternatives
Accepted example: about 0.72 mitochondrial accuracy versus 0.36 nuclear accuracy; 4060 nuclear genes exceed the control expression threshold and about 1015 high-variance genes are used per training fold. Nuclear control recall is about 0.975 but mtSSU recall is near zero; mitochondrial mtSSU recall is about 0.707. Other supported nuclear feature rules and quantitative results can pass if they establish the same comparison.
Prerequisites
C-001, C-002, C-003
Input evidence
Reference evidence
derivation · Eligibility loop and prediction loop; answer_key.json beside task.json; reference/measurements.json.
Recomputes the private reference on the exact packet.
paper · XML paragraphs 112-114 and 143; Figure S8F caption, paragraph 201.
Documents quality rules, control-based scaling and the published comparison. Its day-8 scores are not the answer to this day-6 task.
measurements · Complete reference file.
Day-6 packet computation supporting the observation and inference; this is not a copied day-8 result.
Recorded result / GPT 6 astra
The required prerequisite C-002 fails. The reported mitochondrial-versus-nuclear comparisons cannot earn this point because the criterion explicitly requires a passing mitochondrial_accuracy field.
Tie uncertainty to measured transfer performance
Earns credit
C-001, C-002 and the population conclusion in C-003 are correct and supported by the submitted analysis. All prediction cells meet the stated eligibility rules. The submission reports at least one quantitative sensitivity check of its unseen-target estimate and connects observed variation or class-level support to a bounded conclusion about prediction on a new target. It states what population the conclusion covers.
Does not earn credit
Any prerequisite is wrong or missing, or wrong cells are used even if the reported numbers match, or the submission gives only generic caveats, treats many cells as many independent target genes, or supplies no measured sensitivity result.
Accepted alternatives
Accepted examples: target-held-out fold or seed changes give roughly 0.719–0.727; changing leaf size gives about 0.742; proteostasis has eight eligible target genes and lower recall than the ribosomal classes, so new-target estimates for it are less secure. Restrict the claim to this day-6 K562 panel. Alternative measured resampling, predictor or preprocessing sensitivity is accepted. The wider tested range across tree settings is about 0.7034–0.7421.
Prerequisites
C-001, C-002, C-003
Input evidence
Reference evidence
derivation · Eligibility loop and prediction loop; answer_key.json beside task.json; reference/measurements.json.
Recomputes the private reference on the exact packet.
paper · XML paragraphs 112-114 and 143; Figure S8F caption, paragraph 201.
Documents quality rules, control-based scaling and the published comparison. Its day-8 scores are not the answer to this day-6 task.
measurements · Complete reference file.
Day-6 packet computation supporting the observation and inference; this is not a copied day-8 result.
sensitivity · Complete reference file.
Day-6 packet computation supporting the observation and inference; this is not a copied day-8 result.
additional · Complete reference file.
Day-6 packet computation supporting the observation and inference; this is not a copied day-8 result.
default-checks · Complete reference file.
Day-6 packet computation supporting the observation and inference; this is not a copied day-8 result.
Recorded result / GPT 6 astra
The required prerequisite C-002 fails. Bullet 7 provides quantitative sensitivity checks and limits the claim to eligible day-six K562 perturbations, but the explicit prerequisite prevents a pass.