2c:null 2d:["$","title",null,{"children":"23andme Ancestry | lorenasandoval88 | Observable"}] 32:T1532,

Genetic Ancestry Classification from 23andMe Data

A 23andMe genotype file contains hundreds of thousands of directly observed variants, but it does not directly report a research-grade genetic ancestry classification. In this analysis, ancestry is inferred by comparing each participant's autosomal genotype pattern with reference individuals from the 1000 Genomes Project.

Classification workflow

23andMe genotype file
↓
Autosomal SNP quality control + genome-build harmonization
↓
Match SNPs shared with 1000 Genomes
↓
LD pruning to reduce highly correlated markers
↓
1000 Genomes reference PCA
↓
Project the 23andMe participant into the same PC space
↓
Compare with reference ancestry clusters
↓
AFR · AMR · EAS · EUR · SAS

AFR
African

AMR
Admixed American

EAS
East Asian

EUR
European

SAS
South Asian

What PCA is doing

Principal component analysis (PCA) summarizes genome-wide differences among individuals into a small number of axes. The 1000 Genomes reference samples define the ancestry structure, and each 23andMe participant is then projected into that same PCA space. The participant can be described according to the reference population or broad ancestry cluster to which their genetic profile is most similar.

Genetic ancestry is not the same as race or ethnicity

Self-reported race and ethnicity are retained as participant characteristics and are analyzed separately from genotype-based ancestry. The ancestry result represents genetic similarity to reference populations; it should not be interpreted as a prediction of a person's race, ethnicity, culture, or identity.

Why ancestry matters for polygenic scores

PGS performance can differ across ancestry groups because allele frequencies, linkage-disequilibrium patterns, and estimated genetic effects may differ between populations. Many PGS models were developed or evaluated in specific ancestry groups. Therefore, Diseasome can compare a participant's inferred genetic ancestry with the development and evaluation ancestry reported for each PGS model and use that information when interpreting score coverage, imputation accuracy, and transferability.

Output retained for each participant:
PCA coordinates · inferred broad ancestry group · distance or classification confidence · self-reported race · self-reported ethnicity

33:T60a,## 2. Decompress VCF Data (IMPORTANT) **Tabix** is a tool (and file format) that enables fast, region-based access to large genomic files without downloading the entire dataset. However, it requires Node-style buffer support and is not reliably supported in browser-based environments like Observable. As an alternative, we fetch a VCF slice via HTTP and perform decompression in the browser. ### ⚠️ Challenge A full VCF file is extremely large: - ~200 MB compressed (`.vcf.gz`) - ~1–2 GB uncompressed text Using: ```js pako.ungzip(vcfData, { to: "string" }) ``` attempts to create one massive string, which exceeds browser memory limits and results in: ``` RangeError: Invalid string length ``` ### Solution: Streaming Decompression Instead of loading the entire file into memory, use streaming decompression and process the data incrementally. This allows us to: - Avoid memory overflow - Extract only relevant lines - Stop early once enough SNPs are collected ### Filtering Strategy During streaming, apply lightweight filters: - **SNP filter:** keep only biallelic variants - `REF` and `ALT` length = 1 (A/T/C/G) - **Allele Frequency (AF) / Minor Allele Frequency (MAF):** - Extract from the `INFO` field - Example: `AF=0.006` → MAF ≈ 0.006 - Filter for common variants (e.g., `MAF > 0.05`) --- ### Outcome This approach yields a manageable subset of SNPs (e.g., 500–1000 variants) that can be used for: - Genotype matrix construction - PCA and clustering - Coarse ancestry inference 34:T6c8,vcfLines = { const inflator = new pako.Inflate({ to: "string" }); let textBuffer = ""; const lines = []; function getAF(info) { const af = info.match(/AF=([^;]+)/)?.[1]; if (!af) return null; return Math.max(...af.split(",").map(Number)); } inflator.onData = chunk => { textBuffer += chunk; const parts = textBuffer.split("\n"); textBuffer = parts.pop(); for (const line of parts) { if (!line || line.startsWith("#")) continue; const row = line.split("\t"); const ref = row[3]; const alt = row[4]; const info = row[7]; const af = getAF(info); const maf = af == null ? null : Math.min(af, 1 - af); if ( ref.length === 1 && alt.length === 1 && maf !== null && maf > 0.05 ) { lines.push(line); } if (lines.length >= 1000) break; } }; inflator.push(vcfData, true); return lines.slice(0, 1000); } // // vcfText = pako.ungzip(vcfData, { to: "string" }) // vcfLines = { // const inflator = new pako.Inflate({ to: "string" }) // let textBuffer = "" // let lines = [] // inflator.onData = chunk => { // textBuffer += chunk // const parts = textBuffer.split("\n") // textBuffer = parts.pop() // for (const line of parts) { // if (!line || line.startsWith("#")) continue // const row = line.split("\t") // const ref = row[3] // const alt = row[4] // if (ref?.length === 1 && alt?.length === 1) { // lines.push(line) // } // if (lines.length >= 1000) { // inflator.push(new Uint8Array(), true) // } // } // } // inflator.push(vcfData, true) // return lines.slice(0, 1000) // }35:[{"id":"5307de5b871692c78299cf6bbdafe0950438e3b397f376ea2659406838ec85facda58be90a0d49b65220a038494267629deb50f53e5a3d7c0d3fc4790eb5a108","href":"https://static.observableusercontent.com/files/5307de5b871692c78299cf6bbdafe0950438e3b397f376ea2659406838ec85facda58be90a0d49b65220a038494267629deb50f53e5a3d7c0d3fc4790eb5a108","name":"image.png","type":"image/png","lastModified":1777520162701.586,"size":10213}] 30:{"id":"2bc27785d574c488","slug":"23andme_ancestry_js","title":"23andme Ancestry","thumbnail":"2279127c5b8c5a13bc63d1f747558083f5e4c02486f00edc057e04fd8c397a04","access_level":"public","create_time":"$D2026-04-30T01:04:31.147Z","update_time":"$D2026-10-02T20:02:07.717Z","view_time":null,"first_public_time":"$D2026-04-30T01:05:18.332Z","has_unpublished_edits":null,"fork":false,"stars":0,"starred":false,"owner":{"type":"team","id":"5a5147324ed15c62","login":"lorenasandoval88","name":"","avatar_url":"https://avatars-next.observableusercontent.com/c8926e2434569cc1f72aeac62929e824fdec3f0ec037d4a41a38a7728d14e950","tier":"starter_2024","bio":""},"version":266,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"5c479a2c5ae5f52f","login":"lorenasandoval","name":"","avatar_url":"https://avatars-next.observableusercontent.com/c8926e2434569cc1f72aeac62929e824fdec3f0ec037d4a41a38a7728d14e950","tier":null,"bio":""}],"members":[{"type":"individual","id":"5c479a2c5ae5f52f","login":"lorenasandoval","name":"","avatar_url":"https://avatars-next.observableusercontent.com/c8926e2434569cc1f72aeac62929e824fdec3f0ec037d4a41a38a7728d14e950","tier":null,"role":"editor"},{"type":"team","id":"5a5147324ed15c62","login":"lorenasandoval88","name":"","avatar_url":"https://avatars-next.observableusercontent.com/c8926e2434569cc1f72aeac62929e824fdec3f0ec037d4a41a38a7728d14e950","tier":"starter_2024","role":"viewer"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":{"id":"1476de78fbd46e52","slug":"23andme-data","title":"23andme-open-data","owner":{"id":"5a5147324ed15c62","login":"lorenasandoval88","name":""}},"type":"notebook","body":{"titling":"auto","title":"23andme Ancestry","cells":[{"id":0,"value":"# 23andme Ancestry","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":224,"value":"","pinned":true,"hidden":false,"mode":"ts","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":221,"value":"$32","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":100,"value":"Extract, sample, and transform genomic SNP data from large VCF files into matrices for browser-based clustering, visualization, and ancestry inference by comparing 23andMe genotypes to reference population SNP panels.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":94,"value":"## 1. Fetch and Load VCF Data in the Browser (chr22)\n\nDownload a compressed VCF file from a remote source, validate the response, and load it into memory as a binary `Uint8Array` for downstream genomic processing (e.g., decompression, SNP extraction, and matrix construction).\n\n### Chromosome 22 Variant Scale (1000 Genomes Phase 3)\n\n- **Compressed file size:** ~200 MB (.vcf.gz) \n- **Uncompressed size:** ~1–2 GB \n- **Total variants (all types):** ~1.1–1.3 million \n- **Biallelic SNPs (after filtering):** ~800,000 – 1,000,000 \n\n> Note: Only a small subset of these SNPs is needed for downstream tasks such as clustering, PCA, and ancestry inference.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":13,"value":"vcfUrl = \"https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz\"\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":63,"value":"vcfData = {\n const response = await fetch(vcfUrl, { cache: \"no-store\" });\n \n if (!response.ok) {\n throw new Error(`Fetch failed: ${response.status} ${response.statusText}`);\n }\n \n const buffer = await response.arrayBuffer();\n return new Uint8Array(buffer);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":202,"value":"$33","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":65,"value":"pako = (await import(\"https://esm.sh/pako\")).default","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":191,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":194,"value":"You now have 1,000 chr22 SNP lines from the VCF in Observable.\n\nNext convert them into SNP rows + genotype matrix:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":206,"value":"snps = vcfLines.map(d => d.split(\"\\t\"))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":219,"value":"A matrix where:\n\nRows = SNPs (variants)\nColumns = individuals (samples)\nValues = genotype encoded as numbers (0, 1, 2)\n![image.png](${await FileAttachment(\"image.png\").url()})","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":208,"value":"genotypeMatrix = snps.map(row =>\n row.slice(9).map(g => {\n const gt = g.split(\":\")[0]\n if (gt === \"0|0\" || gt === \"0/0\") return 0\n if (gt === \"0|1\" || gt === \"1|0\" || gt === \"0/1\" || gt === \"1/0\") return 1\n if (gt === \"1|1\" || gt === \"1/1\") return 2\n return -1\n })\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":210,"value":"snpLabels = snps.map(row => row[2] !== \".\" ? row[2] : `${row[0]}:${row[1]}`)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":215,"value":"And sample/user labels from the VCF header if you kept it; otherwise:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":213,"value":"sampleLabels = Array.from(\n { length: genotypeMatrix[0].length },\n (_, i) => `sample_${i + 1}`\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":217,"value":"Now you have:\n\nRows = SNPs\nColumns = 1000 Genomes individuals\nValues = genotype dosage 0, 1, 2","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"}],"files":"$W35","stdlib":"1"}} 31:"$30:body"