Concept Engines

Independent machine learning research

The geometry
of learned features.

A closer look at how neural networks organize what they learn.

My first paper studies sparse autoencoders: tools for examining features within a language model. I’m interested in how the size and geometry of those features relate to what they represent.

Research preprintAlexander Knox · Independent researcher

Feature Size, Decoder Geometry, and Semantic Specificity in Sparse Autoencoders

The study examines feature size, defined through encoder preactivation variability, alongside decoder neighborhoods, semantic specificity, and measured explanation quality.

Across the main 27-cell panel, feature size is positively associated with decoder density and tightness. Semantic relationships vary with training recipe and depth. Exploratory interventions investigate whether local geometry can inform activation control.

These are scoped findings about learned representations. They do not establish a universal theory of thought or demonstrate direct programming of a language model’s weights.

Read the preprint ↗Download PDF · 18 pages · 485 KB ↓

Author-supplied preprint · Review draft dated November 6, 2026

Sparse autoencoders / Representation geometry / Mechanistic interpretability

Why this interests me

From a compelling idea
to a measurable question.

“Geometry of thought” is an intuition that draws me to this work. The research itself needs more precise language: features, activation statistics, neighborhoods, and evidence that can be tested.

That distinction matters to me. I want to explore ambitious questions while keeping the claims connected to what the experiments actually show.

Explore my working models ↗