Website Source: blog / deeprobe_blog
Summary
Pending synthesis from local website source.
Original source title: Using MCTS to Navigate Sparse Autoencoder Feature Space
Extracted Preview
Note: I had SAELens open in one tab and an MCTS tutorial in another. At some point I started wondering if you could use tree search to navigate a feature space instead of a game board. This is what came out when I tried.
Using MCTS to Navigate Sparse Autoencoder Feature Space
- Project Home: [Github Repo](https://github.com/yash-srivastava19/deeprobe)
---
The Idea That Eventually Made Sense
Deeprobe did not start as "MCTS on SAE features." It started as a vague conviction that search algorithms and representation learning should be combinable in some interesting way, and then spent a while being wrong about exactly how.
The thinking evolved roughly like this:
Step 1: LLMs + MCTS. The initial intuition was broad - LLMs have a huge output space, MCTS is good at navigating huge spaces, so maybe you can use MCTS to steer generation. Too broad. This is basically just beam search with extra steps, and there is already a lot of work in that direction.
Step 2: GANs + MCTS. GANs have a latent space. MCTS explores spaces. Could you use MCTS to navigate the GAN latent space toward some target? The idea is not crazy but it is vague in a way that makes it hard to pin down what "reward" even means. Abandoned.
Integration Notes
- Source section:
blog - Local source:
/home/yashs/Desktop/Programming/yash_blog/yash-srivastava19.github.io/blog/deeprobe_blog.md - Raw copy:
raw/website/yash-srivastava19-github-io/blog/deeprobe_blog.md