Obsidian Source: Notes / Neel Nanda - MATS Stream
Summary
Pending synthesis from local Obsidian source.
Original source title: Neel Nanda Mats Stream
Extracted Preview
Every second counts, and I have a pretty good idea.
I want to discover circuits where hinglish occurs. I'll find sentences where they differ only by one word, and try to find which one attends to highlish tokens. Then, just to put the final nail in the coffin, I'll use a variety of sentences.
Dataset : https://huggingface.co/datasets/DanArnin/Hinglish2
The GOD Paper : https://arxiv.org/pdf/2211.00593
Look at a lot of attention heads, and try finding difference.
"She likes to dance on bollywood songs" :
13684, 18861, 316, 18487, 402, 7271, 18034, 15374
"She likes to dance on bollywood gaane" :
13684, 18861, 316, 18487, 402, 7271, 18034, 6666, 1986Where does the Hinglish Circuit lies? How do we design the experiment?
Integration Notes
- Source folder:
/home/yashs/Documents/Docs/Obsidian/Research-Notes - Local source:
/home/yashs/Documents/Docs/Obsidian/Research-Notes/Notes/Neel Nanda - MATS Stream.md - Raw copy:
raw/obsidian/research-notes/Notes/Neel Nanda - MATS Stream.md