Obsidian Source: Notes / Why are embeddings confusing
Summary
Pending synthesis from local Obsidian source.
Original source title: Why Are Embeddings Confusing
Extracted Preview
And what are some myths associated with them? We'll use LLMs for that.
Embeddings represent complex relationships about entities, concepts ad relationships in some condensed and useful format.
Meaningful interpretation requires visualization using dimensionality reduction or other ML interpretability methods. How do we make them actually more interpretable and more useful.
1. Enhancing concept activation vectors.
2. Communicating novel embedded entitites.
3. Decoding user preferences in recommender systems.
What they do is train adapter layers to map dimains embedding vectors in to the level embedding space of am LLM. Basically, to treat these vectors as token-level encoding of the entities they represent.
Embedding Level Model - which accepts domain embedding vectors as a part of textual input - which can be queried by natural language.
We now define the Embedding Language Model (ELM), a framework which allows one to train a model using textual prompts mixed with continuous domain embedding vectors. ELM incorporates domain embedding vectors using adapter layers to interface with a text-only LLM. We note that we are not learning ED itself, but rather wish to interpret it. In other words, given an embedding vector ED(v), we wish to train an LLM to meaningfully engage in discourse about the entity represented by ED(v) – even for a hypothetical entity v.Integration Notes
- Source folder:
/home/yashs/Documents/Docs/Obsidian/Research-Notes - Local source:
/home/yashs/Documents/Docs/Obsidian/Research-Notes/Notes/Why are embeddings confusing.md - Raw copy:
raw/obsidian/research-notes/Notes/Why are embeddings confusing.md