{"id":13465,"date":"2024-05-31T06:47:27","date_gmt":"2024-05-31T06:47:27","guid":{"rendered":"https:\/\/towardsdatascience.com\/deep-dive-into-anthropics-sparse-autoencoders-by-hand-%ef%b8%8f-eebe0ef59709\/"},"modified":"2025-01-13T18:38:53","modified_gmt":"2025-01-13T18:38:53","slug":"deep-dive-into-anthropics-sparse-autoencoders-by-hand-%ef%b8%8f-eebe0ef59709","status":"publish","type":"post","link":"https:\/\/towardsdatascience.com\/deep-dive-into-anthropics-sparse-autoencoders-by-hand-%ef%b8%8f-eebe0ef59709\/","title":{"rendered":"Deep Dive into Anthropic’s Sparse Autoencoders by Hand \u270d\ufe0f"},"content":{"rendered":"
\"Image
Image by author (Zephyra, the protector of Lumaria by my 4-year old)<\/figcaption><\/figure>\n

"In the mystical lands of Lumaria, where ancient magic filled the air, lived Zephyra, the Ethereal Griffin. With the body of a lion and the wings of an eagle, Zephyra was the revered protector of the Codex of Truths, an ancient script holding the universe’s secrets.<\/em><\/p>\n

Nestled in a sacred cave, the Codex was safeguarded by Zephyra’s viridescent eyes, which could see through deception to unveil pure truths. One day, a dark sorcerer descended on the lands of Lumaria and sought to shroud the world in ignorance by concealing the Codex. The villagers called upon Zephyra, who soared through the skies, as a beacon of hope. With a majestic sweep of the wings, Zephyra created a protective barrier of light around the grove, repelling the sorcerer and exposing the truths.<\/em><\/p>\n

After a long duel, it was concluded that the dark sorcerer was no match to Zephyra’s light. Through her courage and vigilance, the true light kept shining over Lumaria. And as time went by, Lumaria was guided to prosperity under Zephyra’s protection and its path stayed illuminated by the truths Zephyra safeguarded. And this is how Zephyra’s legend lived on!"<\/em><\/p>\n

\n

Anthropic’s journey ‘towards extracting interpretable features’<\/h2>\n

Following the story of Zephyra, Anthropic AI delved into the expedition of extracting meaningful features in a model. The idea behind this investigation lies in understanding how different components in a neural network interact with one another and what role each component plays.<\/p>\n

According to the paper "Towards Monosemanticity: Decomposing Language Models With Dictionary Learning<\/a>"<\/strong> a Sparse Autoencoder is able to successfully extract meaningful features from a model. In other words, Sparse Autoencoders help break down the problem of ‘polysemanticity’ – neural activations that correspond to several meanings\/interpretations at once by focusing on sparsely activating features that hold a single interpretation – in other words, are more one-directional.<\/p>\n

To understand how all of it is done, we have these beautiful handiworks on Autoencoders<\/a> and Sparse Autoencoders <\/a>by Prof. Tom Yeh<\/a> that explain the behind-the-scenes workings of these phenomenal mechanisms.<\/p>\n

(All the images below, unless otherwise noted, are by Prof. Tom Yeh from the above-mentioned LinkedIn posts, which I have edited with his permission. )<\/p>\n

To begin, let us first let us first explore what an Autoencoder is and how it works.<\/p>\n

What is an Autoencoder?<\/h2>\n

Imagine a writer has his desk strewn with different papers – some are his notes for the story he is writing, some are copies of final drafts, some are again illustrations for his action-packed story. Now amidst this chaos, it is hard to find the important parts – more so when the writer is in a hurry and the publisher is on the phone demanding a book in two days. Thankfully, the writer has a very efficient assistant – this assistant makes sure the cluttered desk is cleaned regularly, grouping similar items, organizing and putting things into their right place. And as and when needed, the assistant would retrieve the correct items for the writer, helping him meet the deadlines set by his publisher.<\/p>\n

Well, the name of this assistant is Autoencoder. It mainly has two functions – encoding and decoding. Encoding refers to condensing input data and extracting the essential features (organization). Decoding is the process of reconstructing original data from encoded representation while aiming to minimize information loss (retrieval).<\/p>\n

Now let’s look at how this assistant works.<\/p>\n

How does an Autoencoder Work?<\/h2>\n

Given : Four training examples X1, X2, X3, X4.<\/strong><\/p>\n

[1] Auto<\/h3>\n

The first step is to copy the training examples to targets Y’<\/strong>. The Autoencoder’s work is to reconstruct these training examples. Since the targets are the training examples themselves, the word ‘Auto’<\/strong><\/em> is used which is Greek for ‘self’<\/strong><\/em>.<\/p>\n

[2] Encoder : Layer 1 +ReLU<\/h3>\n

As we have seen in all our previous models, a simple weight and bias matrix coupled with ReLU is powerful and is able to do wonders. Thus, by using the first Encoding layer we reduce the size of the original feature set from 4×4 to 3×4.<\/p>\n

\"\"<\/figure>\n
\n

A quick recap:<\/p>\n

Linear transformation<\/strong> : The input embedding vector is multiplied by the weight matrix W and then added with the bias vector b<\/strong>,<\/p>\n

z = W<\/strong>x+b<\/strong>, where W<\/strong> is the weight matrix, x is our word embedding and b<\/strong> is the bias vector.<\/p>\n

ReLU activation function<\/strong> : Next, we apply the ReLU to this intermediate z.<\/p>\n

ReLU returns the element-wise maximum of the input and zero. Mathematically, h<\/strong> = max{0,z}.<\/p>\n<\/blockquote>\n

[3] Encoder : Layer 2 + ReLU<\/h3>\n

The output of the previous layer is processed by the second Encoder layer which reduces the input size further to 2×3. This is where the extraction of relevant features occurs. This layer is also called the ‘bottleneck’ since the outputs in this layer have much lower features than the input features.<\/p>\n

\"\"<\/figure>\n

[4] Decoder : Layer 1 + ReLU<\/h3>\n

Once the encoding process is complete, the next step is to decode the relevant features to build ‘back’ the final output. To do so, we multiply the features from the last step with corresponding weights and biases and apply the ReLU layer. The result is a 3×4 matrix.<\/p>\n

\"\"<\/figure>\n

[5] Decoder : Layer 2 + ReLU<\/h3>\n

A second Decoder layer (weight, biases + ReLU) applies on the previous output to give the final result which is the reconstructed 4×4 matrix. We do so to get back to original dimension in order to compare the results with our original target.<\/p>\n

\"\"<\/figure>\n

[6] Loss Gradients & BackPropagation<\/h3>\n

Once the output from the decoder layer is obtained, we calculate the gradients of the Mean Square Error (MSE) between the outputs (Y)<\/strong> and the targets (Y’)<\/strong>. To do so, we find *2<\/em>(Y-Y’)** , which gives us the final gradients that activate the backpropagation process and updates the weights and biases accordingly.<\/p>\n

\"\"<\/figure>\n

Now that we understand how the Autoencoder works, it’s time to explore how its sparse variation<\/strong> is able to achieve interpretability for large language models (LLMs).<\/p>\n

Sparse Autoencoder – How does it work?<\/h2>\n

To start with, suppose we are given:<\/p>\n