Hey everyone!

Passion for knowledge and creation are pillars of return moe;. We embrace an AI-first model, with artificial intelligence at the center of our projects. But it’s not just about using the tools: we want to understand how they work under the hood, contribute to their development, and share this with the community.

That’s why we’re focusing on mechanistic interpretability, the research field that seeks to unveil the internal workings of neural networks. It addresses questions like: which circuits are activated? How does information flow through the model’s layers? How are certain decisions made?

In short, it’s the reverse engineering of neural networks.

A new initiative

We’re launching Miru (from Japanese 見る, “to see/observe”), an initiative dedicated to developing mechanistic interpretability tools.

Our first project under this banner is Miru Tracer, which is already available on our GitHub.

It’s a Gradio application that loads language models from the Hugging Face platform, performs inference with Transformers, and produces confidence analysis through probability and entropy. It also features an interactive mode, through which you can manually intervene in the selection process of each token.

Example of Miru Tracer's interactive mode
Example of Miru Tracer’s interactive mode

What’s next?

Miru Tracer is simple, but represents our first step in a still emerging and complex field.

Since the tool enables accessible, visual, and interactive analyses, we believe it can spark interest in mechanistic interpretability among those who use AI but haven’t yet stopped to think about how models work. That’s why educational use is a priority for us.

Beyond that, we want to explore activation steering techniques, which consist of influencing results through manipulation of the neural network’s internal activations. They open a new path to introduce and modify model behaviors, going beyond prompting and finetuning.

Questions, suggestions, or just want to chat about this? Hit me up on Telegram (@rlaneth)! Let’s keep moving forward, one token at a time. 💜