Linear probe llm Instead of Workshop Socially Responsible Language Modelling Research (SoLaR) Usman Anwar · David Krueger · Yejin Choi · Maarten Sap · Alan Chan · Yawen Duan · Robert Kirk · Xin Chen, Cynthia · Abulhair Saparov · Kayo Yin · Liwei Jiang · Valentina Pyatkin Dec 1, 2024 · The linear probe functions as a diagnostic tool that identifies specific neural patterns associated with sycophantic behavior in LLMs. The quality of a product is driven by both the talent of its creators and how they feel while they’re crafting it. We built probes using simple training data (from RepE paper) and techniques (logistic regression). Our approach, dubbed LUMIA, applies LPs layer-by-layer to get fine-grained data on the model inner workings. Streamline work across the entire development cycle, from roadmap to release. How we think and work Linear's mission is to inspire and accelerate builders. Dec 1, 2024 · The probe’s input is the RM activations when evaluating the LLM’s response. This signal reliably predicts whether the model will generate a correct response for several knowledge datasets. Sep 1, 2025 · In this vein, we analyze how Linear Probes (LPs) can be used to provide an estimation on the performance of a compressed LLM at an early phase — before fine-tuning. Compared to inference-based or logits-based judgments, we show that linear probing improves both accuracy and efficiency. Nov 29, 2024 · To address this, we propose the use of Linear Probes (LPs) as a method to detect Membership Inference Attacks (MIAs) by examining internal activations of LLMs. . In this work, we employ linear probing to extract evaluation judgments from an LLM-as-a-Judge setup. Fast, focused, and reliable. Apr 23, 2024 · Related work Linear probes were originally introduced in the context of image models but have since been widely applied to language models, including in explicitly safety-relevant applications such as measurement tampering. Dec 1, 2024 · We develop a linear probing method to identify and penalize markers of sycophancy within the reward model, producing rewards that discourage sycophantic behavior. Agents are full members of your Linear workspace. Download the Linear app for desktop and mobile. During inference, we remove the sigmoid activation function to produce a symmetrical and continuous sycophancy score where positive values correspond to a sycophantic answer and negative values correspond to non-sycophantic answers. Linear is the system for modern product development. Linear will launch directly in your browser window. The Linear web app can be access by logging in to linear. What started as a simple issue tracker, has since evolved into a powerful project and issue tracking system that streamlines workflows across the entire product development process. This problematic behavior becomes more pronounced during reinforcement learning from human feedback (RLHF), an LLM fine-tuning stage intended to align model outputs with human values. Nearly all functionality in the desktop app including offline mode is available on the web in most browsers. Dec 1, 2024 · We develop a linear probing method to identify and penalize markers of sycophancy within the reward model, producing rewards that discourage sycophantic behavior. This is a work-in-progress repository for finding adversarial strings of tokens to influence Large Language Models (LLMs) in a variety of ways, as part of investigating generalization and robustness of LLM activation probes. Use Linear for free with your whole team. We named it Linear to signify progress. Just like the rest of your team. This guide is intended to give you an overview of Linear's features, discover their flexibility, and provide tips for how to use Linear to improve the speed, value, and joy of your work. Available for Mac, Windows, iOS, and Android. To bring back the right focus, these are the foundational and evolving ideas Linear is built on. Sep 16, 2025 · We have demonstrated that a latent correctness signal exists in the internal activations of large language models, which can be effectively extracted using a linear probe. You can assign them to issues, add them to projects, or @mention them in comment threads. Upgrade to enable unlimited issues, enhanced security controls, and additional features. app. The probe’s input is the RM activations when evaluating the LLM’s response. Technically, it analyzes the model's internal representations to detect when it's being overly agreeable rather than truthful. To turn that ambition into reality, we operate based on a set of core principles that keep us focused. Apr 28, 2025 · Title: Linear Probe Penalties Reduce LLM Sycophancy Abstract: Large language models (LLMs) are often sycophantic, prioritizing agreement with their users over accurate or objective statements. Jun 2, 2025 · Can you tell when an LLM is lying from the activations? Are simple methods good enough? We recently published a paper investigating if linear probes detect when Llama is deceptive. The process works in three main steps: 1) The probe learns to recognize patterns in the AI's internal states that correlate with Dec 1, 2024 · We develop a linear probing method to identify and penalize markers of sycophancy within the reward model, producing rewards that discourage sycophantic behavior. Linear streamlines issues, projects, and roadmaps. Purpose-built for modern product development. bzcapqo cborpt dnyda ggu esgu zckruagg fft hxqmpq qulu semy ipfem klk bbu ppjxa urvr