23 Daily — According to github.com, a 15-year-old developer has created a tool called LaplacesScissors that enables direct editing of weights in large language models (LLMs). This tool rotates the LLM's output matrix to perform protective edits on specific answers, providing a novel method for model customization and fine-tuning.
Technical Principles and Features of LaplacesScissors
LaplacesScissors achieves weight editing by computing rotations of the original output matrix. Its core process includes a "probe" phase, which constructs numerical inputs from model weights and traces directions with weak output discrimination and cancellation across layers along with associated weight coordinates. Then, the "diagnose" phase uses cases with expected answer strings to find planes or joint rotations that fix failed answers while protecting successful ones. All objectives and constraints are verified against actual GPU outputs. Scaling analysis applies this edit policy across checkpoints to measure deviations and recovery amounts.
Environment Setup and Usage
The tool requires a Rust toolchain, a Python environment capable of importing Triton 3.8, and CUDA or HIP drivers. Python manages the Triton compilation bridge, while Rust handles model loading, tokenization, execution, edit computation, and result storage. Users can specify model inputs from Hugging Face organization/model with revisions or a local safetensors directory. Edits are based on JSONL-formatted test cases specifying expected answer indices and prompts. The tool evaluates the original model's answers and identifies targeted weight edits to fix failures while preserving correct outputs.
Precise Weight Adjustments and Output Protection
Based on numerical probes, the tool computes readout rotations that preserve the original output matrix’s row Gram matrix, the midpoint of two hidden states, and reference and control states. Users can set bounds on rotation angles, edit norm, and output error tolerance to balance modification strength and answer preservation. The editing applies actual GPU rotations to verify candidate edits increase divergence while satisfying protection thresholds. Outputs include detailed JSON reports and accepted proposals enabling replay and verification.
Editor's take: Implications and Opportunities for Entrepreneurs
LaplacesScissors introduces a new approach to deep fine-tuning of large language models by enabling precise weight edits without full retraining. For startups developing customized AI applications, this technology can significantly reduce development time and computing costs. Verifying effectiveness involves running tests on private datasets to compare pre- and post-edit accuracy on target and preserved answers. Technically, the open-source integration with Rust, Triton, CUDA, and HIP offers entrepreneurs a flexible and scalable platform for AI model manipulation, potentially accelerating innovation within the AI startup ecosystem.