23 Daily — ashhart/TensorFold is a Large Language Model (LLM) inference engine that appeared on GitHub Trending on November 12, 2026, supporting multiple platforms including Metal (for Apple Silicon), CUDA (NVIDIA GPUs), and Vulkan. It runs without requiring Python or additional machine learning frameworks, enhancing cross-platform AI model performance. According to the project's description, developed by ashhart, it has gained over 1900 stars and is emerging as a versatile solution for LLM inference.
Technical Architecture and Features of TensorFold
TensorFold serves language models via a Zig-compiled binary on Apple Silicon and NVIDIA GPUs. It reads checkpoints, tokenizes input requests, and runs Metal or CUDA kernels directly, eliminating the need for a Python environment or ML extensions.
The engine supports execution on macOS, Linux, and Windows, adapting the number of parallel streams accordingly—for example, handling up to eight simultaneous streams on CUDA. It provides a comprehensive API including model listing, completion requests, and message exchanges, enabling easy integration into applications.
Additionally, TensorFold offers executable binaries, a small web app, and a desktop workspace that bundles model downloads, hardware recommendations, live metrics, coding support, and document editing, facilitating rapid deployment and debugging for developers.
Enterprise and Experimental Modules: TensorFold Starlight and Living Weights
Beyond the core inference engine, TensorFold includes modules tailored for organizational and research use. TensorFold Starlight provides a managed endpoint compatible with OpenAI and Anthropic APIs, supporting identity provider authentication (OIDC or SAML), workload division across Mac, Linux GPU, and Windows nodes, with full offline capability.
The experimental Living Weights feature enables running models to learn new information on the fly by writing facts into their checkpoints using the Nemotron 3.5 Lightning server with validation. This feature is still in development and has challenges such as fact interference and recall accuracy degradation.
Editor's take: A new cross-platform option for LLM inference
TensorFold offers entrepreneurs and development teams a flexible high-performance LLM inference engine with cross-platform support and no dependency on Python. By supporting trending Apple Silicon alongside established NVIDIA GPUs, it broadens application deployment possibilities.
Entrepreneurs can test TensorFold with public checkpoints and desktop applications to verify performance and stability on their target hardware. For enterprise-scale model deployments, TensorFold Starlight's centralized management framework is worth evaluating. The experimental Living Weights feature is suitable for research teams exploring online learning capabilities.