NVIDIA has released VIBETENSOR, an open-source research system software stack for deep learning. VIBETENSOR is generated by LLM-driven coding agents under high-level human guidance.
The system asks a concrete question: Can coding agents generate a coherent deep learning runtime that spans Python and JavaScript APIs to C++ runtime components and CUDA memory management and validate it only through the tool.
Architecture from frontend to CUDA runtime
VIBETENSOR implements a PyTorch-style eager tensor library with a C++20 core for CPU and CUDA, a torch-like Python overlay via nanobind, and an experimental Node.js/TypeScript interface. It targets Linux x86_64 and NVIDIA GPUs via CUDA, and builds without CUDA are intentionally disabled.
The core stack includes its own tensor and storage system, a schema-lite dispatcher, a reverse-mode autograd engine, a CUDA subsystem with streams, events and CUDA graphs, a stream-ordered caching allocator with diagnostics, and a static C ABI for dynamically loaded operator plugins. The frontend in Python and Node.js shares the C++ dispatcher, tensor implementation, autograd engine, and CUDA runtime.
Python overlay highlights one vibetensor.torch Namespaces with tensor factories, operator dispatch, and CUDA utilities. The Node.js frontend is built on the Node-API and focuses on async execution using worker scheduling with limitations on concurrent inflight work described in the implementation sections.
At runtime level, TensorImpl represents a view on reference-counting StorageWith size, strides, storage offset, dtype, device metadata and a shared version counter. It supports non-contiguous views and aliases. A TensorIterator The subsystem calculates iteration sizes and per-operand strides for elementwise and reduction operators, and the same logic is exposed through the plugin ABI, so external kernels follow the same aliasing and iteration rules.
Dispatcher is schema-lite. It maps operator names to implementations in CPU and CUDA dispatch keys and allows wrapper layers for autogradable and Python overrides. Device policies enforce invariants such as “all tensor inputs on the same device”, leaving room for specialized multi-device policies.
Autograd, CUDA Subsystem and Multi-GPU Fabric
Reverse-mode autograd uses node and edge graph objects and per-tensors. AutogradMeta. During backwardation, the engine maintains dependency calculations, per-input gradient buffers, and a ready queue. For CUDA tensor, it records and waits for CUDA events to synchronize cross-stream gradient flows. The system also includes an experimental multi-device autograde mode for research on cross-device performance.
The CUDA subsystem provides C++ wrappers for CUDA streams and events, a caching allocator with stream-ordered semantics, and CUDA graph capture and replay. Allocator includes diagnostics such as snapshots, statistics, memory-fraction caps, and GC ladders to make memory behavior observable in testing and debugging. Allocators integrate with “graph pools” to manage memory lifetime during CUDA graph capture and replay.
The Fabric Subsystem is an experimental multi-GPU layer. It exposes explicit peer-to-peer GPU access through CUDA P2P and integrated virtual addressing when the topology supports it. Fabric focuses on single-process multi-GPU execution and provides observational primitives such as statistics and event snapshots rather than a full distributed training stack.
As a reference extension, VIBETENSOR ships a best-effort CUTLASS-based Ring AllReduce plugin for NVIDIA Blackwell-class GPUs. This plugin binds the experimental Ring-AllReduce kernel, does not call NCCL, and is intended as an example, not as an NCCL replacement. The multi-GPU results in the paper depend on Fabric Plus this optional plugin, and they are only reported for Blackwell GPUs.
Interoperability and extension points
VIBETENSOR supports DLPack import and export for CPU and CUDA tensors and provides a C++20 SafeTensors loader and saver for serialization. Extensibility mechanisms include Python-level inspired overrides torch.libraryA versioned C plugin ABI, and hooks for custom GPU kernels written in Cutlass such as Triton and CUDA template libraries. Plugin ABI exposes dllpack-based DType and device metadata TensorIterator Helpers are therefore integrated with the same iteration and aliasing rules as the external kernel underlying operators.
AI-assisted development
VIBETENSOR was built using LLM-powered coding agents as lead code writers, guided only by high-level human specifiers. Over about 2 months, humans defined goals and constraints, then agents proposed code variations and executed builds and tests to validate them. The work does not introduce a new agent framework, it treats agents as black-box tools that modify the codebase under tool-based investigation. Validation relies on C++ testing (CTest), Python testing via Pytest, and interoperability checks against reference implementations such as PyTorch for selected operators. The research team also includes long-term training regressions and allocator and CUDA diagnostics to catch stateful bugs and performance distortions that are not visible in unit tests.
key takeaways
- AI-generated, CUDA-the first deep learning stack:VIBETENSOR is an Apache 2.0, open-source PyTorch-style eager runtime whose implementation changes were generated by LLM coding agents, targeting Linux x86_64 with NVIDIA GPUs and CUDA as a hard requirement.
- Full runtime architecture, not just the kernel: The system includes a C++20 tensor core (TensorImpl/Storage/TensorIterator), a schema-light dispatcher, reverse-mode autograd, a CUDA subsystem with streams, events, graphs, a stream-ordered caching allocator, and a version C plugin ABI, exposed via Python (
vibetensor.torch) and experimental Node.js frontend. - Tool-driven, agent-centric development workflow: Over ~2 months, humans specified high-level goals, while agents proposed diffs and validated them via discriminant checking, allocator diagnostics, and long-horizon training regressions against CTest, pytest, PyTorch, without per-diff manual code review.
- Strong microkernel speedup, slow end-to-end training: AI-generated kernels in Triton/CuTeDSL achieve up to ~5–6× speedup over the PyTorch baseline in different benchmarks, but full training workloads (Transformer toy tasks, CIFAR-10 ViT, miniGPT-style LM) run 1.7× to 6.2× slower than PyTorch, which emphasizes the difference between kernel and system-level performance.
check it out paper And repo here. Also, feel free to follow us Twitter And don’t forget to join us 100k+ ml subreddit and subscribe our newsletter. wait! Are you on Telegram? Now you can also connect with us on Telegram.
