JJoche Ojeda.NET · XAF · AI · architecture
Oct 19, 20268 min read/2026/10/19/tiny-corp-tinygrad-open-source/

What Is Tiny Corp, and Is tinygrad Open Source? I Ran It on My Mac

tinygradTiny CorpGeorge HotzOpen SourceDeep LearningPyTorch

In my last essay I mentioned George Hotz, the guy who unlocked the first iPhone and later founded comma.ai. His current company is Tiny Corp, and its product is a deep-learning framework called tinygrad. I'd seen the name many times; this time I wanted to know what it actually is, whether it's really open source, and what it does on my own machine.

Is it open source?

Yes. tinygrad is on GitHub at tinygrad/tinygrad under the MIT licence, one of the most permissive there is: you can use it, change it and ship it, commercially or not.

As of 19 October 2026:
Licence MIT
Repository created October 2020
Stars / forks 33,776 / 4,380
Contributors 511
Latest release v0.14.0 (24 August 2026)
Status still pre-1.0 ("alpha" in their own FAQ)

Tiny Corp itself came later: Hotz announced in May 2023 that it had raised $5.1 million (his blog post).

What tinygrad is

tinygrad describes itself as something between PyTorch and Andrej Karpathy's micrograd (a tiny teaching library). In practice it's a complete deep-learning stack:

  • a tensor library with autograd, with an API close to PyTorch's;
  • an IR and compiler that fuses operations and generates GPU code;
  • a JIT that captures and replays kernels;
  • nn, optimizers and datasets for real training.

The difference from PyTorch is philosophy. PyTorch is a huge, mature system; tinygrad's guiding rule is that the whole thing stays small enough to read. Its own line counter reports 9,594 lines of core code for v0.14.0 (26,036 in total; my plain count of every Python file, including auto-generated hardware bindings, is much bigger). And to add a new accelerator, according to the README, it needs to support only about 25 low-level operations.

It runs on a long list of hardware: Apple's Metal, NVIDIA (CUDA and its own NV backend), AMD, Qualcomm (QCOM), OpenCL, WebGPU and plain CPU. The repo even contains its own low-level AMD driver code, which says a lot about how deep they're willing to go.

I ran it on my Mac Studio

Install (the README recommends installing from source):

git clone https://github.com/tinygrad/tinygrad.git
cd tinygrad
python3 -m pip install -e .

It picked the M1 Max's GPU without any configuration:

python3 -c "from tinygrad import Device; print(Device.DEFAULT)"

METAL

Laziness: one kernel instead of a giant tensor

This is the demo from the README, and it shows the core idea:

DEBUG=2 python3 -c "from tinygrad import Tensor;
N = 1024; a, b = Tensor.empty(N, N), Tensor.empty(N, N);
(a.reshape(N, 1, N) * b.T.reshape(1, N, N)).sum(axis=2).realize()"

Read literally, that code builds a 1024 × 1024 × 1024 tensor (a billion numbers, 4 GB) by multiplying everything with everything, then sums one axis. That's a matrix multiplication written in the most wasteful way possible.

tinygrad doesn't run it that way. Every tensor is lazy: nothing is computed until you ask for the result (.realize()). By then it can see the whole expression, so it fuses the multiply and the sum into one GPU kernel and never builds the giant intermediate. With DEBUG=2 it prints each kernel it runs, and on my Mac there was exactly one:

*** METAL      1   ...   tm 612.83us/0.61ms (3504 GFLOPS ...)

One kernel, 0.6 milliseconds, about 3.5 TFLOPS on the M1 Max. If you've read my tensor article, this is "multiplication is the verb", with a compiler deciding the fastest way to say it.

Training: MNIST in about 11 seconds

The repo ships examples/beautiful_mnist.py, a small network for handwritten digits. On my Mac Studio:

loss:   0.07 test_accuracy: 98.31%: 100%|███████| 70/70 [00:11<00:00,  6.35it/s]

98.31% accuracy after about 11 seconds of training (13 seconds including compiling the kernels). The README says about 5 seconds, presumably on faster hardware. Either way: a full training loop, autograd and optimizer included, from a codebase you could read in a few weekends.

How it compares with what I already use

On this Mac I train with Apple's MLX, and some of my projects use PyTorch on the Apple GPU (MPS). So I built a small benchmark and ran the same two tests on all three. The code and every result are public: egarim/tinygrad-vs-mlx.

1. Training the same model. The character-level model from my weights article, 1.84 million weights, on Don Quijote. Same starting weights (generated once and loaded into each framework), the same batches in the same order, the same settings. All three start at exactly the same loss, 4.4469, and after 600 steps they agree to about three decimals: this is the same computation, so the timing is a fair comparison.

Framework ms per step ↓ first step ↓ 600 steps ↓
MLX 34.6 0.9 s 21.9 s
tinygrad + BEAM search (cached) 41.8 6.5 s 33.4 s
PyTorch (MPS) 45.2 3.4 s 30.5 s
tinygrad 56.7 3.5 s 39.6 s

2. Raw matrix multiply, 4096 × 4096, in trillions of operations per second (↑):

Framework 32-bit 16-bit
MLX 8.28 9.23
PyTorch (MPS) 8.40 9.18
tinygrad (plain) 5.52 6.66
tinygrad + TinyJit + BEAM 8.59 9.54

What I take from it:

  • MLX is still the fastest way to train on a Mac, and it compiles almost instantly.
  • Plain tinygrad pays for being written in Python: it plans the work again on every call. Its JIT (TinyJit) captures the plan once and replays it.
  • BEAM is the interesting part. It's tinygrad's kernel search: it tries many versions of each GPU kernel and keeps the fastest. With it, tinygrad's own generated code matched and slightly beat Apple's hand-tuned matrix multiply, and its training step got faster than PyTorch's. The price: the first run spent almost 5 minutes searching. tinygrad saves the results, so the second run started in 6.5 seconds.
  • One setting almost fooled me. MLX's Adam optimizer doesn't apply "bias correction" by default; PyTorch and tinygrad do. With MLX's default, the same training run ended at a loss of 2.32 instead of 1.64, which looks like MLX learning worse. It isn't; it's a different default. Comparing frameworks means checking every default, not only the timings.

One Mac, one run each, a small model: treat differences under about 10% as noise.

How Tiny Corp makes money

tinygrad is free. The company sells hardware: the tinybox, a deep-learning computer with up to four GPUs, configured on their website and made to order. Their FAQ says it was benchmarked in MLPerf Training 4.0 against machines costing ten times more, and that the company's goal is to make petaflops of compute a commodity, so AI isn't reserved for whoever owns a datacentre.

The way they run it is as unusual as the code: no salespeople, no custom procurement forms, payment by bank transfer only, and the FAQ's answer to "can I invest?" is to contribute code. They also pay cash bounties for specific improvements to tinygrad, and say contributing is the best way to get hired.

And it's used in production: according to the FAQ, comma.ai's openpilot uses tinygrad to run its driving model on the Snapdragon 845 GPU in the car device, replacing Qualcomm's own SNPE runtime.

A rule that connects to my last essay

The contribution guidelines in the README have a section that made me smile after writing about AI tools and experience. New contributors whose PRs look AI-written get them closed without feedback, and anyone who used AI has to disclose what for. The reason, in their words: "No human should waste time reading AI slop."

The rest of the rules point the same way: no code golf, every claimed speedup must be benchmarked, big complex diffs won't be reviewed, refactors must be clear wins. It's a project that values simplicity a reviewer can verify over volume of code, which is exactly the problem when anyone can generate thousands of lines in an afternoon.

Who is tinygrad for?

From my short test, my take:

  • Learning how deep learning frameworks really work. This is where it shines. In PyTorch the path from your code to the GPU runs through a very large codebase; in tinygrad, DEBUG=4 shows you the generated GPU code, and the code that produced it is a few files away.
  • Hardware people. If you have an accelerator, ~25 operations get you a full framework, training included.
  • Running models on unusual devices, as openpilot does on a phone-class chip.
  • Not yet for a team that needs stability above all. It's pre-1.0 and changes fast. For production training on NVIDIA, PyTorch is still the safe default; on a Mac, I've been using Apple's MLX (as in my weights experiment).

How much to trust this

  • The numbers are from one Mac Studio (M1 Max), one afternoon, v0.14.0. The 3.5 TFLOPS in the laziness demo is the GPU time of one kernel as tinygrad reports it; the benchmark tables measure wall-clock time per call, including each framework's overhead, which is why plain tinygrad shows less there.
  • The repo statistics are a snapshot from 19 October 2026; stars and contributors grow every day.
  • The company facts come from Tiny Corp's own website and FAQ, and Hotz's blog. I haven't verified the MLPerf comparison or the openpilot details independently.

The one-line mental model

tinygrad is PyTorch's idea rebuilt small enough to read, with a compiler that waits until it sees your whole computation and then runs it as few GPU kernels as possible. It's free and MIT-licensed; Tiny Corp sells the boxes to run it on.