Get all your news in one place.
100's of premium titles.
One app.
Start reading
TechRadar
TechRadar
Rahim Amir

UNBELIEVABLE: The next age of llms dev gets a small llm running at 10 tokens a second locally on a 10 microcontroller | History Defined

MIT
AI chip on a lit circuit board.
  • A developer has a 28.9-million-parameter model generating TinyStories-style text at 9.88 tokens per second, fully offline, on a $10 microcontroller
  • It was achieved by fitting a language model on a chip with 512KB of RAM by leaving most of it in flash storage
  • The project is available on GitHub under an MIT license

A developer going by slvDev has a 28.9-million-parameter language model generating text on an ESP32-S3, a microcontroller built for sensor nodes and smart plugs, at 9.88 tokens per second, with nothing leaving the chip.

Sign up to read this article
Read news from 100's of titles, curated specifically for you.
Already a member? Sign in here
Related Stories
Top stories on inkl right now
One subscription that gives you access to news from hundreds of sites
Already a member? Sign in here
Our Picks
Noah Wyle’s ‘The Pitt’ Plagiarism Controversy Takes New Turn After ‘ER’ Creator’s Widow Gets Greenlight To Sue
“I wonder if she'd be suing if Noah Wyle wasn't the lead,” one user said as ‘The Pitt’ faces a breach of contract lawsuit for alleged similarities with ‘ER’
Bored Panda
Bored Panda
Black couple sits down at Waffle House for breakfast. Then they watch as customers who arrived after them get served: ‘Feel like this is racism’
Allegedly, no one came to their table
We Got This Covered
We Got This Covered
Las Vegas hostess says a customer called her names. Then she called the woman one back and the owner kicked her out: ‘I don’t care’
She didn’t let it slide
We Got This Covered
We Got This Covered
From Katy Perry to drug addiction: What the Russell Brand rape trial jury didn't hear
The matters were deemed to be ‘sensitive’ and were removed from the prosecutor’s speech
The Independent UK
The Independent UK
Dakota Johnson And Machine Gun Kelly Fuel Fresh Dating Rumours After California Dinner Outing
Dakota Johnson and Machine Gun Kelly were photographed together outside a Calabasas restaurant, adding to weeks of dating speculation based on sightings and unnamed sources.
International Business Times UK
International Business Times UK
6 Things You Should Stop Keeping in Your Refrigerator Door
Open your refrigerator, and chances are the door is packed with milk, eggs, salad dressing, and whatever else fits into those convenient little shelves. After...
Saving Advice
Saving Advice
Fourteen days free
Download the app
One app. One membership.
100+ trusted global sources.
Download on the AppStore Get it on Google Play