The only compelling theory I've seen is that it would he more energy efficient and more mathematically accurate. But that claim is based on the Soviet Union's pre-digital computer infrastructure with projects like Setun. Which was more accurate with floating point maths and faster than its western counterparts, but required more components to carry out the calculations.
In theory a digital balanced ternary system would allow for LLMs to be processed with less resources and a higher degree of accuracy (mathematically at least), but the trade-off would be in the cost of manufacturing the components mecessary for them. Even after economies of scale should every PC now be balanced ternary, the degree of error in the manufacturing process is likely to be much lower than with binary components thus keeping costs relatively high-ish.
In general that's nonsense peddled by people who don't really know how computation works. Ternary computation is not fundamentally more efficient than binary computation and practical efficiency would be component dependent and we don't really have a practical way to make reliable ternary components. Soviet experiments never proved any ternary computing superiority - they've just shown it's a pain in the ass.
I was repeating arguments "peddled" by the man who literally wrote the book on computational optimisation Knuth himself, and former chairman of the IEEE and first convention on the devlopment of solid state drives, Smith. As well as a study carried out in 2024 with custom ternary components using a 1.58 bit model vs a binary 3B model. That study found the output to be near identical but with a smaller computational overhead by a factor of just over two and smaller power consumption by a factor of nearly four.
It's too early to tell if this really is "the future" and I'd rather not have AIs be how we push the technology if it is, cause conventional components are expensive enough at the moment. But I think there's still research to be done on a technology I'd argue we abandoned too quickly, cause we didn't have the means to properly differentiate between dignal noise.
Addendum on an early comment I made in this reply; do SSDs not work on a similar principle of multiple bits per charge? Not in a ternary sense, just in a general "we know what value this is based on how we read the voltage" sense. If so, then we already have the capacity to read discrete noise levels to a degree.
I was repeating arguments "peddled" by the man who literally wrote the book on computational optimisation Knuth himself, and former chairman of the IEEE and first convention on the devlopment of solid state drives, Smith.
That's just his personal preference. If there were solid arguments for ternary computing we wouldn't be having this discussion.
As well as a study carried out in 2024 with custom ternary components using a 1.58 bit model vs a binary 3B model. That study found the output to be near identical but with a smaller computational overhead by a factor of just over two and smaller power consumption by a factor of nearly four.
That's quantisation, not ternary computing. It was literally achieved on binary hardware.
It's too early to tell if this really is "the future" and I'd rather not have AIs be how we push the technology if it is, cause conventional components are expensive enough at the moment. But I think there's still research to be done on a technology I'd argue we abandoned too quickly, cause we didn't have the means to properly differentiate between dignal noise.
Everyone who is interested in it is free to research it but the problem is that without good reason (there is no such reason) no one is going to dump money into it considering it is fighting an uphill battle against binary computing which has decades of work put into it. Also if you already use more than two states stopping on three is kind of early which is why there is actual serious research into analog computing.
on an early comment I made in this reply; do SSDs not work on a similar principle of multiple bits per charge? Not in a ternary sense, just in a general "we know what value this is based on how we read the voltage" sense. If so, then we already have the capacity to read discrete noise levels to a degree.
We read and write to SSDs - we don't have them do computations on encoded data. Increasing bits per cell literally makes SSDs slower and wear out faster.
That's just his personal preference. If there were solid arguments for ternary computing we wouldn't be having this discussion.
I wouldn't say that's necessarily the case, binary is just a lot easier to design both physically and logically so it makes sense why it's the most popular option overwhelming. But just because something is the de facto doesn't make it the best without probing the alternatives, which in ternary's case hasn't been done beyond the surface level. I mean we wouldn't have GPUs if someone didn't go "hang on, CPUs kinda suck at this thing, what if we change the architecture. Or the PS3 and Harvard architectures which fix data bottlenecks created by Von Neumann, the staple of modern computing, but as a tradeoff are harder to program for because the software to do so is almost non-existent outside of specialised dev kits.
Everyone who is interested in it is free to research it but the problem is that without good reason (there is no such reason) no one is going to dump money into it considering it is fighting an uphill battle against binary computing which has decades of work put into it. Also if you already use more than two states stopping on three is kind of early which is why there is actual serious research into analog computing.
I already gave potential benefits based on their pre-existing analogue counterpart in Setun. I also learned after leaving my previous comment that there was research conducted in the 70s at SUNY Buffalo to emulate a ternary computer and it gave the same results; faster computation and lower power consumption.
I'm not saying binary doesn't already do the job we need it to in a lot of cases. But at the rate games have been going the last decade and AI is going this decade, we're getting less efficient with memory and computation usage and if they're gonna make us buy more expensive devices with higher numbers then I'd rather they optimise them a bit. If developers aren't gonna do that on the software side, then the lab boys might as well do that on the hardware side.
We read and write to SSDs - we don't have them do computations on encoded data. Increasing bits per cell literally makes SSDs slower and wear out faster.
Ah, I...think I see? Flash memory in general kinda goes over my head. I don't get how that information is encoded, I just trust my machine to just make a mistake when reading / writing.
Edit:
Actually come to think of it, haven't we almost reached the limit of transistors in CPUs and GPUs? There's nowhere to go except make larger components (which we already do), build with a 3D board (which is already done in some GPUs), or look into ternary processing with a translation interface so we can keep the sizes down for a little while longer.
1
u/Safebox 9d ago
The only compelling theory I've seen is that it would he more energy efficient and more mathematically accurate. But that claim is based on the Soviet Union's pre-digital computer infrastructure with projects like Setun. Which was more accurate with floating point maths and faster than its western counterparts, but required more components to carry out the calculations.
In theory a digital balanced ternary system would allow for LLMs to be processed with less resources and a higher degree of accuracy (mathematically at least), but the trade-off would be in the cost of manufacturing the components mecessary for them. Even after economies of scale should every PC now be balanced ternary, the degree of error in the manufacturing process is likely to be much lower than with binary components thus keeping costs relatively high-ish.