PC Serializing data - save wire, space and nerves

Niranufoti

Plantera
Serializing data for compact transmission
Table of Contents

Introduction
Ladies and gents, today I decided to get off my lazy :red: and actually build something I had in mind.
Imagine this: You have a password-protected door or whatever, and in order to prevent smart people with a mechanical lens and binoculars from seeing the correct solution, you have the actual logic somewhere far away, deep within your impenetrable control center. Problem: You have multiple of these doors, and each has a rather long password, leading a tangled mass of 25+ wires into your base that even you can't sort through anymore. If only there was a way to reduce the number of wires needed...
Oh wait, there is: Actual computers send masses of data through a single wire one bit after the other - can we recreate that in Terraria? Spoiler alert: I already did.

I call this "serialization" - the process of putting data in a line to be sent in an orderly fashion. Since we don't have high current and low current in Terraria and I didn't want to do timing-based stuff on the receiving end, I used two wires instead of one - but these two wires can be used to transmit (theoretically) any amount of data.

The Transistor
The core element of this is what I call the "transistor". In electronics, a transistor is a component where a so-called controlled current can only flow if there is a sufficient controlling current.
In Terraria, the transistor works based on the Faulty Logic Gate Lamp. As I outlined previously in the Official Introduction to Wiring thread (and afterwards checked in the decompiled source code), they work like this:
What gate is used doesn't matter [...]; neither do lamps placed above the faulty one.

When the faulty lamp gets a signal, the gate is randomly activated with a certain chance. This chance [is] determined by the logic gate lamps below the faulty one, more specifically (number of active lamps)/(number of total lamps), e.g. if you have three active ones and one inactive one between the faulty lamp and the gate, there's a 75% chance of activation.
An experimental application by @FelixNemis gave me the push I needed to invent this:
9Vx1LMN.png

To stay with the electronics terminology, I'll call the faulty lamp "emitter", the regular lamp "base" and the gate "collector". If the base is active, all signals sent to the emitter will be relayed to the collector. If the base is inactive, none will.
The great advantage of this is that it allows us to check the state of something without changing anything - assuming the state we want to check is tied to the transistor's base.
In order to simplify the terminology, I'll call a transistor active if its base is active. I might also call the collector "output" and the emitter "input".

The Telegraph
The perhaps easiest application for serializing data is to simply get data from one place to another, so I built a basic four-bit telegraph: Players can input a combination on the transmitting end, flick a switch and it is copied to the receiving end.
As stated above, I used two wires to connect the two sides. I could've gone with using one of them for 1 and one for 0, but instead went with an approach that's easier to construct: One of the two wires (green in this example) tells the receiver to move to the next bit, the other one (yellow) to switch the current bit on (or off, theoretically). Also, an additional pulse on the green wire is added at the end to trigger potential feedback (e.g. opening a password-protected door).

Serializing
The serializer is rather simple - when the "transmit" switch is hit, it just needs to go through each bit, send a pulse on the green wire and then send one on the yellow wire if the bit is set. Let's take a look at it:
vTt8yoq.png
sX2qUZ9.png

"Well, that's nice and orderly, but when you said 'simple', I expected the wiring to be simple, not... that D:"
I know, I know, I built it too compact for showcasing purposes, but bear with me. Let's first look at the seperate cells without caring about the wires: Each cell has red and green torches to indicate its status as well as two transistors. Initially, all cells are inactive, indicated by the red torch being active and the green torch and the transistors' bases being inactive.
Now, let's look at only the red and blue wires:
DxgiXNA.png
u5g8Rhd.png

The small lump of blue wire each cell has connects the torches, bases and the switch. If this wire receives a pulse, the cell is toggled. Additionally, the collector (output) of each cell's left transistor is connected to the blue wire. If the leftmost switch is hit, the red wire, connected to the emitters (inputs) of these (left) transistors receives a pulse and all active cells are toggled, i.e. all cells are reset to their initial "off" state. The bottom blue wire, connecting the rightmost switch to the dart trap at the top, starts transmission of the stored data.
To understand how exactly the data is serialized, let's take a look at the green and yellow wires:
2OflZwK.png

When the transmit switch is hit, the dart trap fires a projectile, activating the teal pressure pads in rapid succession (thanks to @AaroSA for exploring that).
The first pad of each cell, as well as the very last one, simply send a pulse through the green wire, telling the receiver to go to the next bit. The second pad of each cell sends a pulse to the right transistor. If the cell is active, this pulse continues to the yellow wire, telling the receiver to set the current bit.

Deserializing
The deserializer is a bit more complex, with more than twice as many transistors. This is the general rule - deserializing is always more difficult than serializing. But let's not be intimidated.
vSF5K3w.png
u4RrpAY.png

"You forgot the walls on the top layer. Also, the bottom corridor is useless"
Seriously? I'm presenting you with yellow notes, red zigzag, a blue and red dashed line and a green and yellow comb and you complain about that? Psh, kids these days.
On a more serious >note<, some of you will have already realized that the bottom right transistors of each cell don't do anything. Those are for state-checking, e.g. to see if the correct passcode was entered. In this basic telegraph, they don't serve any function (yet).
For more clarity, let's divide this into red, blue and yellow&green again:
feJte98.png
euzjixW.png
fuwoxAN.png

The yellow "notes" act like the blue lumps in the transmitter, except this time, they're not connected to a switch, but to the collector of their cell's top-right transistor. The red zigzag line resets the cells' states, which are stored in the bottom transistors. Why is the line zigzagged? To not connect with anything else. I don't have the space to add another layer between the top and bottom part.
The top transistors handle the listening: The leftmost one and each cell's top-left one connect to the green wire coming from the transmitter. Only one of those must be active at any time. When a signal is received through the green wire, the active green-listening transistor switches its own base and that of the other (top-right, yellow-listening) transistor of its cell off and those of the next cell on. When the leftmost transistor toggles itself off, it also resets the data storage (zigzag wire). The OR-gate on the right is simply there to change the wire color from red to blue to connect back to the left.
Now, at any time, one of two things may be the case: Either the leftmost transistor is active - this means the device is ready to receive data - or a pair of top transistors of a cell is active - this means the device is currently receiving data. Whenever a green pulse is received, the active listening cell will move one cell to the right. A pulse on the yellow wire will go through the top-right transistor of the currently active listening cell, toggling the (storage) cell below. The final green pulse returns the listening focus to the leftmost listener and might be used to trigger some kind of feedback.

The Future

So there you have it - any amount of data with just two wires. This thing can be extended to... however far a dart flies, and even then, you could easily have it shoot another dart. It could probably be made quicker and some safeguards could be added to prevent a transmitter from sending more data when transmission is in progress (assuming it takes long enough to transmit that the darts can fire again), and the receiver could answer over the same wire, assuming it stops listening for a moment.
I'm thrilled to see what crazy things you guys can come up with using this - I know I will might use this, as it was my original plan, to solve the problem described in the introduction - a passcode-protected door that doesn't check whether the code was correct by itself, but asks the auth server.
Also, if I find the time, I might add a concrete example to this - though, if someone else builds this in a more easily understandable way and records a video, I'd thankfully include it here as well.

That said, I want to again thank @Yoraiz0r for first pushing me towards the faulty lamps, @FelixNemis for the inspiration from his RS latch and @AaroSA for showing that teal pressure pads are activated by darts just passing by. You guys rock!
 
Last edited:
This post has taught me more about gates and possibilities with these new toys than anything else. Just working threw how this works really helps. I couldn't help but post a thank you!
[doublepost=1464564890,1464564673][/doublepost]Also I am curious, would it be possible to design this in such a way where we transmit the data at the speed of the dart pulse but without the green wire? Id imagine that we send a single pulse at the beginning to tell the deserializer to prepare for a message. The deseralizer would then fire its own dart with the proper offset in timing. The deseralizer's dart would cycle threw each of the cells on it's end. Does this make sense?
[doublepost=1464571349][/doublepost]I tried making a high speed version that worked by sending the main pulse every other frame with the bit pulse in between.... It doesn't seem to work correctly. Can anyone tell me why?

ZcGWKDD.png
 
Also I am curious, would it be possible to design this in such a way where we transmit the data at the speed of the dart pulse but without the green wire? Id imagine that we send a single pulse at the beginning to tell the deserializer to prepare for a message. The deseralizer would then fire its own dart with the proper offset in timing. The deseralizer's dart would cycle threw each of the cells on it's end. Does this make sense?
Of course it makes sense, but I deliberately decided not do to that.
I tried making a high speed version that worked by sending the main pulse every other frame with the bit pulse in between.... It doesn't seem to work correctly. Can anyone tell me why?
You're sending a bit putlse at the very beginning. I'm not sure if that's the problem, but it might be, since the dummy ghost is teleported before the logic gates are evaluated. If that teleport already triggers the (green) pressure plate, that would cause problems. If the teleport doesn't trigger it (at all), that's a problem as well.
Actually, if I'm not mistaken, you wouldn't even need the dummy engine - it should simply work by sending the bits one after the other during a single tick, presuming you make sure there's sufficient delay between the green pulses.
 
**EDIT** Fixed bug in design

I thought about it some more and realized both of those approaches may have been over-complicating things. Here is a working high speed system:

Updated Version:
VVdQABG.png

fCw97Sw.png


This version has a bug....
MGQ8JQ1.png

The hoik engine generates 60 act/s. Each activation we send a pulse on the yellow wire to signal a 1 or a pulse on the green wire to signal a 0. There is an extra pulse on the yellow wire created at the start when pulling the lever triggering a reset on the remote end. The design is almost identical to your original one, but this one can transfer data at 60 bit/s or 0.06 kbit/s. One key difference is every other transistor on the top row of the receiving end is connected to BOTH the green and yellow wire, as a pulse from either represents one clock cycle and one bit.

Is the 60 act/s hoik engine the fastest clock cycle we can generate?
 
Last edited:
Is the 60 act/s hoik engine the fastest clock cycle we can generate?
Using teleporters and weighted pressure plates, it is apparently possible to go faster, but that I highly experimental and I'd be careful around it.
However, it should be possible to hook up the different cells to each other directly in order to transfer all data in a single tick, as long as the receiving end is constructed in a way that can handle it (which it currently is).

Each activation we send a pulse on the yellow wire to signal a 1 or a pulse on the green wire to signal a 0.
Yeah, this is the other possible way to do it. For quicker designs (with multiple actions per tick), it's probably even more useful and intuitive since you don't risk as much confusion regarding the wire colors.
This could be extended using an RS latch - a device with two inputs where a signal to one input puts the latch in one state and signal to the other input in the other state; essentially a switch with two seperate buttons for "on" and "off". Using this as storage on the receiving end would remove the need for a reset mechanism.
If I find the time to, I might build something like that and add it to the OP.
 
From what I have learned so far from that thread I was able to double the transfer speed to 120 bit/s by adding two additional AND gates like so:
ALIXrIX.png


From the testing I have currently done, adding additional AND gates in similar fashion wont increase our act/s. I could use the weighted pressure plates to get another 60 act/s up to a total of 180 bit/s but then the player has to stand in a specific area and the usability suffers imo. I' hoping someone can figure out exactly what the limitations are for duplicating a signal with transistors.

I am starting to expirement with Trellis modulation. I am really curious to see what the fastest data transfer speeds we can reach are and then find a use for them.
 
This is a very cool, seemingly game changing concept, for Terraria computing in particular, perhaps. The bandwidth and the immediacy and flexibility are very impressive!:eek:

I didn't want to do timing-based stuff on the receiving end
You mean, like having a second dart track (or a second dummy engine) at the receiver, activated by the initial pulse, blindly opening up bins (cells) for the bits to fall into, in sequence. There would definitely be an elegance to having a system that used a single wire though ("just one!"), heh.

I had in mind, for a long time, to used skeletons (and then dummy ghosts, when they turned up) to send data along a serial hoik data line. I imagined the 1s and 0s as skeleton vs no-skeleton, physically flying along a hoik track in formation. An additional control bit skeleton always out in front to trigger the read-in mechanism that would take a snapshot of the pattern of NPCs, distributed across plates. The imagery would have been a fun demonstration. A step (perhaps not strictly necessary) towards building a memory interface for a hoiktronic computer.

So, how do each of you fancy this setup of yours for writing to/from a large store of data, in a world spanning machine? Definitely of possible use, or an optional extra? Are there any shortcuts to be taken to assemble a duplex system (read and write)? Perhaps unnecessary, but you could use the same (two) wires for either direction, with just a control transistor to disable input, toggling send/receive modes.

Then the question is, how many of these serial interfaces do you have per 'word' (e.g. byte) of data storage? When it's TEdit copy-paste you could just as easily have a set for each byte, then a memory controller that purely switches the serial line between each location (based on the address stored in it's register).

But that seems kind of wasteful of space, so you'd have a bunch of memory locations per serial controller, I guess. Then part of the memory address would select which serial controller, and the rest would select one of that controller's sub-locations, on a parallel bus.

But, is there any advantage to using a serial interface at all then? Will there be a trade-off between the space consumed by the switching apparatus (presumably less for switching serial line, compared to a parallel bus) and the space taken up by the serial interfaces themselves? One serial controller per stack of (256) memory locations, reaching from sky to hell (of a small world)...? Then how many columns across...? (Or have I got the likely orientations sideways? Should I just stop rambling?)
 
Last edited:
In regards to the single wire system, I was able to build one with synchronized darts before setting up the hoik engine version. The first pulse triggered a dart trap on the remote end and everything worked out fine... But then I thought why not use a hoik engine and make it faster? I tore out the dart traps and put a hoik engine on each side. I tried to start the recv sides engine on the first pulse, but no matter how I wired the thing I couldn't get the dummy ghosts to sync up their movement. Sometimes it would work, sometimes it would be a frame out of sync with the sending side. I wasn't sure how to fix this so I went back to the two wire approach instead.

I think the cost of using two wires is greatly outweighed by the increased speed that you get from a hoik engine powered unit though. Especially if you start to consider transferring large amounts of data. (especially now that unicorn engine's allow us to generate 2,160 activations per second equating to a theoretical 2.16kbit/s transfer speed if the game can handle it).

One possible application would be in the read/write access to the hard drive that was built in another thread. Imagine these devices as the host bus and mass storage bus on your real world computer and hard drives. The cables running between them are your ATA cables.

You would need some type of protocol to communicate between them (think ATAPI).
 
Last edited:
Screw the dart traps. We can (probably) use a single wire and still transmit all the data in a single tick by using relays/diodes on both sides to time everything correctly. Basic protocol would be this: A single pulse as a control pulse, telling the receiving end to read, and then every N ticks steps (where N is as low as possible) another (data) pulse if that bit is 1.
I promise you guys that by today's evening (Central European DST) I'll have a proof-of-concept for that up and running.
 
Last edited:
Is the 60 act/s hoik engine the fastest clock cycle we can generate?

Y-no? You can place a super-agent in the engine, and it will produce as many pulses as there are agents. 100 max by dummy count. 600 acts/s. You just have to be careful about their spawn order. Don't know how useful that is though since the gates can only be used once every game tick.
 
Last edited:
Sorry I should of posted this here sooner. Shortly after working on this we discovered Unicorn Engines. These new engines don't rely on multiple agents and can generate a max of 2,160 activations per second.
 
Ok, here we go: An 8-bit serial transmitter and receiver that transmit all data within one tick and uses only one wire. Since this works with absolute step count instead of wishi-washi projectile flight, I'm also perfectly OK with using this.
4zwAGzr.png

On the left, there's the transmitter. The eight switches connected to blue wire are the input, the red wire is for resetting and the right switch at the top (green&yellow wire) is for transmitting.
Transmitting is pretty straightforward (review the protocol here: When flipping the switch, the first (control) pulse is sent through the yellow wire, telling the receiver to start recording. At the same time, the green wire is hit, sending a signal that propagates through the diodes (I used always-active transistors because it looks better) and causes each active cell to send a pulse to the yellow wire if it's active - one bit per step.
The receiver is a bit more complicated (again): The left column of transistors shifts the currently written cell by one each step after receiving a signal for the first time (control transistors). The middle column actually does the writing (writers). The right column is for resetting (resetters). Extending the blue wires further to the right allows connecting more devices.
Let's see what exactly happens at the receiver in order:
  • Step 1: Wires
    • A signal is received on the yellow wire, triggering the topmost control transistor (as well as all writers)
  • Step 1: Gates
    • The topmost control transistor's base is active, so it outputs a pulse
    • The writers are all inactive, so nothing happens here
  • Step 2: Wires
    • The first control transistor's output wire does four things:
      • Deactivate the transistor, so that subsequent pulses are ignored
      • Enable the current (first) writer
      • Trigger the current (first) cell's reset
      • Relay the signal to the next (second) control transistor
    • At the same time, a signal might be coming in on the yellow wire
  • Step 2: Gates
    • The next (second) control transistor's base is always active
    • If the current (first) cell was previously active, it's reset transistor triggers the blue wire
    • In case a signal was received on the yellow wire, only the current (first) writer is active, hitting the blue wire
  • Step 3: Wires
    • The previous (first) cell's blue wire might have been hit once or twice, possibly changing its state
    • The current (second) control transistor's output wire is hit, which does four things:
      • Disable the previous (first) writer
      • Enable the current (second) writer
      • Trigger the current (second) cell's reset
      • Relay the signal to the next (third) control transistor
    • At the same time, a signal might be coming in on the yellow wire
  • Step 3: Gates
    • The next (third) control transistor's base is always active
    • If the current (second) cell was previously active, it's reset transistor triggers the blue wire
    • In case a signal was received on the yellow wire, only the current (second) writer is active, hitting the blue wire
  • Steps 4-9: Repeat this, always inserting the correct numbers for "previous", "current" and "next".
  • Step 10: Wires
    • The previous (eigth) cell's blue wire might have been hit once or twice, possibly changing its state
    • The current (ninth) control transistor's output wire is hit, which does two things:
      • Disable the previous (eigth) writer
      • Reactivate the first control transistor so that new signals in future iterations can be received
    • At this point, no more signals should be passed through the yellow wire according to protocol
  • Step 10: Gates
    • If another yellow pulse was received, the first control transistor would activate again, but produces smoke instead
TL; DR: Each control transistor resets its respective cell, enables its cell's writer, disables the previous cell's writer and passes on the signal to the next control transistor.

I might build a transceiver that adds an additional transistor layer to block receiving while transmitting, so that the same serial bus can be used to broadcast from multiple different devices. I'll also rework the OP (once I'm done procrastinationg), including this and previous contributions as advanced builds.
 
Last edited:
Sorry I should of posted this here sooner. Shortly after working on this we discovered Unicorn Engines. These new engines don't rely on multiple agents and can generate a max of 2,160 activations per second.

Yes... and if I understand correctly they also require the player to be riding a unicorn instead of doing other things, like... using the machine?
[doublepost=1464977679,1464976671][/doublepost]Just wanted to say, I really enjoy what you guys are doing here!
Before 1.3.1, I attempted pretty much the same thing with dummies. It worked, but it was complicated. Since I am not very computers or electronics savy, I called it "multiplexing". Good to know, it's called "serializing and deserializing" (?)
 
Just wanted to say, I really enjoy what you guys are doing here!
Before 1.3.1, I attempted pretty much the same thing with dummies. It worked, but it was complicated. Since I am not very computers or electronics savy, I called it "multiplexing". Good to know, it's called "serializing and deserializing" (?)

Actually both terms could really be used here depending on what your source data represents i think o_O

A multiplexer (MUX) by definition is a device that combines multiple analog/digital signals into one signal over some medium. The demultiplexer (DMUX) does the reverse, unpacking the original signals. Serialization is the process of translating some data into a format that can be stored or transferred.

In the hardware world people will usually say that the MUX produces a serialized signal that the DMUX later unpacks. Here "serialized" just describes the signal, MUX is the device. But if you look at programs or things that directly interact / understand data structures then people write serializers for them... The way I see it either term really works. The more accurate one I think depends on if your sending some specific data structure or just providing a constant connection for some other "signal".
 
I was very pleased to find that changing the colour arrangement will not break the telegraph.
Seeing as everything happens within a tick, was I wrong to assume it would?

What you've made here is brilliant, I never could made this much-needed mechanism myself. Thanks to you, I can send a 32-bit signal from a good distance away and decode it without any problem. May you continue to conceptualize great things in the future.
 
I was very pleased to find that changing the colour arrangement will not break the telegraph.
Seeing as everything happens within a tick, was I wrong to assume it would?
The color order only matters when working with pumps or teleporters, and only when multiple wires connect to the same pump/teleporter. See this thread for details.
 
Thanks for the previous answers; I'm back again to annoy you ('you' includes anyone who understands this).

I have my 32-bit signal which I would like to place through a transistor (to return either the input signal or no pulse); can I do this on a single wire without ruining it for the deserializer within a tick?

I hope there is a better method than the use of 32 transistors.

After reading your 8-bit one-tick pulse, I'm assuming there is a way to do this. I should probably consider (and actually start) learning the basics of this myself so that I may solve my own problems.


Also, how can I pass the signal across one wire through several gates, to split it into one of, say, 8 paths, and within how little time can it be done?

Thanks
 
Last edited:
I have my 32-bit signal which I would like to place through a transistor (to return either the input signal or no pulse); can I do this on a single wire without ruining it for the deserializer within a tick?

I hope there is a better method than the use of 32 transistors.
I'm not sure if I understand what you're trying to do. If you're essentially in neeed of what I posted above, you'll need at least one (I needed two) transistor for every cell, since a single logic gate can only activate once per iteration, so sending the whole signal through a single transistor wouldn't work.
Also, how can I pass the signal across one wire through several gates, to split it into one of, say, 8 paths, and within how little time can it be done?
You can take the basic form of any deserializer, but instead of switching storage cells when receiving a signal, it sends the single signals through different wires. That would be a simple serial → parallel bus adapter.
 
Back
Top Bottom