STTP

Semantic Token Transfer Protocol. What if a new internet was built around this idea that the data stored on our devices automatically have prepared tokens upon creation and modification, where these tokens are available at all times and our machines are able to communicate and share their tokens with other machines, peer-to-peer? Depending on the request, these tokens are aggregated and predicted against, within a layer that replicates the tail end of the layers in a transformer, prior to the prediction that generates a text completion.

Every application and piece of software built on this new operating system automatically has the ability to read the aggregated token context, "compiling" it into a built-in prediction layer prior to rendering. What type of system would that be? What would application surfaces look like? What would websites look like? Should this be an internet-only protocol, or should this be at a lower level? This is what vector databases and RAG do, a proven architecture, just not decentralized at the OS level or implemented fully at the network level as a default yet.

I have been trusting my intuition lately, and I have had this idea since last year, but I didn't want to think I was forcing a reality just because of a new system I was implementing myself in practice. Getting closer to finishing MaryOS, and understanding its system of systems, has helped clarify and/or provide enough confidence that such a future is actually possible. Working with federated technologies and running across Privacy Preserving Measurement (PPM) and its Distributed Aggregation Protocol (DAP) has helped provide this vision of dots that I am not sure anyone is connecting yet, or if they are already in the works.

PPM addresses exactly the piece left implicit: how do multiple machines contribute to an aggregate without any of them, or any server, seeing each other's raw values.

A small headless model outputs raw logits over the vocabulary (a vector of size |vocab|, unnormalized scores, no softmax). Summing those logit vectors across many models and then applying one softmax is mathematically different from averaging each model's own probability distribution. Summing logits before normalizing is equivalent, once renormalized, to multiplying the individual probability distributions together. I guess this is Hinton's Product of Experts (1999), not a Mixture of Experts.

It's a metaphor for consensus in the purest sense, powered by networking, resulting in the true idea of aggregation or general intelligence. Maybe extracting the softmax is still elementary, but I wonder if these thoughts activate an understanding of what might be the proper end game, especially when working towards the idea that connecting millions of smaller models will eventually be a better outcome than a few monolithic frontier models.