Royalty Language & Diffusion Models

In October 2025 I started a relentless, year-long journey exploring the right medium, human-computer interaction, backend infrastructure, LLM architecture, and diffusion architecture to make royalty-based language and diffusion models possible. The earlier trials of 2026, covered in Ambient Intelligent Systems, taught me quite a bit. Honestly, so much happened before I reached MaryOS.

It started as an iPhone app, where every answer from Seer carried a Contributions sheet: which of your own documents shaped it, which authors on the network did, and the share of the royalty owed to each.

While thinking through go-to-market strategy — how to get users into the system, how to earn their trust, how to aggregate their thoughts properly — I built a couple of front-end web systems on the side, including a Chrome extension that added more data to what I believe the HCI might be. The extension was quite neat. A user could highlight anything and the Seer logo would appear, letting them search the database for sources aligned with the selection, with an LLM as the medium between those sources and the user.

To prove the retrieval worked, I downloaded the entire arXiv database to index and serve, so every result cited actual researchers. Opening Contributions listed the people behind the retrieved passages, each with a short summary of their work and keywords pulled from their papers. At the bottom of every panel sat the whole point in one line: creators are compensated every time their knowledge is used.

It showcased the idea of an LLM as a machine that stitches reality together rather than fabricating it.

These were very fruitful experiences, and they left me with more questions than I started with. Should users upload their data directly, or is that too much friction? Should this tool double as a truth teller, verifying LLM outputs against a database of real data that can be cited to a person, rather than to a web link that could very well be misinformation itself? And how does one even verify that the data submitted to the network is true?

So many thoughts. It became a vortex of its own, because building a system whose whole selling point is truth, in a sea of smoke and mirrors, is another level of complexity. I'm not naturally someone who feels safe releasing a product to customers without verifying the approach is bulletproof, but there is a balance between pushing an idea forward and fixing the plane while it's flying, and it's one you just have to get used to.

Now I've reached a streamlined version that I want to release on macOS first. MaryOS will be built on the side as a holistic operating system on Linux, while the macOS application, Ambient, is a gentle introduction to the idea. I'll get into Ambient in the post following this one, but I wanted to mention now how it will serve as a vessel for this core idea: language and image models that can be served while attributing content to its authors and paying them their dues in the generative age.

Iterating on the right approach for both, while researching safety procedures for deepfakes this summer (Veil & Scorpion, with the code for detection itself in Garde), led me to a single solution. An equilibrium of sorts, where a proper royalty tracking system can be built so that it also provides a gating mechanism to protect the likeness of individuals. I'll cover the royalty tracking mechanism here, and in a future post I'll write about how the same system can stop generative models from authoring protected content through a security-forward retrofit.

The idea is to add a new layer before the model's activations and after its adapters. This layer queries a database of stored data assigned to users who want to protect the copyright of their visuals, from human identity to brand placement. The database is searched at every step of the diffusion process, and an attribution matrix is built step by step until generation completes, logging to the network the payouts owed before the asset is served to the requesting user. It's a step that can be applied to service-based models and even on-device models.

The payout half already runs on the language side. Here is Gita, Seer's attribution layer, settling a single response back in April: the one owner whose data carried it receives the full payout, and the service charge is billed on top.

The implementation can also work the other way. Instead of building an attribution matrix, the generation can stop early and return a censored output, marking it as infringing and guarding against the generation of a protected identity. I'll go into the technical details in an ML Architecture post once the first version, for image and visual protection, is released later this month.

The design of this database started with Seer and its custom vector storage, and it soon split out into Thread, which can be explored through its Partition files. Thread is designed to be everyone's personal database, with their data structured and prepared for retrieval in a royalty-based system, in a way that leads toward the P2P ideology I've been envisioning for a while. That's something I'll explore in Ambient when the time comes.

In the end, I'm hoping this year's experiments arrive at a single understanding that fosters a more stable growth trajectory for artificial intelligence: for writers, creators, and scientists, and for everyday consumers who aren't aware their data is being used without their consent. All while offering an experience profound enough to help us reassess what connection truly means, in a social world where social networks did nothing but deprive us of it.

Victims feel safe because the harms are being addressed by implementations that are realistic and actionable. Open-source communities see boundaries they agree on. Corporations have clear regulations to follow before they capitalize. Privacy-focused users who prefer on-device can show they used Veil through an invisible watermark on their generations.

So I'd say the final vision begins with Ambient on macOS and its eventual transition into MaryOS on Linux, while I write Gita's Ballad on the side.