Technologies Nearby

Technologies Nearby
Fun With Data

I previously wrote about other platforms that aim to tackle problems similar to our own Time Safari, but I haven't yet written about the technologies under the covers. It is time.

First, I must say that the underlying philosophy for our tech is "distributed". (It's similar to the project's underlying philosophy for economic development: it is based on the voluntarily choices of the participants.) One part of that goal is to minimize the trust of server code: if something is private, it should be totally private to everyone including the servers; the aim is a zero-trust service for critical things, much like Signal, but where some of the data is meant to be public.

For a quick summary, here are how the different data elements are represented in the system:

  • Server On-chain, ie embedded in a verifiable history of data
    • Publicly Visible, ie seen by anyone in the world
      • Descriptions
      • Counts of Time & Money
      • Project Map Locations
    • Blockable, ie potentially hidden from others
      • Personal ID
  • Server Off-chain, ie self-contained, modifiable data points
    • Personal ID Visibility, ie who can see what other IDs
    • Personal Interests & Map Locations
    • Gift & Project Images
  • Client, ie only held on individual devices (unless shared directly)
    • Contacts
    • Profile Images

Those are the design principles implemented by the Endorser Search server and the Time Safari client. The following comparisons will focus on how the data is shared.

Note that the ultimate goal is the ability for all server operations to be possible on the clients, directly P2P. Decisions have been made to prioritize speed-to-customer, and some server features may always be supported due to their ease-of-use but we expect some to choose to migrate to a fully P2P ecosystem.

I'll first discuss a few full-fledged data storage systems that seem to match our needs the closest, ones that use cryptography & networks to manage identities and permissions and even storage.

  • Holochain has DHT discovery built-in and is a compelling replacement for the Endorser server back-end: it's a chain of activity shared with your selected peers. It's a fit for the contact data but not the text and counts because the addresses go with the content, just like every other ledger, and I don't want to reveal any identifiers publicly that can be used to correlate data. (There may be an approach here for a pure P2P app.)
  • The SOLID Project is great for private data: it's a mature set of APIs and permissions for locating your data storage wherever you want. Unfortunately, this seems to also have the quality that the creator IDs/addresses are bound with data, so public data could be correlated. (There may be an approach here for a pure P2P app.)
  • The Midnight Network is an up-and-coming blockchain that aims to allow private data on the platform. It's by the makers of Cardano and I believe there is potential there, but I have not yet evaluated it because it's not on mainnet yet and I don't plan to spend time until it's more mature. Our long-term solution could benefit from zero-knowledge proofs, though I'm not sure; the best way to keep secrets is to not publish them anywhere, even encrypted.
  • ActivityPub (used by Mastodon and soon even ghost.io) is designed for decentralized messaging and is worth watching; we have built a few facilities for reputation and authentic chains of data, so we would have to replicate that somehow.
  • The AT Protocol (by Bluesky) is also making noise but there are no plans to even look until they add federation.

Now I'll share more low-level tools that would take even more development to integrate.

  • The Ethereum Attestation Service aims to simplify attestations, either on-chain or off-chain. They just got awarded a grant to include ZK (zero-knowledge) tech in their attestations, which could be interesting.
  • The FLO cryptocurrency was designed to make arbitrary statements on-chain and it has been used by Medici Land Governance (my employer) for land records. However, there is no private data... and nowadays no more users, either.
  • Obsidian now has facilities for selectively sharing data. It is still gaining traction as an easy-to-use tool, so it's worth a look for direct P2P functionality.
  • The Reticulum Network is a good underlying network. It is aimed at uncensorable, robust node communication, so personal identities could be delivered with messages inside it (ie. a P2P approach).
  • NOSTR is a simple protocol for publishing data widely – but it's all public.

You may have noticed I mention P2P a lot: future iterations of our server could involve advanced cryptography such as ZKP (zero-knowledge proofs), but the very best way to keep private data private is to not share it anywhere, even encrypted. So the ultimate goal is to allow for some combination of public data with fully private data on your own devices. Other tech in this area includes Arachnode, Pear, Gun, PeerBit, TinySSB, Yjs, Syncthing, etc.

Further Activities

  • Obviously, if you're technical enough to get this far then you might try out some of those tools.
    • For our tech, see in videos how the Endorser server currently keeps ID information private, and if you like code then see tests for visibility here and here.
  • Try Obsidian to keep your notes, journal, and other writings. I mention it because it is the most user-friendly of all those tools, it is on all platforms including mobile now, and it has extensions to support the needs even of hard-core data nerds. I keep meaning to seriously try it and see how their sync feature shares with others, so this is partly a reminder to myself.