Whence '\n'?
computers · programming

If you do just foo, the following justfile will write a single byte 0x0A to a file named bar:

x := "\n"

foo:
  printf '{{x}}' > bar

Let's find out where that 0x0A byte comes from.

just is written in Rust, and the just parser has a function called cook_string, which transforms a just string token containing escape sequences into a UTF-8 string.

The code is here here.

With some irrelevant details elided, it looks like this:

for c in text.chars() {
  match state {
    …
    State::Backslash => {
      match c {
        'n' => cooked.push('\n'),
        …
      }
      …
    }
    …
  }
}

So just asks rustc to insert the result of evaluating the Rust '\n' character escape. Let's take a look at how rustc handles '\n'.

rustc's escape code handling is in the lexer, in a function called scan_escape, which is here.

With some details removed:

let res: char = match chars.next().ok_or(EscapeError::LoneSlash)? {
    …
    'n' => '\n',
    …
};

rustc is written in Rust and compiles itself, so somehow rustc is delegating to rustc to figure out what '\n' means, which seems odd, to say the least, and we still haven't seen the naked 0x0A byte we're looking for.

rustc wasn't always written in Rust though. Before it was self-hosted, early versions were written in OCaml.

GitHub has old versions of the OCaml version of rustc, which handled character escapes in the lexer here.

and char_escape = parse
  …
  | 'n' { end_char (Char.code '\n') lexbuf }
  …

So rustc asks the OCaml compiler to insert the result of evaluating the OCaml character escape '\n'. Which is totally reasonable, but still not a 0x0A in sight.

Going one step deeper, let's look the OCaml lexer here.

And finally, some clarity:

let char_for_backslash = function
    'n' -> '\010'
  …

When the OCaml compiler sees \n, it inserts the result of evaluating the OCaml character escape \010, which is a decimal character escape, and since 0x0A is 10, we finally have our byte value.

So when have a \n character escape in your justfile, the just binary contains a 0x0A byte in some form, which it will then write to your final string.

That 0x0A byte was put there by rustc, which contained it's own 0x0A byte somewhere in the binary, which was stuffed there by its rustc progenitor.

rustc is currently at version 1.81.0, so this has happened at least 81 times since rustc 1.0 was first released, and probably many more times than that before 1.0, with rustcs furtively smuggling 0x0A bytes from one to the other, all the way back to when it was written in OCaml, when finally the first 0x0A byte was stuffed into a rustc binary by the OCaml compiler, which evaluated it from a decimal character escape '\010'.

This post was inspired by another post about exactly the same thing. I couldn't find it when I looked for it, so I wrote this. All credit to the original author for noticing how interesting this rabbit hole is.


Ordinal Theory
bitcoin · computers · internet · cryptocurrency · ordinals

I've been working on a numbering scheme for satoshis that allows tracking and transferring individual sats. These numbers are called ordinals, and constitute a numeric namespace for Bitcoin. Satoshis are numbered in the order in which they're mined, and transferred from transaction inputs to transaction outputs in first-in-first-out order. More details are available in the BIP.

Ordinals don't require a separate token, another blockchain, or any changes to Bitcoin. They work right now.

Ordinals can be represented in a few ways:

With raw notation, like so 1905530482684727°. The number is the ordinal number, and the "°" is the Romance language ordinal symbol.

With decimal notation, like so 738848.482684727°. The first number is the block height, and the second is the index of the ordinal within the block.

With degree notation, like so 0°108848′992″482684727‴. We'll get to that in a moment.

A block explorer is available at ordinals.com. You can explore recent blocks, and look up ordinals by number, decimal, degree, or name.

Arbitrary assets, such as NFTs, security tokens, accounts, or stablecoins can be attached to Ordinals.

Ordinals is an open-source project, developed on GitHub. The project consists of a BIP describing the ordinal scheme, an index that communicates with a Bitcoin Core node to track the location of all ordinals, a wallet that allows making ordinal-aware transactions, a block explorer for interactive exploration of the blockchain, and functionality for minting ordinal NFTs.

Rarity

Since ordinals can be tracked and transferred, people will naturally want to collect them. Ordinal theorists can decide for themselves which sats are rare and desirable, but I wanted to provide some hints.

Bitcoin has periodic events, some frequent, some more uncommon, and these naturally lend themselves to a system of rarity. These periodic events are:

This gives us the following rarity levels:

Which brings us to degree notation, which unambiguously represents an ordinal in a way that makes rarity easy to see at a glance:

A°B′C″D‴
│ │ │ ╰─ Index of sat in the block
│ │ ╰─── Index of block in difficulty adjustment period
│ ╰───── Index of block in halving epoch
╰─────── Cycle, numbered starting from 0

Ordinal theorists often use the terms "hour", "minute", "second", and "third" for A, B, C, and D, respectively.

Now for some examples. This ordinal is common:

1°1′1″1‴
│ │ │ ╰─ Not first sat in block
│ │ ╰─── Not first block in difficutly adjustment period
│ ╰───── Not first block in halving epoch
╰─────── Second cycle

This ordinal is uncommon:

1°1′1″0‴
│ │ │ ╰─ First sat in block
│ │ ╰─── Not first block in difficutly adjustment period
│ ╰───── Not first block in halving epoch
╰─────── Second cycle

This ordinal is rare:

1°1′0″0‴
│ │ │ ╰─ First sat in block
│ │ ╰─── First block in difficulty adjustment period
│ ╰───── Not the first block in halving epoch
╰─────── Second cycle

This ordinal is epic:

1°0′1″0‴
│ │ │ ╰─ First sat in block
│ │ ╰─── Not first block in difficulty adjustment period
│ ╰───── First block in halving epoch
╰─────── Second cycle

This ordinal is legendary:

1°0′0″0‴
│ │ │ ╰─ First sat in block
│ │ ╰─── First block in difficulty adjustment period
│ ╰───── First block in halving epoch
╰─────── Second cycle

And this ordinal is mythic:

0°0′0″0‴
│ │ │ ╰─ First sat in block
│ │ ╰─── First block in difficulty adjustment period
│ ╰───── First block in halving epoch
╰─────── First cycle

If the block offset is zero, it may be omitted. This is the uncommon ordinal from above:

1°1′1″
│ │ ╰─ Not first block in difficutly adjustment period
│ ╰─── Not first block in halving epoch
╰───── Second cycle

Supply

Total Supply

Current Supply

At the moment, even uncommon ordinals are quite rare. As of this writing, 745,855 uncommon ordinals have been mined - one per 25.6 bitcoin in circulation.

Names

Each ordinal has a name, consisting of the letters A through Z, that get shorter the larger the ordinal is. They could start short and get longer, but then all the good, short names would be trapped in the unspendable genesis block.

As an example, 1905530482684727°'s name is "iaiufjszmoba". The name of the last ordinal to be mined is "a". Every combination of 10 characters or less is out there, or will be out there, some day.

Exotics

Ordinals may be prized for reasons other than their name or rarity. This might be due to a quality of the number itself, like having an integer square or cube root. Or it might be due to a connection to a historical event, such as ordinals from block 477,120, the block in which SegWit activated, or ordinal 2099999997689999°, the last ordinal that will ever be mined.

Such ordinals are termed "exotic". Which ordinals are exotic and what makes them so is subjective. Ordinal theorists are are encouraged to seek out exotics based on criteria of their own devising.

Archaeology

A lively community of archaeologists devoted to cataloging and collecting early NFTs has sprung up. Here's a great summary of historical NFTs by Chainleft.

A commonly accepted cut-off for early NFTs is March 19th, 2018, the date the first ERC-721 contract, SU SQUARES, was deployed on Ethereum.

Whether or not ordinals are of interest to NFT archaeologists is an open question! In one sense, ordinals were created in early 2022, when I finalized the Ordinals specification. In this sense, they are not of historical interest.

In another sense though, ordinals were in fact created by Satoshi Nakamoto in 2009 when he mined the Bitcoin genesis block. In this sense, ordinals, and especially early ordinals, are certainly of historical interest.

I personally favor the latter view. This is not least because the ordinals were independently discovered on at least two separate occasions, long before the era of modern NFTs began.

On August 21st, 2012, Charlie Lee posted a proposal to add proof-of-stake to Bitcoin to the Bitocin Talk forum. This wasn't an asset scheme, but did use the ordinal algorithm, and was implemented but never deployed.

On October 8th, 2012, jl2012 posted a scheme to the the same forum which uses decimal notation and has all the important properties of ordinals. The scheme was discussed but never implemented.

These independent inventions of ordinals indicate in some way that ordinals were discovered, or rediscovered, and not invented. The ordinals are an inevitability of the mathematics of Bitcoin, stemming not from their modern documentation, but from their ancient genesis. They are the culmination of a sequence of events set in motion with the mining of the first block, so many years ago.


Multisig Is Probably Better Than Proof Of Stake
computers · cryptocurrency

A large reputable member threshold multisig operating as functionaries for a Bitcoin-pegged deterministic replicated state machine sidechain with as-compatible-as-possible-with-mainchain semantics is probably more reliable and secure that most alternative chains.


Investable Desiderata
computers · cryptocurrency · internet · markets

There are a lot of things that I wish would happen, but don't have the time to actually do myself. I complain about such things all the time to basically anyone who will listen. Such efforts are all well and good, and sometimes actually pay off, but additionally, I'd like to materially support people who might actually do these things.

This post, which I'll try to keep up-to-date, if I remember, documents the projects which I wish some talented go-getter would take on, and in which I would invest money in if given the opportunity.

If you are one of these aforementioned go-getters, email me!


Standards-based Social Networking
computers · internet

Expanding on this tweet: You could launch a super fancy social networking site with all the features that everyone expects, but built entirely on existing open standards:


Upgrading Email
computers · internet

Why?

Return email to its rightful place as the one true messaging protocol. It's this or use a dozen messaging apps for the rest of your life, there is no other realistic option.

How?

Add the features that make people use other messaging protocols to email.

Which features?

Contacts

Description: Chat, mobile notifications, read receipts, and other features, should not be available to arbitrary addresses. User approves addresses who can use these features.

Implementation: Add-to-contacts UI in client.

Degredation: N/A.

Chat

Description: Dedicated chat UI. No subject line. Return to send.

Implementation: Add dedicated chat UI to client. Messages sent from chat UI have header chat: true. Messages received with chat: true are shown in chat UI. Only chat from contacts appear in chat UI. Chat from non-contacts appears in message request UI.

Degradation: Message desplayed in inbox UI.

Mobile Notifications

Description: Some messages trigger mobile notification. User configurable.

Implementation: Mobile app has notification category for mail from contacts with header urgent: true.

Degradation: No mobile notification.

Read Receipts and Typing Indicators

Description: See read receipts and typing indicators on sent mail.

Implementation: Add receipt-endpoint: <URL> header to outgoing mail. Display read receipts and typing indicators sent to own receipt-endpoint. Only send read receipts and typing indicators to contacts.

Degredation: No read receipts or typing indicators.

Voice Calling

Description: Initiate voice calls from within client.

Implementation: Query MX server for voice call endpoint. Display voice call button next to conversations where MX server supports voice calls.

Degredation: No voice call button displayed when not supported.

Video Calling

Same as voice calling.

Message Requests

Description: Email from arbitrary senders should go into a message request folder, instead of in the inbox, to prevent phishing attacks, and encourage users to whitelist contacts which enables richer features.

Implementation: Email from senders who are not contacts goes into message request folder. Message request folder is distinct from spam folder.

Degredation: Email appears in main inbox.

Emoji

Description: Mail consisting of a small number of emoji is displayed in a larger font.

Implementation: Display mail consisting of a small number of emoji in a larger font.

Degredation: Emoji appear smol and sad.

Stickers

Description: Users can send "stickers" which are scaled appropriately.

Implementation: Display mail consisting of a single image scaled to UI.

Degredation: Images appear at fixed size.

Improved Rich Text Support

Description: Allow rich formatting without breaking plain text readers or using arbitrary HTML.

Implementation: Messages can indicate that they are formatted as markdown. Format messages for graphical clients and display with added line-breaks for plain text readers.

Degredation: Messages appear as plain text markdown, which is highly readable.

End-to-end Encryption

Description: Messages between users are end-to-end encrypted to avoid being read by third parties.

Implementation: Query MX server for public key of receipient, encrypt mail for recipient with pubkey.

Degredation: Messages are not encrypted. This is strict improvement to not encrypting any email. Once email E2EE is widespread enough, refuse to send mail to recipients that don't support it.

Prevent Replies Leaking Information

Description: Including messages in replies bloats messages, can accidentally leak information, and has is not necessary due to threaded clients.

Implementation: Don't include original message in reply by default. User may use dedicated quote reply UI to include quotes of original message.

Degredation: None.

Faster Message Delivery

Description: Deliver email instantly without any delay.

Implementation: Define mapping of email symantics over HTTP/3 as email/2, providing persistant connections and an efficient binary encoding. Query MX server email/2 support and use it if available, using persistant connections and binary encoding for instant message delivery.

Degredation: Messages are delivered at normal speed.

Disappearing Messages

Description: Messages can be configured to dissappear after some amount of time.

Implementation: Query if recipient MX server supports disappearing messages. If so, let sender set amount of time before messages disappear. Enforced by recipient, not sender, since sender enforcement is impossible.

Degredation: Option to turn on disappearing messages is not available when not supported by recipient.

Group Chats

Description: It should be possible to easily add and remove addresses from group chats.

Implementation: Email already supports group emails, but configuring adding and removing addresses from group chats is difficult. Query MX server for group chat support endpoint. Dedicated API for creating and updating group chats, referenced by ID. Group chat emails are associated with ID, instead of just being arbitrary lists of addresses, allowing any participant, if they have the permissions, to add and remove members from chat.

Degredation: Group chats are received as ordinary multi-address emails.


The Path to Email World Domination
computers · internet

A letter to Maalika Manoharan, Product Manager, Gmail. Sent via LinkedIn Inmail, which is palpably ironic.

Hi Maalika,

My name is Casey Rodarmor, and I am a former Google SRE and SWE.

I am writing you today because you may be one of the few people that can deliver us from the multi-messenger hell that the human race finds itself in today. Everyone needs to use six or more messaging apps, and everyone hates it.

However, there is an easy way out of this. Email is a federated, decentralized, standards-based messaging platform that everyone already uses. The only reason people use other messaging protocols is because those messaging protocols have features that email doesn't have.

Here is the path to email world domination. It is very simple:

Add the features that email is missing, one by one, in a backwards compatible way.

These features include:

These features can all be implemented in backwards compatible ways that gracefully degrade when they aren't supported.

These features should be designed and implemented in an open process, involving the internet community, public RFCs, and implementations by multiple clients and service providers.

This is a major undertaking, but the benefit is huge, both for Google, Gmail, and the internet community.

Every single non-email protocol will fall by the wayside, and we can return to a simpler, more productive, and more secure world, one where everyone just uses email to communicate, an open, standards-based, internet native protocol.

This could be, without exaggeration, the most important work of your life.

Do you have time to discuss this with me? I have no agenda, just someone who thinks that this is possible, and ultimately very achievable.

With best regards, Casey Rodarmor


Configuration
computers

Conventional software configuration is backwards.

If you want an HTTP server to listen on port 80 and serve some directory full of files, you tell the service to bind to port 80 and pass it the path of the files to serve:

$ http-server --port 80 --files /srv/www
Serving `/srv/www` via HTTP on port 80.

This is backwards. Let's call this kind of configuration "internal configuration", because it's configuration that happens inside programs.

Instead, processes should declare a namespace of resources that they consume or produce:

$ mount http-server /http
Mounted `http-server` instance at `/http`.
$ tree /http
/http
  /socket
    /listen
  /imports
    /files

After mounting, resources would be mapped as you see fit, from outside the process, without having to stop or reconfigure the service:

$ listen 80 /http/socket/listen
Connections to port 80 are being forwarded to socket `/http/socket/listen`.
$ export /srv/www /http/imports/files
Filesystem `/srv/www` exported to `/http/imports/files`.

Let's call this "external configuration", because it's configuration that happens outside programs.

External configuration has a number of benefits:


Lightning Mints
cryptocurrency · internet · computers · markets

Federated blind mints have attractive privacy, scaling, and security properties that are highly complementary to those of Bitcoin and the Lightning Network.

I originally became interested in blind mints while thinking about Lightning Network wallet usability issues. When Lightning works, it is fantastic, but keeping a node running and managing a wallet present a number of challenges, such as channel unavailability due to force closes, the unpredictability of the on-chain fee environment, the complexity of channel backup, and the involved and often subtle need to manage liquidity.

All of these problems are tractable for a skilled node operator, but may not be soluble in the context of self-hosted wallets operated by non-technical users, hereafter normies. If this is the case, then normies may have no choice but to use hosted Lightning wallets, compromising their privacy and exposing them to custodial risk.

Chaumian mints, also known as Chaumian banks, or blind mints, offer a compelling solution to these problems, particularly when operation is federated. Chaumian mints, through the use of blind signatures, have extremely appealing privacy properties. The mint operators do not know the number of users, their identities, account balances, or transaction histories. Additionally, mint transactions are cheap and can be performed at unlimited scale.

Mint implementations, typified by eCash, have hitherto been centralized, and thus, like all centralized, custodial services, expose users to custodial risk in the form of operator absquatulation and mismanagement. To fix this, mint operation can be federated, with all operations performed by a quorum of nodes controlled by different parties.

Despite these interesting properties, Chaumian mints have largely been forgotten. This post gives an excellent overview of the phenomenon. I believe that Chaumian mints are currently severely underrated in general, and in particular deserve consideration as a potential avenue for improving custodial Lightning Network wallets.

Compared to a naïve hosted Lightning Network wallet, a service operated as a federated Chaumian mint offers excellent privacy, usability, security, and scaling.

Privacy: Privacy leaks from a Lightning mint come in two forms, internal and external, when a mint operator or an outside actor, respectively, observes sensitive information.

Blind signatures protect against internal privacy leaks, making them a strict improvement in that respect over custodial Lightning wallets.

When compared to a single-user Lightning network wallet, Lightning mints also protect against external privacy leaks. If the activity of a single-user Lightning Network wallet can be observed, which is possible but non-trivial, all such activity is preemptively that of the owner of the wallet. However, similar to a standard custodial Lightning Network wallet, any observable Lightning Network activity of a Lightning mint is the aggregate activity of its users, who thus form an anonymity set. If the number of users, and thus the anonymity set size, is large, external privacy leaks are also prevented.

Usability: Compared to a self-managed Lightning Network wallet, and similar to a standard custodial Lightning Network wallet, Lightning mint wallets offer superior usability. A user need not be concerned with the details of node operation or channel management, and can deposit to and withdraw from their account with standard Lightning Network invoices.

Security: The security of a Lightning mint is weaker than that of a self-hosted wallet. A quorum of federation members can abscond with funds. However, compared to a standard custodial Lightning Network wallet, security is greatly improved. Additionally, federation members might be located in different jurisdictions, making the mint robust to regulatory interference. Furthermore, members might be entities with online reputations, such as anonymous Bitcoin Twitter users with an established history of productive shitposting, providing further assurances against mismanagement and fraud.

Scaling: Mint operations are extremely lightweight, similar to Lightning Network transactions, so scaling properties are similar to the Lightning Network itself. Additionally, users need not manage their own channels, so a well-capitalized federation can open channels efficiently, lowering the per-transaction channel management overhead.

Interoperability and market dynamics: Additionally, my hope is that such systems will be developed with a standardized protocol for communication between wallet interfaces and mint backends. This would allow users to use different backends with the same local wallet interface, encouraging competition in the market.

For more discussion of Chaumian mints and their applicability to Bitcoin, see fedimint.org. Elsirion, the author, is also at work on MiniMint, a federated Chaumian mint with Bitcoin and eventually Lightning Network support.

To close with a bit of speculation, I believe that Chaumian mints were never of particular interest or importance because they were limited to interoperating with the fiat currencies of the time. With the ascendance of Bitcoin, mints now have access to a powerful, decentralized, and uncensorable currency , made economical and fast by the Lightning Network.

I believe this layering of Chaumian mints on top of Bitcoin and the Lightning Network will, in the fullness of time, be demonstrated to be enormously powerful, and make Chaumian mints themselves worthy of renewed study and consideration.