A browser in the RAM IRC left over

TL;DR

• A Gemini App on the Cardputer: gemtext rendered for a 40-column screen, Tab between links, Enter to follow, Back to where you were, g for an address.
• Trust on first use, as Gemini expects: each capsule's certificate is pinned the first time; if it changes, the page stops and a dialog shows both fingerprints.
• Redirects, input prompts (searching works) and server errors handled; links to the web are refused politely.
• Bookmarks with b, and Saved Pages with s, or S for a page and everything it links to on the same capsule. They're listed on the start page and read offline.
• Every page streams through the SD card, so it arrives whole even with IRC connected, and pages bigger than memory are read from the card as you scroll.
• Two memory floors, measured on the device, and one dip I decided to accept.
• Tagged v0.5.0; the code is on my Gitea.

Why Gemini

The Cardputer already talks IRC and walks the Wi-Fi spectrum. A browser was the natural next app, and the web, with its megabyte pages and its JavaScript, is not something an ESP32-S3 with 340 KB of RAM should be asked to render. Gemini is. A page is a text file. A request is one line. The format is so small that the whole specification reads in a coffee break, and the community writes for exactly this kind of screen: gemlogs, link lists, bulletin boards, search engines, all in plain text.

It also wasn't on the plan. Milestone three, the LoRa radio, was next, so this became a side milestone, G1, with the same routine as the others: questions first, tests first, measured on the device.

The cast

Geminiport 1965, TLS, text/gemini

A protocol from 2019 that fits in a page: send a URL and CRLF, get a status line and a body. Its page format, gemtext, has six kinds of line, which is exactly as many as a 40-column screen needs.

The capsulesgeminiprotocol.net, Kennedy, Cosmos, Bubble

Gemini's sites are called capsules. These four were the test bench: the project's own, a search engine, an aggregator that redirects, and a bulletin board full of emoji my fonts can't draw.

The heapabout 68 KB free with IRC connected

What's left once Wi-Fi, IRC over TLS, the screen and the debug tools have taken theirs. A second TLS connection needs about 45 KB of it, for a while.

The SD card7.3 GB, mostly empty

Where the memory problem went to be solved. Pages stream through it, and Saved Pages live on it.

Designed by interrogation, round four

Sixteen questions this time, Q70 to Q85: trust on first use, which responses to handle, how big a page can be, how gemtext should look on 240 pixels, which keys do what, what the start page shows. Then a late request of my own: save pages to the card to read later, offline. That added four more questions (Q82 to Q85) and a word to the glossary. A Saved Page is kept until you delete it, and Storage Clean-up never offers it by age, the same rule as Notes: you chose to keep it.

Two more questions came up only once the device had answered some of mine, and one of those I answered twice. They're the memory story below.

The parsers, tested first

Before any network code, three small parsers, host-tested like everything else in this project:

• URLs. A gemtext link is usually relative (docs/faq.gmi, ../, ?q), so the client has to resolve it the way RFC 3986 says, dot segments and all. The resolver passes the RFC's own 32 reference examples, normal and abnormal, with gemini:// in place of http://. ../../../g from three levels deep is gemini://a/g, in case you were wondering. Nobody ever is, until a link breaks.
• The response header: two digits, a space, up to 1024 bytes of meta. 20 text/gemini; charset=utf-8 is a page, 31 gemini://… a redirect, 10 Enter search query a prompt.
• Gemtext, line by line: three heading levels, lists, quotes, links, and preformatted blocks between ``` lines, where nothing is interpreted.

Then a fourth piece I hadn't planned, for the fonts. They're Latin-1: accents fine, emoji and CJK not. So text goes through a filter that keeps everything up to U+00FF and turns the rest into ?. Bubble, the bulletin board, labels its links with emoji, and on the Cardputer they read ? Subspaces and ? Help. Honest, if not pretty.

A fetch, and whose certificate it is

Each fetch runs on a short-lived task of its own. A TLS handshake can take two seconds, and the main loop has a watchdog since the last milestone. Also, an idle browser should cost nothing, not a 6 KB stack sitting around waiting.

Gemini capsules mostly use self-signed certificates, so checking them against a certificate authority would reject half of Geminispace. The convention is trust on first use: the first time the Cardputer meets a capsule, it pins the certificate's SHA-256 in NVS. If the certificate is ever different, the page doesn't load; a dialog shows the old and new fingerprints and asks. The IRC client already did exactly this for self-signed IRC servers, so it was a matter of doing it again, per host and port, under a key short enough for NVS's 15-character limit (two letters and 12 hex digits of a hash).

Where to put a page

This is where the night went.

The first version read the body into one std::string. Then Cosmos, an aggregator, came back cut short at 8.7 KB with 107 KB free. Two reasons. Growing a string copies it into a block twice its size, and my check, rightly, refused to let that happen near the floor. Worse: with IRC connected, the largest free block of memory is about 31 KB. A 64 KB page in one piece was never going to happen, however much memory was free in total.

So a page became a TextBuffer: lines packed into 4 KB chunks, with an index of 6 bytes per line. No big block, no doubling copies. Storing each line as its own string would have cost about 40 bytes a line in overhead; on a 400-line page, that's a page.

Then the next wall. While the TLS connection is open it holds about 45 KB, so a single "stay above 40 KB" check during the download left room for barely 20 KB of page, and with IRC connected, none. Cosmos stopped at 4.6 KB. But the connection's memory comes back the moment it closes. So, first decision: two floors. Above 40 KB once the page is in; above 20 KB while the connection is open. And no fetch starts below 55 KB free.

Without IRC, Cosmos now arrived whole: 31.6 KB, 419 lines. With IRC, it still stopped at 4.6 KB, because during the transfer the heap sits right at the 20 KB line. Second decision: every page streams to the card. While the connection is open, the body goes to a cache file on the SD card in 1 KB pieces, each written by the storage task, which owns every card access. Only once the connection has closed and given its 45 KB back is the page loaded into memory, as much of it as the steady floor allows. Cosmos with IRC connected: the whole page on the card, 20 KB of it on screen.

And the rest of it? That was "Cut short: only part of it fits in memory", until I asked for the obvious.

Reading from the card as you scroll

Opening a page that doesn't fit, one pass over its file counts the lines and notes where every 64th one starts, plus whether it's inside a preformatted block, so a window starting there knows how to draw it. That's 4 bytes per 64 lines: about 1 KB for a 1 MB page. The same pass loads the first window. Scroll near the end of it and the next window is read in the background, starting at the index entry just before the line on top of the screen, so what you're reading doesn't move. The scrollbar follows your place in the whole page, not the window. Tab, at the last link in memory, pages down instead of wrapping to the top.

The first window read while scrolling held 20 lines. Its budget was computed with the old window still in memory, so it got what was left beside it. Now the request says how much the old window will give back. The next window was 227 lines: the whole rest of the page.

Pages for the screen alternate between two cache files, so the page you're reading is never overwritten by the next fetch. Background jobs, like saving a capsule's linked pages, use a third.

The dip I accepted

With IRC's TLS connection open and a Gemini one alongside, the lowest free heap during a fetch came out at 24 KB the first time I measured it, then 19.5, 15.5 and 13 KB. Same page, same code. The receive buffers grow with the size of the records the server sends, so the dip depends on the other end. My own code keeps its allocations above 20 KB; the TLS stack doesn't ask.

The options were: refuse to fetch below 70 KB, which with IRC connected would refuse almost every fetch; close IRC's connection for every page; or accept a dip to about 12 KB for a second or two. I took the third. It's written down with the numbers, in the plan's own words, as a decision and not an accident.

Saved Pages

s copies the page from the cache file to /gemini/saved/<capsule>/<path>.gmi, with one line added at the top: > Saved from <url> on <date>. It shows as a quote, so you know what you're reading, and it gives relative links their base. S saves the page and everything it links to on the same capsule, up to 30 pages, in the background with progress Toasts: Project Gemini and its five linked pages, six of six, in about twenty seconds.

The start page lists them by capsule, newest first. Inside a Saved Page, a link to another saved page opens the saved copy, and anything else goes online if it can, or says "Not saved, and offline" if it can't. r refreshes a Saved Page from the network, d deletes it after asking.

Where it hurt

• The bug from the first post, again. A status message ("Not Gemini: https://…") was cleared before it was ever drawn. It was timestamped with millis() after the main loop had read the clock, and in unsigned arithmetic, three milliseconds in the future is 49 days ago. That's exactly the bug that made toasts expire before they appeared in the first post. Same fix: a signed comparison. Same feeling.
• A space where none was. Wrapped link and list lines are indented, which meant re-wrapping their continuation rows. I rebuilt that text by joining the rows with spaces, so a long URL the first wrap had cut mid-word came out as faq.gm i. Now it re-wraps the exact rest of the line.
• 436 bytes. The refresh job needed to know where a Saved Page came from, so it opened the page, all of it, next to the App's copy of the same page and a fresh TLS connection. The heap's lowest point since boot: 436 bytes. Nothing crashed, which is less reassuring than it sounds. Now it reads one line, and the 55 KB start floor guards every fetch, not just the ones the App asks for. Lowest afterwards: 53.8 KB.
• The same trap, twice in one milestone. A forward declaration of NetworkClientSecure inside the project's namespace declares a different class that doesn't exist. I'd made that mistake in the Debug Console two milestones ago. I made it again, in the same way, and the compiler explained it again, at the same length.
• Antenna is down. The aggregator I meant as a default bookmark doesn't answer, from the Cardputer or from my PC. Cosmos took its place, and its redirect became the test case for redirects.

By the numbers

Where it stands

The Cardputer now carries a small library on its SD card: capsules saved on the train, read on the plane, refreshed when Wi-Fi comes back. All of it in the memory IRC wasn't using.