ATProto Browser
Experimental browser for the Atmosphere
Experimental browser for the Atmosphere
{
"uri": "at://did:plc:6ll5xi67lyuyovt6fiv4fnjo/industries.geesawra.website/7f2633d8d2e059e30a789516935b6d4dcc3cd2ecabc6dec32d7f94a4ff0260f6",
"cid": "bafyreihkzhcf6gqdwt5g6vkjfwy5w5sg6hufyg6esvv4vbzfgeb3wmzgvy",
"value": {
"$type": "industries.geesawra.website",
"title": "Hosting a website on your ATProto PDS | geesawra.industries",
"embeds": [
{
"$type": "blob",
"ref": {
"$link": "bafkreigenauclkihgxjiqkciyum46maffnsdqxmhidr223jnzq67ae2jfu"
},
"mimeType": "*/*",
"size": 1017
},
{
"$type": "blob",
"ref": {
"$link": "bafkreidamltn7c4xcbt24vpgkgbxkmjcneldzazvmlmpva3cps7zvylini"
},
"mimeType": "*/*",
"size": 148
},
{
"$type": "blob",
"ref": {
"$link": "bafkreiao2jv7wrvyrwicczk3cecvoccs7r3qrt2c4ci2lvx2bs6on6xv7u"
},
"mimeType": "*/*",
"size": 1988
}
],
"content": "<!DOCTYPE html><html lang=\"en-us\"><head>\n <meta charset=\"utf-8\">\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n <title>Hosting a website on your ATProto PDS | geesawra.industries</title>\n <link rel=\"stylesheet\" href=\"/at/geesawra.industries/blobs/bafkreigenauclkihgxjiqkciyum46maffnsdqxmhidr223jnzq67ae2jfu\">\n <link rel=\"stylesheet\" href=\"/at/geesawra.industries/blobs/bafkreidamltn7c4xcbt24vpgkgbxkmjcneldzazvmlmpva3cps7zvylini\">\n <link rel=\"preconnect\" href=\"https://fonts.googleapis.com\">\n<link rel=\"preconnect\" href=\"https://fonts.gstatic.com\" crossorigin=\"\">\n<link href=\"https://fonts.googleapis.com/css2?family=Atkinson+Hyperlegible+Mono:ital,wght@0,200..800;1,200..800&family=Atkinson+Hyperlegible+Next:ital,wght@0,200..800;1,200..800&family=Bungee+Shade&display=swap\" rel=\"stylesheet\">\n<link rel=\"stylesheet\" href=\"/at/geesawra.industries/blobs/bafkreiao2jv7wrvyrwicczk3cecvoccs7r3qrt2c4ci2lvx2bs6on6xv7u\">\n<script type=\"module\" src=\"https://cdn.jsdelivr.net/npm/@daviddarnes/bluesky-post\"></script>\n\n </head>\n\n <body>\n <nav>\n <ul class=\"menu\">\n \n <li><a href=\"/\">Home</a></li>\n \n <li><a href=\"/at/geesawra.industries/industries.geesawra.website/df50555e2c1437af886a0c0e56855ac376a853f0b7e99f89b0fc4dc98b2b6e29\">Posts</a></li>\n \n </ul>\n <hr>\n </nav>\n\n<div class=\"article-meta\">\n<h1><span class=\"title\">Hosting a website on your ATProto PDS</span></h1>\n<h3 class=\"author\">Authored by geesawra</h3>\n<h3 class=\"date\">2024/12/16</h3>\n</div>\n\n<main>\n<p>I built a thing called <a href=\"https://github.com/geesawra/atpage\"><code>atpage</code></a>, it publishes websites to an ATProto PDS and provides a handy viewer tool.</p>\n<p>You can find it <a href=\"https://github.com/geesawra/atpage\">here</a>.</p>\n<hr>\n<p>Have you ever wanted to host a fully-fledged website on your <a href=\"https://atproto.com/\">ATProto</a> <a href=\"https://atproto.com/guides/glossary#pds-personal-data-server\">PDS</a>?</p>\n<p>No?</p>\n<p>Too bad, it’s gonna happen anyway.</p>\n<blockquote>\n<p><strong>Disclaimer</strong>: <em>I’m not an ATProto expert, I may very well be doing nasty stuff, please do not emulate!</em></p></blockquote>\n<h2 id=\"how-it-works\">How it works</h2>\n<p>In ATProto, objects can be uniquely identified by an <a href=\"https://atproto.com/specs/at-uri-scheme\">AT URI</a>, made up of three components:</p>\n<ul>\n<li><strong>User identifier</strong>: can either be a DID, or an @-handle<sup id=\"fnref:1\"><a href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\">1</a></sup>.</li>\n<li><strong>Collection name</strong>: essentially the “table name” of ATProto, containing a set of well-defined objects called <em>records</em>.</li>\n<li><strong>Record key</strong>: a string that uniquely identifies the record.</li>\n</ul>\n<p>This is an example of an AT URI:</p>\n<pre tabindex=\"0\"><code>at://geesawra.industries/industries.geesawra.website/0J5SYQ0SVQTKF\n</code></pre><p><code>geesawra.industries</code> is the user identifier, <code>industries.geesawra.website</code> is the collection name, while <code>0J5SYQ0SVQTKF</code> is the record key.</p>\n<p>You can see the raw content of the record associated to this AT URI <a href=\"https://pdsls.dev/at/did:plc:6ll5xi67lyuyovt6fiv4fnjo/industries.geesawra.website/0J5SYQ0SVQTKF\">here</a>.</p>\n<p><code>atpage</code> is made of a series of components:</p>\n<ul>\n<li>Lexicon definition</li>\n<li>Web viewer</li>\n<li>Publishing tool</li>\n</ul>\n<p>The sum of these components allows you to publish websites on an ATProto PDS, and view them in a web browser without third-party services intervention.</p>\n<h3 id=\"lexicon-definition\">Lexicon definition</h3>\n<p>Records needs schemas, which in ATProto terms are called <strong><a href=\"https://atproto.com/guides/glossary#lexicon\">Lexicons</a></strong>.</p>\n<p>A web page contains HTML, which can reference resources like CSS, JavaScript, images or even other HTML pages.</p>\n<p>The most basic lexicon would contain just the HTML for a page, but in <code>atpage</code> <em>every resource</em> that would be loaded from the same origin <em>must</em> live on the PDS: no half-measures.</p>\n<p>In order to do that, I decided to store referenced objects as <strong>blobs</strong>, and the page content in a <code>Page</code> lexicon record:</p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-json\" data-lang=\"json\"><span style=\"display:flex;\"><span>{\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"lexicon\"</span>: <span style=\"color:#ae81ff\">1</span>,\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"id\"</span>: <span style=\"color:#e6db74\">\"industries.geesawra.website.page\"</span>,\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"defs\"</span>: {\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"main\"</span>: {\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"type\"</span>: <span style=\"color:#e6db74\">\"record\"</span>,\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"description\"</span>: <span style=\"color:#e6db74\">\"A record holding the content of a page, addressable by its record key\"</span>,\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"key\"</span>: <span style=\"color:#e6db74\">\"any\"</span>,\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"record\"</span>: {\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"type\"</span>: <span style=\"color:#e6db74\">\"object\"</span>,\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"required\"</span>: [<span style=\"color:#e6db74\">\"content\"</span>],\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"properties\"</span>: {\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"content\"</span>: { <span style=\"color:#f92672\">\"type\"</span>: <span style=\"color:#e6db74\">\"string\"</span> },\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"embeds\"</span>: {\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"type\"</span>: <span style=\"color:#e6db74\">\"array\"</span>,\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"items\"</span>: {\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"type\"</span>: <span style=\"color:#e6db74\">\"blob\"</span>,\n</span></span><span style=\"display:flex;\"><span> <span style=\"color:#f92672\">\"accept\"</span>: [<span style=\"color:#e6db74\">\"*\"</span>]\n</span></span><span style=\"display:flex;\"><span> }\n</span></span><span style=\"display:flex;\"><span> }\n</span></span><span style=\"display:flex;\"><span> }\n</span></span><span style=\"display:flex;\"><span> }\n</span></span><span style=\"display:flex;\"><span> }\n</span></span><span style=\"display:flex;\"><span> }\n</span></span><span style=\"display:flex;\"><span>}\n</span></span></code></pre></div><p>Blobs are an interesting beast: they can contain arbitrary data, and are referenced and keyed by the PDS at upload time.</p>\n<p>The Bluesky <a href=\"https://docs.bsky.app/docs/api/com-atproto-repo-upload-blob\">documentation</a> for the <code>com.atproto.repo.uploadBlob</code> method says:</p>\n<blockquote>\n<p>The blob will be deleted if it is not referenced within a time window (eg, minutes).</p></blockquote>\n<p>The implications are two-fold:</p>\n<ul>\n<li>If your page contains CSS in a referenced resource, it must be uploaded as a blob and referenced in the <code>embeds</code> lexicon field.</li>\n<li>If you delete a page, the associated blobs will be deleted automatically as well.</li>\n</ul>\n<p>Neat!</p>\n<h3 id=\"web-viewer\">Web viewer</h3>\n<p>The original <code>atpage</code> design started as a proof-of-concept around the idea of referencing data from a PDS, <strong>without</strong> using third-party servers: the browser should be able to solve AT URIs and display its content as a web page.</p>\n<p>Let’s say Firefox or Chrome decided to natively support <code>atpage</code>’s Lexicon, an AT URI resolution process would look like this:</p>\n<ol>\n<li>Validate and split the AT URI in its parts.</li>\n<li>Solve the <a href=\"https://atproto.com/guides/glossary#did-decentralized-id\">DID</a> document to look up the PDS URL associated with the user identifier.</li>\n<li>Fetch the specified record from the associated collection from said PDS URL.</li>\n</ol>\n<p>I focused on making sure that this flow would work without forking Chrome and/or Firefox to add this functionality.</p>\n<p>Armed with lots of patience and a healthy dose of stubbornness, I ventured my way into the Mozilla MDN Web Docs in search of inspiration and guidance.</p>\n<p>I’m a little rusty<sup id=\"fnref:2\"><a href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\">2</a></sup> on web technologies, but a quick search opened my mind about <a href=\"https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API/Using_Service_Workers\">Service Workers</a>: <strong>you can intercept <code>fetch</code> calls with them</strong>!</p>\n<p>A Service Worker-based solution for the flow I outlined earlier looks like this:\n0. Install the Service Worker (<code>SW</code>) as soon as possible, blocking everything else in the meantime.</p>\n<ol>\n<li>Browser does a <code>fetch</code>.</li>\n<li><code>SW</code> intercepts the <code>fetch</code> call.</li>\n<li>Check if the request URL’s first path component is <code>/at</code>, if not returns without replacing the request content.</li>\n<li>If yes, do as explained earlier.</li>\n</ol>\n<p>I wanted to keep the browser experience as seamless as possible: clicking on a link that points to an AT URI must yield the same outcome as clicking on an HTTP URL.</p>\n<p>You still need to host an <code>index.html</code> page that loads the Service Worker, but once that’s installed, you can browse AT URIs seamlessly.</p>\n<p>If you noticed, <a href=\"https://geesawra.industries\">geesawra.industries</a> shows a loading page first, then redirects the user to an AT URI which contains the equivalent of an <code>index.html</code>.</p>\n<h4 id=\"a-quick-rust-detour\">A quick Rust detour</h4>\n<p>I’m in a phase of my life in which I’m seeking change, and as a lifelong Gopher I figured “change” in this context means <strong>Rust</strong>: I decided that this thing should be written in it, compiling down to a WebAssembly blob because I don’t know JavaScript.</p>\n<p>On top of that, at the moment Go WASM binary size doesn’t scale down well as Rust’s, it also made sense from a technological perspective: the total binary size is roughly 104KB uncompressed, ~40KB compressed!</p>\n<p>To achieve this result, I had to:</p>\n<ul>\n<li>Write a small ATProto abstraction, focusing on bringing along just what’s needed for <code>atpage</code> to work.</li>\n<li>Don’t throw around as many <code>#[derive(Debug)]</code> as I usually do.</li>\n<li>Use <a href=\"https://docs.rs/serde_json/latest/serde_json/value/enum.Value.html\"><code>serde_json::Value</code></a> to navigate ATProto <code>JSON</code> responses rather than <code>derive</code>-ing serializers/deserializers at compile time.</li>\n<li>Leverage the browser’s <code>fetch()</code> method and <code>Request</code> object through the magical <a href=\"https://docs.rs/web-sys/latest/web_sys/\"><code>web_sys</code></a> crate instead of bringing in dependencies like <a href=\"https://docs.rs/reqwest/latest/reqwest/\"><code>reqwest</code></a>.</li>\n</ul>\n<p>Tooling like <a href=\"https://github.com/rustwasm/wasm-bindgen\"><code>wasm-bindgen</code></a> are a godsend: incredibly powerful and very well documented.</p>\n<p>I had to <a href=\"https://bsky.app/profile/geesawra.industries/post/3ld57lg6jek2b\">jump through</a> <a href=\"https://github.com/geesawra/atpage/commit/f897ec00de6cdcabc0b3445d4e1d9ceb22d5c829\">some hoops</a> to make <code>atpage</code> work on Firefox as well since it <a href=\"https://bugzilla.mozilla.org/show_bug.cgi?id=1360870\">lacks support</a> for ES2015 JavaScript modules, but in the end I was able to implement it thanks to helpful folks on the Rust Discord server.</p>\n<h3 id=\"publishing-tool\">Publishing tool</h3>\n<p>Client rendering is done, but how do I publish my website?</p>\n<p><a href=\"https://github.com/geesawra/atpage/tree/main/publish\"><code>publish</code></a> is a small CLI tool that publishes the content of a directory, the <code>source</code> directory, on the specified PDS for the logged-in user.</p>\n<p>Publishing is a two-step process:</p>\n<ol>\n<li>Find all HTML files.</li>\n<li>Find all tags with an <code>src</code> or <code>href</code> attribute.</li>\n</ol>\n<p>To do that, <code>publish</code> recursively explores the <code>source</code> directory looking for HTML files.</p>\n<p>For example, let’s say your website <code>source</code> directory is <code>./my-website</code> and it looks like this:</p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-bash\" data-lang=\"bash\"><span style=\"display:flex;\"><span>$ tree ./my-website/\n</span></span><span style=\"display:flex;\"><span>./my-website/\n</span></span><span style=\"display:flex;\"><span>├── dog.jpg\n</span></span><span style=\"display:flex;\"><span>└── index.html\n</span></span></code></pre></div><p>and the <code>index.html</code> file contains an image like this:</p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-html\" data-lang=\"html\"><span style=\"display:flex;\"><span><<span style=\"color:#f92672\">html</span>>\n</span></span><span style=\"display:flex;\"><span>\t<<span style=\"color:#f92672\">body</span>>\n</span></span><span style=\"display:flex;\"><span>\t\t<<span style=\"color:#f92672\">h1</span>>Hello!</<span style=\"color:#f92672\">h1</span>>\n</span></span><span style=\"display:flex;\"><span>\t\t<<span style=\"color:#f92672\">img</span> <span style=\"color:#a6e22e\">src</span><span style=\"color:#f92672\">=</span><span style=\"color:#e6db74\">\"/dog.jpg\"</span>>\n</span></span><span style=\"display:flex;\"><span>\t</<span style=\"color:#f92672\">body</span>>\n</span></span><span style=\"display:flex;\"><span></<span style=\"color:#f92672\">html</span>>\n</span></span></code></pre></div><p>During step 1, <code>publish</code> will find the <code>index.html</code> file, then during step 2 it will look for <code>./my-website/dog.jpg</code>: any resource that’s missing on disk will stop the process.</p>\n<p>It also provides a handy <code>nuke</code> command, whenever you feel a lil crazy and want to get rid of your PDS-hosted website.</p>\n<h3 id=\"conclusions\">Conclusions</h3>\n<p><code>atpage</code> has been an interesting experiment, it allowed me to space out of my comfort zone and explore ideas “just because”: I should do that more often.</p>\n<p>Sure, there are still some rough corners here and there, but where’s the fun without a little uncertainty?</p>\n<p>I think it’s also important to highlight the limitations that this approach imposes on the users: without a JavaScript-enabled browser, it’s impossible to browse a website powered by <code>atpage</code>.</p>\n<p>It must also be said that’s requiring a decentralized network, a Service Worker, WebAssembly and the Rust programming language to look at what would otherwise be a statically-generated website is over-engineered, silly and most probably goofy as well.</p>\n<p>That said, tread lightly, and have fun hosting websites on your PDSes!</p>\n<h3 id=\"update-1-16-dec-2024\">Update 1 (16 Dec, 2024)</h3>\n<p>Right after posting a link to this post, it occurred to me that in order to load it you first need to navigate to the root of the website, have the Service Worker install itself and then navigate back to the original link: too inconvenient!</p>\n<p>As a workaround, I pushed a <a href=\"https://github.com/geesawra/atpage/commit/30e66b37178f1d71a3740236a21e5ccae45fc9fe\">hot-fix</a> and modified my Caddy configuration:</p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-diff\" data-lang=\"diff\"><span style=\"display:flex;\"><span>geesawra.industries {\n</span></span><span style=\"display:flex;\"><span><span style=\"color:#a6e22e\">+\tredir /at/* /?redir={uri} permanent\n</span></span></span><span style=\"display:flex;\"><span><span style=\"color:#a6e22e\"></span>\troot * /geesawra.industries\n</span></span><span style=\"display:flex;\"><span>\tfile_server\n</span></span><span style=\"display:flex;\"><span>}\n</span></span></code></pre></div><p>If the Service Worker is installed already, navigating to an <code>/at/{AT URI}</code> page will go through it, but if it isn’t, redirect it to <code>/</code>, allow for the initial setup to happen and then redirect to the <code>redir</code> value, if and only if <code>uri</code> begins with <code>/at/</code>.</p>\n<h3 id=\"update-2-19-dec-2024\">Update 2 (19 Dec, 2024)</h3>\n<p>Instead of relying on redirects, a better fix for the issue described in <strong>Update 1</strong> is to handle all <code>/at/</code> URLs with the same code, and distinguish between index or sub-page navigation by checking <code>window.location.pathname</code>.</p>\n<p><a href=\"https://github.com/geesawra/atpage/commit/4bb871f036e1756250d6a52b530ddfb8f5e58d54\">This commit</a> implements the described change.</p>\n<p>The Caddy configuration looks like this:</p>\n<div class=\"highlight\"><pre tabindex=\"0\" style=\"color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;\"><code class=\"language-diff\" data-lang=\"diff\"><span style=\"display:flex;\"><span>geesawra.industries {\n</span></span><span style=\"display:flex;\"><span><span style=\"color:#f92672\">- redir /at/* /?redir={uri} permanent\n</span></span></span><span style=\"display:flex;\"><span><span style=\"color:#f92672\"></span><span style=\"color:#a6e22e\">+ handle /at/* {\n</span></span></span><span style=\"display:flex;\"><span><span style=\"color:#a6e22e\">+ root * /geesawra.industries\n</span></span></span><span style=\"display:flex;\"><span><span style=\"color:#a6e22e\">+ }\n</span></span></span><span style=\"display:flex;\"><span><span style=\"color:#a6e22e\"></span>\troot * /geesawra.industries\n</span></span><span style=\"display:flex;\"><span>\tfile_server\n</span></span><span style=\"display:flex;\"><span>}\n</span></span></code></pre></div><div class=\"footnotes\" role=\"doc-endnotes\">\n<hr>\n<ol>\n<li id=\"fn:1\">\n<p>Handles can change, so it is best to use DIDs in AT URIs. <a href=\"#fnref:1\" class=\"footnote-backref\" role=\"doc-backlink\">↩︎</a></p>\n</li>\n<li id=\"fn:2\">\n<p>I.e. I know basically zero. <a href=\"#fnref:2\" class=\"footnote-backref\" role=\"doc-backlink\">↩︎</a></p>\n</li>\n</ol>\n</div>\n\n</main>\n\n <footer>\n \n \n <hr>\n © <a href=\"https://geesawra.industries\">geesawra</a> 2024 – 2025 | <a href=\"https://github.com/geesawra\">Github</a> | <a href=\"https://bsky.app/profile/geesawra.industries\">Bluesky</a>\n \n </footer>\n \n\n\n</body></html>"
}
}