ATProto Browser

Record data

{
  "uri": "at://did:plc:by3jhwdqgbtrcc7q4tkkv3cf/com.whtwnd.blog.entry/3laega7icmi2q",
  "cid": "bafyreiehp4ff643hvevz4rku3g3hkk4ldqebnwdbp2eobvpuzl6ygg5k2a",
  "value": {
    "$type": "com.whtwnd.blog.entry",
    "theme": "github-light",
    "title": "How to self-host all of Bluesky except the AppView (for now)",
    "content": "Did you know? You can self-host and/or mirror almost all of Bluesky's infrastructure today!\n\n> This article assumes familiarity with the architecture of Bluesky, general *nix knowledge and basic knowledge of TypeScript, Go and Rust toolchains\n\n## PDS\n\nThe most obvious one: having your own [PDS](https://docs.bsky.app/docs/advanced-guides/federation-architecture#personal-data-server-pds) and owning your data. You can find the GitHub repository with instructions [here](https://github.com/bluesky-social/pds?tab=readme-ov-file#self-hosting-pds); you can migrate your existing account with the [GOAT tool](https://github.com/bluesky-social/indigo/tree/main/cmd/goat), for which [Bryan Newbold](https://bsky.app/profile/bnewbold.net) has written an [excellent howto post](https://whtwnd.com/bnewbold.net/entries/Migrating%20PDS%20Account%20with%20%60goat%60).\n\n## Relay\n\nBryan has a great blog post on how to set up your own [Relay](https://docs.bsky.app/docs/advanced-guides/federation-architecture#relay) [right here](https://whtwnd.com/bnewbold.net/entries/Notes%20on%20Running%20a%20Full-Network%20atproto%20Relay%20(July%202024)). Things have grown since, so make sure you have at least ~4.5 TB of disk space (maybe more). Two important comments, not in the post: make sure to use the `--disk-persister-dir=/data/events` flag as well as [enable compaction](https://github.com/bluesky-social/indigo/blob/d1686e9b80835948407fd7ad2428e92379298dac/cmd/bigsky/main.go#L136-L141).\n\n## Jetstream\n\n[Jetstream](https://github.com/bluesky-social/jetstream) is like having your own Relay firehose, but uses a fraction of the bandwidth, storage, and gives you friendly JSON instead of [CBOR](https://en.wikipedia.org/wiki/CBOR)-encoded [MST](https://en.wikipedia.org/wiki/Merkle_tree) blocks. After cloning the GitHub repository, you can either use the included `docker-compose.yaml` with `make up` or build it directly with `make build` and use your favorite keep-this-process-running-pretty-please tool (systemd, pm2 etc.) Don't forget to take a look at the [available CLI arguments/env variables](https://github.com/bluesky-social/jetstream/blob/0ab10bd041fe1fdf682d3964b20d944905c4862d/cmd/jetstream/main.go#L36-L103), which let you do things like change the default retention (24 hours) or override the Firehose cursor and backfill it with 3 days of data—which is what's available from the official relays.\n\n\n## plc.directory mirror\n\nTo get a mirror of [plc.directory](https://plc.directory), with almost all users on Bluesky (they [do support did:web](https://atproto.com/specs/did#blessed-did-methods), but they are few and far between), clone [str4d](https://bsky.app/profile/str4d.xyz)'s [`plc` repo](https://github.com/str4d/plc), switch to the `mirror` branch, run `cargo run --features mirror -- mirror run mirror.db`, then sit back and wait 5-10 hours. Once it's done and up-to-date, you have a full replica in a neat sqlite3 DB. You can monitor its status with something like `watch -n60 'sqlite3 mirror.db \"select count(*) from identity;\"'` in a [tmux](https://github.com/tmux/tmux/wiki) pane.\n\n## The official web/mobile app, also known as `social-app`\n\nClone the [repo](https://github.com/bluesky-social/social-app), run `yarn && yarn web`, and you have a fully-functional copy of it in mere minutes that you can modify to your heart's desire. Doing this on mobile is a lot more involved and [documented here](https://github.com/bluesky-social/social-app/blob/main/docs/build.md#iosandroid-build).\n\n## AppView\n\nThe elephant in the room is, of course, running your own Bluesky [AppView](https://atproto.com/guides/glossary#app-view). If you're interested, DM me on Bluesky to join my working group to make it happen!",
    "createdAt": "2024-11-08T18:15:30.976Z",
    "visibility": "public"
  }
}