Experimental browser for the Atmosphere
To no one in particular: If you're looking to adopt a reasonable approach to robots.txt for a platform, look to this guide from 2015. Simple and sets expectations for what this (now) standard can and can't do: https://www.dreamwidth.org/support/faqbrowse?faqid=113
Mar 25, 2025, 10:46 AM
{ "cid": "bafyreihxhfi75qbdl3vni7nd6uto26n7dtpbzu7cualfvo6ppacgmavv44", "uri": "at://did:plc:sxkhphi7q7yejv7qiniyznnc/app.bsky.feed.post/3ll73rzkprh62", "value": { "$type": "app.bsky.feed.post", "bridgyOriginalText": "<p>To no one in particular: If you're looking to adopt a reasonable approach to robots.txt for a platform, look to this guide from 2015. Simple and sets expectations for what this (now) standard can and can't do: <a href=\"https://www.dreamwidth.org/support/faqbrowse?faqid=113\" target=\"_blank\" rel=\"nofollow noopener noreferrer\" translate=\"no\"><span class=\"invisible\">https://www.</span><span class=\"ellipsis\">dreamwidth.org/support/faqbrow</span><span class=\"invisible\">se?faqid=113</span></a></p>", "bridgyOriginalUrl": "https://techpolicy.social/@mallory/114222668451620092", "createdAt": "2025-03-25T10:46:44.000Z", "embed": { "$type": "app.bsky.embed.external", "external": { "$type": "app.bsky.embed.external#external", "description": "", "title": "Captcha Check", "uri": "https://www.dreamwidth.org/captcha?returnto=https://www.dreamwidth.org/support/faqbrowse%3Ffaqid%3D113" } }, "facets": [ { "$type": "app.bsky.richtext.facet", "features": [ { "$type": "app.bsky.richtext.facet#link", "uri": "https://www.dreamwidth.org/support/faqbrowse?faqid=113" } ], "index": { "byteEnd": 264, "byteStart": 210 } } ], "langs": [ "en" ], "text": "To no one in particular: If you're looking to adopt a reasonable approach to robots.txt for a platform, look to this guide from 2015. Simple and sets expectations for what this (now) standard can and can't do: https://www.dreamwidth.org/support/faqbrowse?faqid=113" } }