ATProto Browser

ATProto Browser

Experimental browser for the Atmosphere

Post

Unveiling the true inspiration behind the ‘attention’ operator in Transformers! From Bahdanau’s emails to Karpathy, we learn how attention transformed neural networks. Surprising that ‘Attention is All You Need’ outshines its predecessor by Bahdanau et al. #AI #ML #Transformers

Dec 3, 2024, 8:06 PM

{
  "text": "Unveiling the true inspiration behind the ‘attention’ operator in Transformers! From Bahdanau’s emails to Karpathy, we learn how attention transformed neural networks. Surprising that ‘Attention is All You Need’ outshines its predecessor by Bahdanau et al. #AI #ML #Transformers",
  "$type": "app.bsky.feed.post",
  "embed": {
    "$type": "app.bsky.embed.images",
    "images": [
      {
        "alt": "",
        "image": {
          "$type": "blob",
          "ref": {
            "$link": "bafkreibxn2vt3omvlqv7ndv6u2dfpowbmodocqodywkusrt3i2dvrepotm"
          },
          "mimeType": "image/jpeg",
          "size": 839571
        },
        "aspectRatio": {
          "width": 2000,
          "height": 1547
        }
      }
    ]
  },
  "langs": [
    "en"
  ],
  "facets": [
    {
      "index": {
        "byteEnd": 270,
        "byteStart": 267
      },
      "features": [
        {
          "tag": "AI",
          "$type": "app.bsky.richtext.facet#tag"
        }
      ]
    },
    {
      "index": {
        "byteEnd": 274,
        "byteStart": 271
      },
      "features": [
        {
          "tag": "ML",
          "$type": "app.bsky.richtext.facet#tag"
        }
      ]
    },
    {
      "index": {
        "byteEnd": 288,
        "byteStart": 275
      },
      "features": [
        {
          "tag": "Transformers",
          "$type": "app.bsky.richtext.facet#tag"
        }
      ]
    }
  ],
  "createdAt": "2024-12-03T20:06:19.539Z"
}