Experimental browser for the Atmosphere
Unveiling the true inspiration behind the ‘attention’ operator in Transformers! From Bahdanau’s emails to Karpathy, we learn how attention transformed neural networks. Surprising that ‘Attention is All You Need’ outshines its predecessor by Bahdanau et al. #AI #ML #Transformers
Dec 3, 2024, 8:06 PM
{
"text": "Unveiling the true inspiration behind the ‘attention’ operator in Transformers! From Bahdanau’s emails to Karpathy, we learn how attention transformed neural networks. Surprising that ‘Attention is All You Need’ outshines its predecessor by Bahdanau et al. #AI #ML #Transformers",
"$type": "app.bsky.feed.post",
"embed": {
"$type": "app.bsky.embed.images",
"images": [
{
"alt": "",
"image": {
"$type": "blob",
"ref": {
"$link": "bafkreibxn2vt3omvlqv7ndv6u2dfpowbmodocqodywkusrt3i2dvrepotm"
},
"mimeType": "image/jpeg",
"size": 839571
},
"aspectRatio": {
"width": 2000,
"height": 1547
}
}
]
},
"langs": [
"en"
],
"facets": [
{
"index": {
"byteEnd": 270,
"byteStart": 267
},
"features": [
{
"tag": "AI",
"$type": "app.bsky.richtext.facet#tag"
}
]
},
{
"index": {
"byteEnd": 274,
"byteStart": 271
},
"features": [
{
"tag": "ML",
"$type": "app.bsky.richtext.facet#tag"
}
]
},
{
"index": {
"byteEnd": 288,
"byteStart": 275
},
"features": [
{
"tag": "Transformers",
"$type": "app.bsky.richtext.facet#tag"
}
]
}
],
"createdAt": "2024-12-03T20:06:19.539Z"
}