cross-posted from: https://lemmy.intai.tech/post/5502
cross-posted from: https://lemmy.intai.tech/post/5501
You must log in or register to comment.
I was going to ask which are transformer based but it’s actually nearly all of them (non grey) 😳. I thought BERT was still pre transformers. Goes to show how quickly things have evolved in recent years.
Can someone explain Decoder only vs Encoder/Decoder?
Companies that open source their code, you love to see it.
“open” ai…
Also of note is RWKV, the only purely RNN structured LLM. I’m keeping an eye on that one because of the theoretically infinite context length