AROUND SOME TRANSFORMER ARCHITECTURES

---

Start: 23.04.2025 @ 18:00

---

We invite you to the next PyData meetup. Lecture on AROUND SOME TRANSFORMER ARCHITECTURES will be held by ** Marko Karbevski, M.Sc.**. The lecture will take place at Base42 on Wednesday 23.4.2025 at 18:00.

In this presentation, Marko will explore key Transformer architectures. First, we’ll discuss the decoder-only Transformer, which is central to autoregressive models like GPT and widely used in natural language processing.

Next, he’ll present the Mixture-of-Experts (MoE) architecture, which introduces sparse activation to enhance scalability and efficiency by using only a subset of experts during training. This allows for large-scale models with lower computational cost.

Finally, if time allows, he will talk about Visual Transformers, which adapt the Transformer model for computer vision tasks by treating image patches as sequences, offering an alternative to traditional Convolutional Neural Networks (CNNs).

Call for presenters

---

Do you enjoy sharing knowledge and like public speaking? Do you just enjoy sharing knowledge but are unsure how you feel about public speaking? Even if you're not 100% about public speaking, PyData is a very welcoming community and we appreciate any talks about sharing knowledge given by anyone passionate enough to share them.

Sign up to speak on the next PyData:

./speak.sh