Machine Learning

Further reading Mixture of experts explained - Hugging face What is mixture of experts? - IBM Mixture of experts (MoE) - Wikipedia Related pages DBRX

The three main branches of machine learning are: Supervised Unsupervised Reinforcement learning Further reading Related pages Grokking Deep Reinforcement Learning, by Miguel Morales

The Transformer is a deep learning architecture introduced in the 2017 paper Attention Is All You Need by Vaswani et al. It serves as the foundation for modern large language models, including GPT, Claude, and Gemini, as well as vision and audio systems. Key Concepts Self-Attention Mechanism: Traditional architectures like Recurrent Neural Networks (RNNs) processed words one by one, making it difficult to remember long-range context. Transformers evaluate an entire sequence simultaneously, assigning dynamic attention weights to capture relationships between words regardless of distance. Massive Parallelisation: Because the model does not depend on sequential step-by-step recurrence, training scales efficiently across thousands of GPUs. Positional Encodings: Since tokens are ingested concurrently rather than in order, mathematical position vectors are added to token embeddings so the model retains word order. Encoder-Decoder Design: The original design paired an Encoder (to analyse input sequences) with a Decoder (to generate output sequences). Modern variants often use decoder-only (e.g., GPT-style causal language models) or encoder-only (e.g., BERT-style representations) setups depending on the objective. Further reading Transformer - Wikipedia Attention Is All You Need - Wikipedia

The Turing Test, proposed by Alan Turing in 1950, is a measure of a machine’s ability to present intelligent behavior that indistinguishable from that of a human.

Model Context Protocol (MCP) is an open standard developed by Anthropic that defines how AI models share structured context - including messages, observations and actions - in a predictable and machine-readable way. Released in early 2024, MCP has been described as an “API for AI models”. However, describing it as just an API underplays its fundamental purpose: to provide a common, structured language for inter-model communication, which goes beyond a simple API call and introduces new considerations for systems design and governance. ...