welcome to x-jeff blog

【从零开始构建大语言模型】【7】【Fine-tuning to follow instructions】

Introduction to instruction fine-tuning，Preparing a dataset for supervised instruction fine-tuning，Organizing data into training batches，Creating data loaders for an instruction dataset，Loading a pretrained LLM，Fine-tuning the LLM on instruction data，Extracting and saving responses，Evaluating the fine-tuned LLM

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。原书链接：Build a Large Language Model (From Scratch)。官方示例代码：LLMs-from-scratch。本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on March 31, 2025

【从零开始构建大语言模型】【6】【Fine-tuning for classification】

Different categories of fine-tuning，Preparing the dataset，Creating data loaders，Initializing a model with pretrained weights，Adding a classification head，Calculating the classification loss and accuracy，Fine-tuning the model on supervised data，Using the LLM as a spam classifier

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。原书链接：Build a Large Language Model (From Scratch)。官方示例代码：LLMs-from-scratch。本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on March 21, 2025

【从零开始构建大语言模型】【5】【Pretraining on unlabeled data】

Evaluating generative text models，Training an LLM，Decoding strategies to control randomness，Loading and saving model weights in PyTorch，Loading pretrained weights from OpenAI

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。原书链接：Build a Large Language Model (From Scratch)。官方示例代码：LLMs-from-scratch。本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on March 13, 2025

【从零开始构建大语言模型】【4】【Implementing a GPT model from scratch to generate text】

Coding an LLM architecture，Normalizing activations with layer normalization，Implementing a feed forward network with GELU activations，Adding shortcut connections，Connecting attention and linear layers in a transformer block，Coding the GPT model，Generating text

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。原书链接：Build a Large Language Model (From Scratch)。官方示例代码：LLMs-from-scratch。本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on March 3, 2025

【机器学习基础】第五十八课：[半监督学习]半监督SVM

S3VM

【机器学习基础】系列博客为参考周志华老师的《机器学习》一书，自己所做的读书笔记。本文为原创文章，未经本人允许，禁止转载。转载请注明出处。 1.半监督SVM 半监督支持向量机（Semi-Supervised Support Vector Machine，简称S3VM）是支持向量机在半监督学习上的推广。在不考虑未标记样本时，支持向量机试图找到最大间隔划分超平面，而在考虑未标记样本后...

Posted by x-jeff on March 1, 2025

【从零开始构建大语言模型】【3】【Coding attention mechanisms】

The problem with modeling long sequences，Capturing data dependencies with attention mechanisms，Attending to different parts of the input with self-attention，Implementing self-attention with trainable weights，Hiding future words with causal attention，Extending single-head attention to multi-head attention

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。原书链接：Build a Large Language Model (From Scratch)。官方示例代码：LLMs-from-scratch。本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on February 21, 2025

【从零开始构建大语言模型】【2】【Working with text data】

Understanding word embeddings，Tokenizing text，Converting tokens into token IDs，Adding special context tokens，Byte pair encoding，Data sampling with a sliding window，Creating token embeddings，Encoding word positions

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。原书链接：Build a Large Language Model (From Scratch)。官方示例代码：LLMs-from-scratch。本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on February 19, 2025

【从零开始构建大语言模型】【1】【Understanding large language models】

What is an LLM?，Applications of LLMs，Stages of building and using LLMs，Introducing the transformer architecture，Utilizing large datasets，A closer look at the GPT architecture，Building a large language model

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。原书链接：Build a Large Language Model (From Scratch)。官方示例代码：LLMs-from-scratch。本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on February 18, 2025

【机器学习基础】第五十七课：[半监督学习]生成式方法

生成式方法

【机器学习基础】系列博客为参考周志华老师的《机器学习》一书，自己所做的读书笔记。本文为原创文章，未经本人允许，禁止转载。转载请注明出处。 1.生成式方法本章节没太理解，在此仅作记录，相关公式的详细推导可参考南瓜书PumpkinBook。生成式方法（generative methods）是直接基于生成式模型的方法。此类方法假设所有数据（无论是否有标记）都是由同一个潜...

Posted by x-jeff on February 17, 2025

【CUDA编程】【30】【6.C++ Language Extensions】【Part5】

Warp Reduce Functions，Warp Shuffle Functions，Nanosleep Function，Warp Matrix Functions，DPX

【CUDA编程】系列博客参考NVIDIA官方文档“CUDA C++ Programming Guide（v12.6）”。本文为原创文章，未经本人允许，禁止转载。转载请注明出处。 1.Warp Reduce Functions __reduce_sync(unsigned mask, T value)内置函数在同步由mask指定的线程后，对提供的value数据执行归约操作（red...

Posted by x-jeff on February 17, 2025

x-jeff blog

【从零开始构建大语言模型】【7】【Fine-tuning to follow instructions】

【从零开始构建大语言模型】【6】【Fine-tuning for classification】

Different categories of fine-tuning，Preparing the dataset，Creating data loaders，Initializing a model with pretrained weights，Adding a classification head，Calculating the classification loss and accuracy，Fine-tuning the model on supervised data，Using the LLM as a spam classifier

【从零开始构建大语言模型】【5】【Pretraining on unlabeled data】

Evaluating generative text models，Training an LLM，Decoding strategies to control randomness，Loading and saving model weights in PyTorch，Loading pretrained weights from OpenAI

【从零开始构建大语言模型】【4】【Implementing a GPT model from scratch to generate text】

Coding an LLM architecture，Normalizing activations with layer normalization，Implementing a feed forward network with GELU activations，Adding shortcut connections，Connecting attention and linear layers in a transformer block，Coding the GPT model，Generating text

【机器学习基础】第五十八课：[半监督学习]半监督SVM

S3VM

【从零开始构建大语言模型】【3】【Coding attention mechanisms】

【从零开始构建大语言模型】【2】【Working with text data】

Understanding word embeddings，Tokenizing text，Converting tokens into token IDs，Adding special context tokens，Byte pair encoding，Data sampling with a sliding window，Creating token embeddings，Encoding word positions

【从零开始构建大语言模型】【1】【Understanding large language models】

What is an LLM?，Applications of LLMs，Stages of building and using LLMs，Introducing the transformer architecture，Utilizing large datasets，A closer look at the GPT architecture，Building a large language model

【机器学习基础】第五十七课：[半监督学习]生成式方法

生成式方法

【CUDA编程】【30】【6.C++ Language Extensions】【Part5】

Warp Reduce Functions，Warp Shuffle Functions，Nanosleep Function，Warp Matrix Functions，DPX

FEATURED TAGS

ABOUT ME