x-jeff blog

Make progress every day.

【机器学习基础】第六十一课：[半监督学习]半监督聚类

约束k均值算法，约束种子k均值算法

【机器学习基础】系列博客为参考周志华老师的《机器学习》一书，自己所做的读书笔记。 本文为原创文章，未经本人允许，禁止转载。转载请注明出处。 1.半监督聚类 聚类是一种典型的无监督学习任务，然而在现实聚类任务中我们往往能获得一些额外的监督信息，于是可通过半监督聚类（semi-supervised clustering）来利用监督信息以获得更好的聚类效果。 聚类任务中获得的监督信息...

Posted by x-jeff on April 13, 2025

【机器学习基础】第六十课：[半监督学习]基于分歧的方法

基于分歧的方法

【机器学习基础】系列博客为参考周志华老师的《机器学习》一书，自己所做的读书笔记。 本文为原创文章，未经本人允许，禁止转载。转载请注明出处。 1.基于分歧的方法 与生成式方法、半监督SVM、图半监督学习等基于单学习器利用未标记数据不同，基于分歧的方法（disagreement-based methods）使用多学习器，而学习器之间的“分歧”（disagreement）对未标记数据的...

Posted by x-jeff on April 12, 2025

【从零开始构建大语言模型】【7】【Fine-tuning to follow instructions】

Introduction to instruction fine-tuning，Preparing a dataset for supervised instruction fine-tuning，Organizing data into training batches，Creating data loaders for an instruction dataset，Loading a pretrained LLM，Fine-tuning the LLM on instruction data，Extracting and saving responses，Evaluating the fine-tuned LLM

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。 原书链接：Build a Large Language Model (From Scratch)。 官方示例代码：LLMs-from-scratch。 本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on March 31, 2025

【从零开始构建大语言模型】【6】【Fine-tuning for classification】

Different categories of fine-tuning，Preparing the dataset，Creating data loaders，Initializing a model with pretrained weights，Adding a classification head，Calculating the classification loss and accuracy，Fine-tuning the model on supervised data，Using the LLM as a spam classifier

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。 原书链接：Build a Large Language Model (From Scratch)。 官方示例代码：LLMs-from-scratch。 本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on March 21, 2025

【从零开始构建大语言模型】【5】【Pretraining on unlabeled data】

Evaluating generative text models，Training an LLM，Decoding strategies to control randomness，Loading and saving model weights in PyTorch，Loading pretrained weights from OpenAI

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。 原书链接：Build a Large Language Model (From Scratch)。 官方示例代码：LLMs-from-scratch。 本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on March 13, 2025

【机器学习基础】第五十九课：[半监督学习]图半监督学习

图半监督学习

【机器学习基础】系列博客为参考周志华老师的《机器学习》一书，自己所做的读书笔记。 本文为原创文章，未经本人允许，禁止转载。转载请注明出处。 1.图半监督学习 本章节没太理解，在此仅作记录，相关公式的详细推导可参考南瓜书PumpkinBook。 给定一个数据集，我们可将其映射为一个图，数据集中每个样本对应于图中一个结点，若两个样本之间的相似度很高（或相关性很强），则对应的...

Posted by x-jeff on March 9, 2025

【从零开始构建大语言模型】【4】【Implementing a GPT model from scratch to generate text】

Coding an LLM architecture，Normalizing activations with layer normalization，Implementing a feed forward network with GELU activations，Adding shortcut connections，Connecting attention and linear layers in a transformer block，Coding the GPT model，Generating text

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。 原书链接：Build a Large Language Model (From Scratch)。 官方示例代码：LLMs-from-scratch。 本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on March 3, 2025

【机器学习基础】第五十八课：[半监督学习]半监督SVM

S3VM

【机器学习基础】系列博客为参考周志华老师的《机器学习》一书，自己所做的读书笔记。 本文为原创文章，未经本人允许，禁止转载。转载请注明出处。 1.半监督SVM 半监督支持向量机（Semi-Supervised Support Vector Machine，简称S3VM）是支持向量机在半监督学习上的推广。在不考虑未标记样本时，支持向量机试图找到最大间隔划分超平面，而在考虑未标记样本后...

Posted by x-jeff on March 1, 2025

【从零开始构建大语言模型】【3】【Coding attention mechanisms】

The problem with modeling long sequences，Capturing data dependencies with attention mechanisms，Attending to different parts of the input with self-attention，Implementing self-attention with trainable weights，Hiding future words with causal attention，Extending single-head attention to multi-head attention

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。 原书链接：Build a Large Language Model (From Scratch)。 官方示例代码：LLMs-from-scratch。 本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on February 21, 2025

【从零开始构建大语言模型】【2】【Working with text data】

Understanding word embeddings，Tokenizing text，Converting tokens into token IDs，Adding special context tokens，Byte pair encoding，Data sampling with a sliding window，Creating token embeddings，Encoding word positions

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。 原书链接：Build a Large Language Model (From Scratch)。 官方示例代码：LLMs-from-scratch。 本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on February 19, 2025

FEATURED TAGS
README Machine Learning Series Mathematics Series Image Segmentation Python Series Fundamentals of Computers OpenCV Series C++ Series Deep Learning Series Conda Tensorflow Series Algorithms Series AI Papers Object Recognition Natural Language Processing Aha! Algorithms Camera Calibration How Program Works Optical Flow Estimation How Networks Work Computer Vision Source Code CUDA C++ Programming Guide C++ Concurrency IN ACTION Large Language Models
ABOUT ME

Aspire to be an excellent artificial intelligence algorithm engineer!

✉️jeff.xinsc@gmail.com