x-jeff blog

Make progress every day.

【从零开始构建大语言模型】【5】【Pretraining on unlabeled data】

Evaluating generative text models，Training an LLM，Decoding strategies to control randomness，Loading and saving model weights in PyTorch，Loading pretrained weights from OpenAI

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。 原书链接：Build a Large Language Model (From Scratch)。 官方示例代码：LLMs-from-scratch。 本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on March 13, 2025

【从零开始构建大语言模型】【4】【Implementing a GPT model from scratch to generate text】

Coding an LLM architecture，Normalizing activations with layer normalization，Implementing a feed forward network with GELU activations，Adding shortcut connections，Connecting attention and linear layers in a transformer block，Coding the GPT model，Generating text

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。 原书链接：Build a Large Language Model (From Scratch)。 官方示例代码：LLMs-from-scratch。 本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on March 3, 2025

【机器学习基础】第五十八课：[半监督学习]半监督SVM

S3VM

【机器学习基础】系列博客为参考周志华老师的《机器学习》一书，自己所做的读书笔记。 本文为原创文章，未经本人允许，禁止转载。转载请注明出处。 1.半监督SVM 半监督支持向量机（Semi-Supervised Support Vector Machine，简称S3VM）是支持向量机在半监督学习上的推广。在不考虑未标记样本时，支持向量机试图找到最大间隔划分超平面，而在考虑未标记样本后...

Posted by x-jeff on March 1, 2025

【从零开始构建大语言模型】【3】【Coding attention mechanisms】

The problem with modeling long sequences，Capturing data dependencies with attention mechanisms，Attending to different parts of the input with self-attention，Implementing self-attention with trainable weights，Hiding future words with causal attention，Extending single-head attention to multi-head attention

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。 原书链接：Build a Large Language Model (From Scratch)。 官方示例代码：LLMs-from-scratch。 本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on February 21, 2025

【从零开始构建大语言模型】【2】【Working with text data】

Understanding word embeddings，Tokenizing text，Converting tokens into token IDs，Adding special context tokens，Byte pair encoding，Data sampling with a sliding window，Creating token embeddings，Encoding word positions

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。 原书链接：Build a Large Language Model (From Scratch)。 官方示例代码：LLMs-from-scratch。 本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on February 19, 2025

【从零开始构建大语言模型】【1】【Understanding large language models】

What is an LLM?，Applications of LLMs，Stages of building and using LLMs，Introducing the transformer architecture，Utilizing large datasets，A closer look at the GPT architecture，Building a large language model

【从零开始构建大语言模型】系列博客为”Build a Large Language Model (From Scratch)”一书的个人读书笔记。 原书链接：Build a Large Language Model (From Scratch)。 官方示例代码：LLMs-from-scratch。 本文为原创文章，未经本人允许，禁止转载。转载请注明出...

Posted by x-jeff on February 18, 2025

【机器学习基础】第五十七课：[半监督学习]生成式方法

生成式方法

【机器学习基础】系列博客为参考周志华老师的《机器学习》一书，自己所做的读书笔记。 本文为原创文章，未经本人允许，禁止转载。转载请注明出处。 1.生成式方法 本章节没太理解，在此仅作记录，相关公式的详细推导可参考南瓜书PumpkinBook。 生成式方法（generative methods）是直接基于生成式模型的方法。此类方法假设所有数据（无论是否有标记）都是由同一个潜...

Posted by x-jeff on February 17, 2025

【CUDA编程】【30】【6.C++ Language Extensions】【Part5】

Warp Reduce Functions，Warp Shuffle Functions，Nanosleep Function，Warp Matrix Functions，DPX

【CUDA编程】系列博客参考NVIDIA官方文档“CUDA C++ Programming Guide（v12.6）”。 本文为原创文章，未经本人允许，禁止转载。转载请注明出处。 1.Warp Reduce Functions __reduce_sync(unsigned mask, T value)内置函数在同步由mask指定的线程后，对提供的value数据执行归约操作（red...

Posted by x-jeff on February 17, 2025

【CUDA编程】【29】【6.C++ Language Extensions】【Part4】

Address Space Conversion Functions，Alloca Function，Compiler Optimization Hint Functions，Warp Vote Functions，Warp Match Functions

【CUDA编程】系列博客参考NVIDIA官方文档“CUDA C++ Programming Guide（v12.6）”。 本文为原创文章，未经本人允许，禁止转载。转载请注明出处。 1.Address Space Conversion Functions 本部分都是用于地址空间转换的函数。一些预备知识点： CUDA编程中，通用地址是一种抽象的地址形式，可能指向device内...

Posted by x-jeff on January 23, 2025

【CUDA编程】【28】【6.C++ Language Extensions】【Part3】

Load Functions Using Cache Hints，Store Functions Using Cache Hints，Time Function，Atomic Functions，Address Space Predicate Functions

【CUDA编程】系列博客参考NVIDIA官方文档“CUDA C++ Programming Guide（v12.6）”。 本文为原创文章，未经本人允许，禁止转载。转载请注明出处。 1.Load Functions Using Cache Hints 这些加载函数仅支持计算能力在5.0及以上的device。 1 2 3 4 5 T __ldcg(const T* address)...

Posted by x-jeff on January 17, 2025

FEATURED TAGS
README Machine Learning Series Mathematics Series Image Segmentation Python Series Fundamentals of Computers OpenCV Series C++ Series Deep Learning Series Conda Tensorflow Series Algorithms Series AI Papers Object Recognition Natural Language Processing Aha! Algorithms Camera Calibration How Program Works Optical Flow Estimation How Networks Work Computer Vision Source Code CUDA C++ Programming Guide C++ Concurrency IN ACTION Large Language Models
ABOUT ME

Aspire to be an excellent artificial intelligence algorithm engineer!

✉️jeff.xinsc@gmail.com