]]>起一篇文章，记录一下笔者的基础设施。包含博客相关服务、服务器、工作流、网络服务、生产力工具、AI 工具和科研工具等。记一次软件源归档导致的 CI 构建问题https://miyamura.top/posts/gitlab-ci-1.html2025-08-23T16:00:00.000Z2026-02-05T20:31:41.244Z 记一次软件源归档导致的 CI 构建问题
E: The repository '[http://deb.debian.org/debian](http://deb.debian.org/debian) buster Release' does not have a Release file. E: The repository '[http://deb.debian.org/debian-security](http://deb.debian.org/debian-security) buster/updates Release' does not have a Release file. E: The repository '[http://deb.debian.org/debian](http://deb.debian.org/debian) buster-updates Release' does not have a Release file.
Because the model is invertible and its Jacobian has unit determinant, the log-likelihood and its gradients can be tractably computed. This architecture imposes some constraints on the functions the network can represent; for instance, it can only represent volume-preserving mappings.
Note that Algorithm 1 is agnostic to the form of the residual functions F and G. The steps which use the Jacobians of these functions are implemented in terms of ordinary backprop, which can be achieved by calling automatic differentiation routines (e.g. tf.gradients or Theano.grad).
A. N. Gomez, M. Ren, R. Urtasun, and R. B. Grosse, “The Reversible Residual Network: Backpropagation Without Storing Activations,” in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2017.
IDM 采用了 SD1.5 的噪声估计器，复用其在 LAION 数据集上预训练的网络权重，并对 U-Net 结构进行剪枝 (we further simplify the U-Net by removing its time embeddings, cross-attention layers, and the final three scales to balance efficiency and performance.) 实际上，这个操作将原始参数规模 (~860M) 降至 116.6M，这使得在 3090 上训练模型成为可能。
DDIM 采样过程一般应说明起始步数和步长，如 T=8 可以是从预训练的噪声表的第 96 步开始采样，每隔 12 步采样。(p.s. 一般从噪声表较低步数开始采样，因为噪声分量更低)。比较奇怪的是，IDM 淡化了这一概念，且直接移除了时间步嵌入，这样 T 就变成了一个纯纯的超参数。
实验部分对去噪步数分析了参数敏感性，如下图所示。
可逆布线
IDM 引入可逆神经网络思想，例如：
Notably, with our two level invertible models, the peak memory use per GPU is suppressed to 14.64 GB, making it manageable not only for A100 but also for other GPUs like 3090 and 4090 (24GB).
TypeError: OpGraph.conv2d() takes from 3 to 7 positional arguments but 8 were given TypeError: TorchCudaBackend._gen_ewise_uni() missing 1 required positional argument: 'x'
这里有输出就不用管，简单忽略即可。另有一些数据类型上的问题：
1
RuntimeError: expected scalar type Float but found BFloat16(Half)
点击直达]]>MAGIS - Memory Optimization via Coordinated Graph Transformation and Scheduling for DNN浅谈非易失内存https://miyamura.top/posts/nvm-page-migration.html2025-04-20T16:00:00.000Z2026-02-05T20:31:41.248Z 非易失内存 总结篇
安装 CUDA 前，首先确保已安装了 Nvidia 驱动，并执行 nvidia-smi 检查当前 Nvidia 驱动支持的最高 CUDA 版本。如果机器上已经有 CUDA，建议切换版本安装，不卸载当前版本 CUDA。
检查 CUDA 安装情况
1 2
whereis cuda # cuda: /usr/lib/cuda /usr/include/cuda
1 2 3
sudo dpkg -l | grep cuda # ii libcuda1:amd64 535.183.01-1~deb12u1 amd64 NVIDIA CUDA Driver Library # ii libcudart11.0:amd64 11.8.89~11.8.0-5~deb12u1 amd64 NVIDIA CUDA Runtime Library
卸载当前 CUDA 命令如下。不建议此方式。
1
sudo dpkg --remove libcuda1:amd64
安装 CUDA。官方网站提供 .deb .run 等安装方式，这里使用 .run
1 2
wget https://developer.download.nvidia.com/compute/cuda/11.6.2/local_installers/cuda_11.6.2_510.47.03_linux.run sudo sh cuda_11.6.2_510.47.03_linux.run
# Do you accept the license terms? [yes|no] # >>> yes # ... # You can undo this by running `conda init --reverse $SHELL`? [yes|no] # [no] >>> no # ... # Thank you for installing Anaconda3!
环境变量设置
在 /etc/profile 中配置 Conda 路径，如下。
1 2 3
# vim /etc/profile export PATH=/root/anaconda3/bin:$PATH $ source /etc/profile
CondaVerificationError: The package for pytorch located at /root/anaconda3/pkgs/pytorch-2.5.1-py3.11_cuda12.1_cudnn9.1.0_0 appears to be corrupted. The path 'lib/python3.11/site-packages/torch/include/ATen/cuda/CUDAApplyUtils.cuh' specified in the package manifest cannot be found.
执行如下命令清理损坏的包
1
(torch) conda clean --packages
交互内容使用默认选项即可。
1 2 3
# Will remove 158 (10.90 GB) package(s). # Proceed ([y]/n)? y # reinstall
In classification, the goal is to assign input data to specific, predefined categories. The output in classification is typically a label or a class from a set of predefined options.
In regression, the goal is to establish a relationship between input variables and the output. The output in regression is a real-valued number that can vary within a range.
In both supervised learning approaches the goal is to find patterns or relationships in the input data so we can accurately predict the desired outcomes. The difference is that classification predicts categorical classes (like spam), while regression predicts continuous numerical values (like age, income, or temperature).
在许多任务中，例如目标检测、图像分割或图像分类，Backbone 会与其他网络组件结合使用，以实现更复杂的功能。例如，在目标检测中，Backbone 提供特征图，这些特征图随后被其他网络层（如 RPN 或 ROI Pooling 层）处理，以完成检测任务。
总之，Backbone 是深度学习模型中的一个重要组成部分，负责特征提取，并为后续的任务提供基础。
图像理解任务
Classification
Object Detection
Semantic Segmentation
Unsupervised & Self-Supervised
While both methods learn from data without human-annotated labels, the primary difference lies in the way they use the data:
Self-supervised learning makes use of the structure within the data to generate its own labels.
Unsupervised learning seeks to uncover hidden patterns or structures within the data itself.
Self-supervised
1 2 3
Input: "The quick brown fox jumps over the _____" Prediction: "lazy" Actual: "lazy"
In this case, the model is learning to predict the next word (“lazy”) based on the input (“The quick brown fox jumps over the”). It’s a self-supervised task because the label for training (the word “lazy”) is part of the data itself.
Unsupervised
For unsupervised learning, let’s consider an example with text clustering. Suppose we have multiple sentences, including multiple instances of our sentence, and we want to group similar sentences together:
1 2 3 4 5 6 7 8 9 10
1. "The quick brown fox jumps over the lazy" 2. "A fast brown fox jumps over the lazy" 3. "The quick brown dog runs under the active" 4. "An energetic brown dog runs under the quick" 5. "The quick brown fox jumps over the lazy" 6. "A swift brown fox leaps over the lazy"
In this case, the model is learning to group similar sentences together without any explicit labels. Note that this is a highly simplified example, and real-world text clustering tasks would involve much more complex datasets and models.
Swap memory, also known as swap space, is a section of a computer’s hard disk or SSD that the operating system (OS) uses to store inactive data from Random Access Memory (RAM). This allows the OS to run even when RAM is full, preventing system slowdowns or crashes.
When the OS exhausts its available RAM, it swaps data between RAM and the swap space. This mechanism, known as swapping, enhances memory management efficiency.
Swap memory is important for systems with restricted RAM or those executing memory-intensive tasks. Without swap memory, these systems are susceptible to crashing when RAM capacity is exceeded.
Map the file into the application’s address space to allow our program to access the contents as if in memory. In the second parameter, we pass the length of the file, requesting Linux to initialize memory with the full file. We also map the file with both READ and WRITE access and also as SHARED allowing other processes to map the same file.
1 2 3 4 5 6 7 8 9 10 11 12 13
/* * Map the file into our address space for read * & write. Use MAP_SHARED so stores are visible * to other programs. */ if ((pmaddr = mmap(NULL, stbuf.st_size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0)) == MAP_FAILED) err(1, "mmap %s", argv[1]);
/* Don't need the fd anymore because the mapping * stays around */ close(fd);
Now we can write data into the file by accessing it like byte-addressable memory.
1
strcpy(pmaddr, "This is new data written to the file");
Finally, we explicitly flush the data back to the storage device. MS_SYNC is set to make msync wait for the writeback to complete before returning.
1 2 3 4 5 6
/* * Simplest way to flush is to call msync(). * The length needs to be rounded up to a 4k page. */ if (msync((void *)pmaddr, 4096, MS_SYNC) < 0) err(1, "msync");
Figure 3-4. Memory-mapped files with storage
P43
First, a portion of the limited kernel memory page cache in main memory is used to store a copy of the file.
持久性内存的直接内存访问是通过文件系统实现的。使用时，管理员账户在 PMem Module 中创建一个文件系统，并将其挂载到 Linux 的文件系统树中。
Listing 3-7. pmem_map_file.c
We use DAX to write a string directly into persistent memory in the code.
This code uses libpmem, one of the persistent memory API libraries included in Linux and Windows. This sample code is portable across both operating system platforms.
The pmem_map_file function handles opening the file and mapping it into our address space. Since the file resides on persistent memory, the operating system maps the persistent memory region directly into the application’s virtual address space.
1 2 3 4 5 6 7
/* Create a pmem file and memory map it. */ if ((pmemaddr = pmem_map_file(argv[1], PMEM_LEN, PMEM_FILE_CREATE, 0666, &mapped_len, &is_pmem)) == NULL) { perror("pmem_map_file"); exit(1); }
Notice that is_pmem should be set if we desire a DAX memory access with Persistent Memory. This is because pmem_map_file can also be used for memory mapping diskbased files through kernel main memory as well as directly mapping persistent memory.
After strcpy() something to the persistent memory region, the persistence should be guaranteed. If the file resides on persistent memory, pmem_persist() makes data flushed through CPU cache levels to the power-failure safe domain and ultimately to persistent memory. If the file resided on disk-based storage, mmap() would be used to flush data to storage.
1 2 3 4 5 6 7 8
/* Flush our string to persistence. */ if (is_pmem) pmem_persist(pmemaddr, sizeof(s)); else pmem_msync(pmemaddr, sizeof(s));
/* Delete the mappings. */ pmem_unmap(pmemaddr, mapped_len);
Finally, we pmem_unmap() the persistent memory region.
Note that we can pass small sizes to persist() (even the size of a small string) instead of requiring flushes at page granularity when using msync().
补充
DAX 包括 fsdax (default) / devdax 等模式.
fsdax: Provides a block-device that can support a DAX-enabled filesystem
devdax: Emits a single character device file (/dev/daxX.Y).
Figure 3-6 shows the complete view of the operating system support that this chapter describes. As we discussed, an application can use persistent memory as a fast SSD, more directly through a persistent memory-aware file system, or mapped directly into the application’s memory space with the DAX option. DAX leverages operating system services for memory-mapped files but takes advantage of the server hardware’s ability to map persistent memory directly into the application’s address space. This avoids the need to move data between main memory and storage.
We introduce Virtual Memory for the following focuses:
Capacity of Physical Memory Devices are not decoupled from address space determined by ISAs.
Memory-access with physical address (PA) only is not a easy task for programmers and compilers.
Poor scalability, not conducive to scheduling.
With virtual memory, programs always think that they have exclusive access to the address space and therefore do not need to consider address restrictions.
Protection
Another purpose of using VM is permissions and protection. Programs’ access to data is isolated because their virtual address spaces are different, even though they appear to be accessing the same address.
Sharing
Program segments and data segments shared by multiple programs do not need to be copied multiple times, and the OS will link them to the same physical address, e.g.,printf(). This saves physical memory device capacity and avoids Memory Bloat.
Address Translation
In this section, we will talk about Paging, the most commonly used implemention of virtual memory model.
In a paging scheme, the virtual address space is divided into pages, while the physical address space is divided into frames, they both represent a data block. The sizes of a page and a frame should be the same ( ? I think these can be decoupled).
Demand Paging
Pages are loaded into physical memory only when they are needed. Suppose that a program is allowed to use a virtual address (called VA0), but before VA0 is actually accessed the addressed, the VA → PA mapping is not established and there is no corresponding PTE. Not until a Page Fault occurs does the OS know it should fetch data from disks and put them into memory devices.
This brings below advantages:
Avoid redundant memory usage, traffic and bandwidth consumption.
Allow the processor to run programs that are larger than the memory device can hold by decoupling physical capacity and memory space.
For example, mapping of 4KB pages is likely to be:
