Is there a way to disable those fsync() calls? Fortunately, yes: we can use the so-called LD_PRELOAD trick. The above means that the more files a package contains, the more time it will take to install it. As well as flushing the file data, fsync() also flushes the metadata information associated with the file. The call blocks until the device reports that the transfer has been completed. This includes writing through or flushing a disk cache if present. Newer filesystems (like btrfs or ext4) that implement delayed allocation do require those fsync(2)s as they trade data safety for performance, and expect programs to performs those fsync(2)s, but at the same time they have shown very poor performance on the behaviour they require from all applications, or they might end up producing zero-length files on system crashes or abrupt shutdowns.įsync() transfers all modified in-core data of (i.e., modified buffer cache pages for) the file referred to by the file descriptor to the disk device (or another permanent storage device) so that all changed information can be retrieved even if the system crashes or is rebooted. ![]() ![]() To guarantee that the filesystem data is always consistent and safe, dpkg performs fsync(2)s on its database and files unpacked from packages. In this post, I will try to explain how to use eatmydata to speed up some operations by the example of Debian-based images.įirst of all, why are package managers slow? The dpkg FAQ says (the same is true for rpm and probably other package managers): Therefore, if we could hint to the package manager that we don’t care much about the integrity of the data, we could speed up our builds. ![]() However, stability is not very important when you build an image: if the build fails, the system discards the image anyway, and you have to start over. And this perfectly makes sense: if something terrible happens (like a power failure or kernel panic), the system must remain in a usable state. This process is traditionally slow because package managers choose stability over performance. Quite often, one of the most time-taking parts of the build process is the installation of dependencies.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |