To cache or not to cache - that is the D(irect) question

• When doing I/O to a block device - we can still work with the buffer cache
  • From C, this is the normal consequence of writing into a file.
  • From the shell - one can zero-out a device file, using the buffer cache, as follows:

              dd if=/dev/zero of=/dev/sdb bs=1MB
              

    NOTE: be careful - do this on the wrong device - and your data is gone capoot!
  • Note: the above command will start very fast (writing into cache), and then slow down to disk speed.
• But the more common use-case is using the block device to bypass the cache.
  • From C, This is done by using the O_DIRECT flag to the open() system call.
  • In this mode - file-offsets, buffer sizes and buffer addresses must be sector-aligned.
  • From the shell - one can zero-out a device file as follows:

              dd if=/dev/zero of=/dev/sdb bs=1MB oflag=direct
              

    NOTE: be careful - do this on the wrong device - and your data is gone capoot!
• We can see the difference if we run "iostat" while these "dd" commands execute.

Originally written by guy keren