Block Devices

1. Definition

• Block storage devices are accessed in units of blocks (512 bytes, few KBs, etc), while recalling that memory is byte addressable.
• Optimizing Disk I/O with Memory: Caching and Buffering
  • Caching: the Linux page cache stores pages from files in RAM (usually 4KB pages, often larger than device blocks)
  • Buffering: Python (and other) programs might buffer chunks of data to avoid asking Linux too many times for small pieces of data (store small read & writes to large one)
• Importance of Data Format: no good way to ready one column without everything else/

2. Hard Disk Drives (HDDs)

• Steps to Read/Write
  • move head to correct track
  • wait for spinning disk to rotate until data is under head
  • transfer the data
• Layout (Sequential) & Access Time
  • Assign block numbers to platter locations so sequential reads/writes will be fast
  • Programmers should assume random accesses will be slow
• Short Stroking
  • Head moves over platter faster near outside track
  • Only use the outside track on many HHDs
  • Advantages: faster I/O
  • Disadvantages: less space used on each HHDs, total cost increases

3. Solid State Drives (SSDs)

• Characteristics

  • no moving parts
  • inherently parallel
  • SSD internals

    • "block" and "page" have different meanings in this context

    • "page" => unit that we can read or write (couple KBs)

      pages cannot be individually re-written

    • "block" => unit that is erased together (maybe 100s of KBs)

  

• Write:

  • Method I: erase whole block and re-write A, B, and D with new X
    • Disadvantage: slow
  • Method II: write X somewhere else
    • Disadvantage: need extra bookkeeping (in SSD) to know where data is, and need to eventually move things around to reclaim the space wasted by B.
  • Method III: sequentially write whole blocks at one time when possible

4. HDDs vs. SSDs

• Metrics:

  • Capacity: how many bytes can we store?
  • Latency: how long does it take to start transferring data
  • IOPS (I/O operations, of some max size, per second): how many small/random transfers can we do per second
  • Throughput: how many bytes can we transfer per second

• Comparison

  	HDDs	SSDs
  Capacity	+	-
  Latency	-	+ (no moving parts)
  Random IOPS	-	+ (low latency & in parallel)
  Throughput (Sequential)	-	+ (little better)
  Throughput (Random Writes)	-	+ (but block erase is a concern)
  Throughput (Random Reads)	-	+

5. Partitions and RAID

• Block devices can be divided into partitions:

  

• RAID (Redudant Array of Inexpensive Disks) controllers: make multiple devices appear as one:

  

• Redundancy: same data on >1 disk, usually used to avoid data loss when one device dies.