Internals

packets on the wire

All packets have a common header:

typedef struct {
  __u32       magic;
  __u16       command;
  __u16       length;
} Drbd_Packet;

The command fields decides the nature of the packet, the length field holds the length of the data part of the packet, which is 0 for all other packets than data packets.

The data packet ships a block over the wire:

typedef struct {
  __u64       block_nr;
  __u64       block_id;
} Drbd_Data_P;

This header is followed by the actual data.
block_nr The linear address of the block.
block_id If this is ID_SYNCER, this this block belongs to the synchronisation process, otherwise these bits should be sent back with the acknowledge packet. The sender stores here the pointer to the request, so it can find the request when it receives an acknowledgement packet for the block.

This is a write barrier packet.

typedef struct {
  __u32       barrier;
  __u32       _fill;
} Drbd_Barrier_P;

barrier Is used to ship a number around. This number is used to identify the barrier.

execution Contexts (Threads)

At first I want to introduce you into the different execution contexts, that are used in DRBD:

The process context of a process that wishes to write a block. Quite often this is the kflushd process, but also basically any process writing to a file can be this process.
The drbdd_X kernel thread. This thread listens on the TCP socket and processes incomming packets.
The drbd_asender_X kernel thread is a used to send packets via the TCP socket, if it is not allowed to sleep in the current context (e.g. an interrupt hander).
The drbd_syncer_X kernel thread is sending out the packets of a resynchronisation process.

data structures

The transfer_log is a circular buffer, which holds references to recently sent blocks and write barriers. The blocks and write barriers are put into the trasfer_log in the same order as they are sent over the socket.
The transfer_log is used to analyse the dependencies between blocks, and to decide if it is neccessary to issue a write barrier. If a write-barrier-acknowledge packet is received the oldest epch set is released from the transfer_log.

The epoch_set is used on the receiver's side to hold references to the blocks of the current epoch set. When a block is received, an entry is put into the epoch_set. Entries are remove from the epoch set when the according acknowledge packet is sent back. On receiption of a write-barrier packet, the drbdd thread must wait until all blocks of the current epoch_set are on the disk, before it may continue to receive further data blocks.

The sync_log is quite simmilar to the epoch_set, but only blocks sent by the drbd_syncer_X thread are going into the sync_log.

back

Philipp Reisner

Last modified: Sat Sep 2 15:42:17 CEST 2000