org.apache.derby.impl.store.replication.master
Class AsynchronousLogShipper

java.lang.Object
  extended by java.lang.Thread
      extended by org.apache.derby.impl.store.replication.master.AsynchronousLogShipper
All Implemented Interfaces:
java.lang.Runnable, LogShipper

public class AsynchronousLogShipper
extends java.lang.Thread
implements LogShipper

Does asynchronous shipping of log records from the master to the slave being replicated to. The implementation does not ship log records as soon as they become available in the log buffer (synchronously), instead it does log shipping in the following two-fold scenarios 1) Periodically (i.e.) at regular intervals of time. 2) when a request is sent from the master controller (force flushing of the log buffer). 3) when a notification is received from the log shipper about a log buffer element becoming full and the load on the log buffer so warrants a ship.


Nested Class Summary
 
Nested classes/interfaces inherited from class java.lang.Thread
java.lang.Thread.State, java.lang.Thread.UncaughtExceptionHandler
 
Field Summary
static int DEFAULT_FORCEFLUSH_TIMEOUT
          The number of millis a call to forceFlush will wait before giving up sending a chunk of log to the slave
private  ReplicationMessage failedChunk
          Store the log chunk that failed during a previous shipping attempt so that it can be re-shipped to the slave.
private  long failedChunkHighestInstant
          The highest log instant in failedChunk
private static int FI_HIGH
          Fill information value indicative of a high load in the log buffer.
private static int FI_LOW
          Fill information value indicative of a low load in the log buffer.
private  java.lang.Object forceFlushSemaphore
          Used to synchronize forceFlush calls
private  long highestShippedInstant
          The highest log instant shipped so far
private  long lastShippingTime
          Will store the time at which the last shipping happened.
private  ReplicationLogBuffer logBuffer
          Replication log buffer that contains the log records that need to be transmitted to the slave.
private  MasterController masterController
          The master controller that initialized this log shipper.
private static long MAX
          If the fill information is less than FI_LOW the log shipper will ship with a MAX ms delay or when a buffer becomes full whichever comes first.
private  long maxShippingInterval
          Minimum interval (in milliseconds) between log shipping.
private static long MIN
          If the fill information (obtained from the log buffer) is less than FI_HIGH but greater than FI_LOW the log shipper will ship with a MIN ms delay.
private  long minShippingInterval
          Minimum interval (in milliseconds) between log shipping.
private  java.lang.Object objLSTSync
          Object used to synchronize on while the log shipper thread is moved into the wait state, or while notifying it.
private  ReplicationLogger repLogger
           
private  long shippingInterval
          Time interval (in milliseconds) at which the log shipping takes place.
private  boolean stopShipping
          Indicates whether a stop shipping request has been sent.
private  ReplicationMessageTransmit transmitter
          Replication message transmitter that is used for the network transmission of the log records retrieved from the log buffer (on the master) to the slave being replicated to.
 
Fields inherited from class java.lang.Thread
MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY
 
Constructor Summary
AsynchronousLogShipper(ReplicationLogBuffer logBuffer, ReplicationMessageTransmit transmitter, MasterController masterController, ReplicationLogger repLogger)
          Constructor initializes the log buffer, the replication message transmitter, the shipping interval and the master controller.
 
Method Summary
private  long calculateSIfromFI()
          Will be used to calculate the shipping interval based on the fill information obtained from the log buffer.
 void flushBuffer()
          Transmits all the log records in the log buffer to the slave.
 void flushedInstance(long latestInstanceFlushedToDisk)
          updates the information about the latest instance of the log record that has been flushed to the disk.
 void forceFlush()
          Transmits a chunk of log record from the log buffer to the slave, used by the master controller when the log buffer is full and some space needs to be freed for further log records.
 long getHighestShippedInstant()
          Get the highest log instant shipped so far
private  void getLogShipperProperties()
          Load relevant system properties: max and min log shipping interval
 void run()
          Ships log records from the log buffer to the slave being replicated to.
private  boolean shipALogChunk()
          Retrieves a chunk of log records, if available, from the log buffer and transmits them to the slave.
 void stopLogShipment()
          Stop shipping log records.
 void workToDo()
          Used to notify the log shipper that a log buffer element is full.
 
Methods inherited from class java.lang.Thread
activeCount, checkAccess, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, toString, yield
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

logBuffer

private final ReplicationLogBuffer logBuffer
Replication log buffer that contains the log records that need to be transmitted to the slave.


transmitter

private ReplicationMessageTransmit transmitter
Replication message transmitter that is used for the network transmission of the log records retrieved from the log buffer (on the master) to the slave being replicated to.


shippingInterval

private long shippingInterval
Time interval (in milliseconds) at which the log shipping takes place.


minShippingInterval

private long minShippingInterval
Minimum interval (in milliseconds) between log shipping. Defaults to MIN, but can be configured using system property derby.replication.minLogShippingInterval

See Also:
MIN

maxShippingInterval

private long maxShippingInterval
Minimum interval (in milliseconds) between log shipping. Defaults to MAX, but can be configured using system property derby.replication.maxLogShippingInterval

See Also:
MAX

lastShippingTime

private long lastShippingTime
Will store the time at which the last shipping happened. Will be used to calculate the interval between the log ships upon receiving a notification from the log buffer.


stopShipping

private volatile boolean stopShipping
Indicates whether a stop shipping request has been sent. true - stop shipping log records false - shipping can continue without interruption.


masterController

private MasterController masterController
The master controller that initialized this log shipper.


objLSTSync

private java.lang.Object objLSTSync
Object used to synchronize on while the log shipper thread is moved into the wait state, or while notifying it.


forceFlushSemaphore

private java.lang.Object forceFlushSemaphore
Used to synchronize forceFlush calls


DEFAULT_FORCEFLUSH_TIMEOUT

public static final int DEFAULT_FORCEFLUSH_TIMEOUT
The number of millis a call to forceFlush will wait before giving up sending a chunk of log to the slave

See Also:
Constant Field Values

failedChunk

private ReplicationMessage failedChunk
Store the log chunk that failed during a previous shipping attempt so that it can be re-shipped to the slave.


failedChunkHighestInstant

private long failedChunkHighestInstant
The highest log instant in failedChunk


highestShippedInstant

private long highestShippedInstant
The highest log instant shipped so far


FI_LOW

private static final int FI_LOW
Fill information value indicative of a low load in the log buffer.

See Also:
Constant Field Values

FI_HIGH

private static final int FI_HIGH
Fill information value indicative of a high load in the log buffer.

See Also:
Constant Field Values

MIN

private static final long MIN
If the fill information (obtained from the log buffer) is less than FI_HIGH but greater than FI_LOW the log shipper will ship with a MIN ms delay. MIN is a value that is only as large as not to affect the performance of the master database significantly.

See Also:
Constant Field Values

MAX

private static final long MAX
If the fill information is less than FI_LOW the log shipper will ship with a MAX ms delay or when a buffer becomes full whichever comes first. The delay however will not be smaller than MIN. max(MAX, DEFAULT_NUMBER_LOG_BUFFERS*MIN) is the maximum delay between a log record is committed at the master until it is replicated to the slave. Hence the default latency should be atleast greater than the maximum latency offered by the choice of MIN, hence MAX > DEFAULT_NUMBER_LOG_BUFFERS*MIN.

See Also:
Constant Field Values

repLogger

private final ReplicationLogger repLogger
Constructor Detail

AsynchronousLogShipper

public AsynchronousLogShipper(ReplicationLogBuffer logBuffer,
                              ReplicationMessageTransmit transmitter,
                              MasterController masterController,
                              ReplicationLogger repLogger)
Constructor initializes the log buffer, the replication message transmitter, the shipping interval and the master controller.

Parameters:
logBuffer - the replication log buffer that contains the log record chunks to be transmitted to the slave.
transmitter - the replication message transmitter that is used for network transmission of retrieved log records.
masterController - The master controller that initialized this log shipper.
repLogger - The replication logger that will write messages to the log file (typically derby.log)
Method Detail

run

public void run()
Ships log records from the log buffer to the slave being replicated to. The log shipping happens between shipping intervals of time, the shipping interval being derived from the fill information (an indicator of load in the log buffer) obtained from the log buffer. The shipping can also be triggered in the following situations, 1) Based on notifications from the log buffer, where the fill information is again used as the basis to decide whether a shipping should happen or not 2) On a forceFlush triggered by the log buffer becoming full and the LogBufferFullException being thrown.

Specified by:
run in interface java.lang.Runnable
Overrides:
run in class java.lang.Thread

shipALogChunk

private boolean shipALogChunk()
                       throws java.io.IOException,
                              StandardException
Retrieves a chunk of log records, if available, from the log buffer and transmits them to the slave. Used for both periodic and forced shipping.

Returns:
true if a chunk of log records was shipped. false if no log records were shipped because log buffer is empty.
Throws:
java.io.IOException - If an exception occurs while trying to ship the replication message (containing the log records) across the network.
StandardException - If an exception occurs while trying to read log records from the log buffer.

flushBuffer

public void flushBuffer()
                 throws java.io.IOException,
                        StandardException
Transmits all the log records in the log buffer to the slave.

Specified by:
flushBuffer in interface LogShipper
Throws:
java.io.IOException - If an exception occurs while trying to ship the replication message (containing the log records) across the network.
StandardException - If an exception occurs while trying to read log records from the log buffer.

forceFlush

public void forceFlush()
                throws java.io.IOException,
                       StandardException
Transmits a chunk of log record from the log buffer to the slave, used by the master controller when the log buffer is full and some space needs to be freed for further log records.

Specified by:
forceFlush in interface LogShipper
Throws:
java.io.IOException - If an exception occurs while trying to ship the replication message (containing the log records) across the network.
StandardException - If an exception occurs while trying to read log records from the log buffer.

getHighestShippedInstant

public long getHighestShippedInstant()
Get the highest log instant shipped so far

Returns:
the highest log instant shipped so far

flushedInstance

public void flushedInstance(long latestInstanceFlushedToDisk)
updates the information about the latest instance of the log record that has been flushed to the disk. Calling this method has no effect in this asynchronous implementation of the log shipper.

Specified by:
flushedInstance in interface LogShipper
Parameters:
latestInstanceFlushedToDisk - a long that contains the latest instance of the log record that has been flushed to the disk.

stopLogShipment

public void stopLogShipment()
Stop shipping log records. If a ship is currently in progress it will not be interrupted, shipping will stop only after the current shipment is done.


workToDo

public void workToDo()
Used to notify the log shipper that a log buffer element is full. This method would basically use the following steps to decide on the action to be taken when a notification from the log shipper is received, a) Get FI from log buffer b) If FI >= FI_HIGH b.1) notify the log shipper thread. c) Else If the time elapsed since last ship is greater than minShippingInterval c.1) notify the log shipper thread.

Specified by:
workToDo in interface LogShipper

calculateSIfromFI

private long calculateSIfromFI()
Will be used to calculate the shipping interval based on the fill information obtained from the log buffer. This method uses the following steps to arrive at the shipping interval, a) FI >= FI_HIGH return -1 (signifies that the waiting time should be 0) b) FI > FI_LOW and FI < FI_HIGH return minShippingInterval c) FI <= FI_LOW return maxShippingInterval.

Returns:
the shipping interval based on the fill information.

getLogShipperProperties

private void getLogShipperProperties()
Load relevant system properties: max and min log shipping interval


Built on Thu 2010-12-23 20:49:13+0000, from revision ???

Apache Derby V10.6 Internals - Copyright © 2004,2007 The Apache Software Foundation. All Rights Reserved.