com.mckoi.store
Class AbstractStore

java.lang.Object
  extended by com.mckoi.store.AbstractStore
All Implemented Interfaces:
Store
Direct Known Subclasses:
JournalledFileStore

public abstract class AbstractStore
extends java.lang.Object
implements Store

Provides an abstract implementation of Store. This implements a bin based best-fit recycling algorithm. The store manages a structure that points to bins of freed space of specific sizes. When an allocation is requested the structure is searched for the first bin that contains an area that best fits the size requested.

Provided the derived class supports safe atomic IO operations, this store is designed for robustness to the level that at no point is the store left in a unworkable (corrupt) state.

Author:
Tobias Downer

Field Summary
protected static long BIN_AREA_OFFSET
          The offset into the file that the bin area starts.
protected static int BIN_ENTRIES
           
protected static long DATA_AREA_OFFSET
          The offset into the file that the data areas start.
protected static long FIXED_AREA_OFFSET
          The offset into the file of the 64 byte fixed area.
protected  long[] free_bin_list
          The free bin list contains 128 entries pointing to the first available block in the bin.
protected  byte[] header_buf
           
protected static int MAGIC
          The magic value.
protected  boolean read_only
          True if this is read-only.
protected  long total_allocated_space
          The total amount of allocated space within this store since the store was openned.
protected  long wilderness_pointer
          A pointer to the wilderness area (the last deleted area in the store), or -1 if there is no wilderness area.
 
Constructor Summary
protected AbstractStore(boolean read_only)
          Constructs the store.
 
Method Summary
protected  void checkPointer(long pointer)
          Checks the pointer is valid.
 void close()
          Closes the store.
protected  void coalescArea(long pointer, long size)
          Coalesc one or more areas into a larger area.
 AreaWriter createArea(long size)
          Allocates a block of memory in the store of the specified size and returns an AreaWriter object that can be used to initialize the contents of the area.
 void deleteArea(long id)
          Deletes an area that was previously allocated by the 'createArea' method by the area id.
protected abstract  long endOfDataAreaPointer()
          Returns a pointer to the end of the current data area.
protected  long expandDataArea(long minimum_size)
          Expands the data area by at least the minimum size given.
 java.util.ArrayList findAllocatedAreasNotIn(java.util.ArrayList list)
          Scans the area list, and any areas that aren't deleted and aren't found in the given ArrayList are returned as leaked areas.
 java.util.List getAllAreas()
          Returns a List of Long objects that contain a complete list of all areas in the store.
 Area getArea(long id)
          Returns an object that allows for the contents of an area (represented by the 'id' parameter) to be read.
protected  void getAreaHeader(long pointer, long[] header)
          Sets the 'header' array with information from the header of the given pointer.
 java.io.InputStream getAreaInputStream(long id)
          Returns an InputStream implementation that allows for the area with the given identifier to be read sequentially.
 MutableArea getMutableArea(long id)
          Returns an object that allows for the contents of an area (represented by the 'id' parameter) to be read and written.
protected  long getNextAreaHeader(long pointer, long[] header)
          Sets the 'header' array with information from the next header to the given pointer, and returns a pointer to the next area.
protected  long getPreviousAreaHeader(long pointer, long[] header)
          Sets the 'header' array with information from the previous header to the given pointer, and returns a pointer to the previous area.
protected abstract  void internalClose()
          Internally closes the backing area.
protected abstract  void internalOpen(boolean read_only)
          Internally opens the backing area.
protected static boolean isValidBoundarySize(long size)
          Returns true if the given area size is valid.
 boolean lastCloseClean()
          Returns true if the store was closed cleanly.
 boolean open()
          Opens the data store.
 void openScanAndFix(UserTerminal terminal)
          Opens/scans the store looking for any errors with the layout.
protected  void readBins()
          Reads the bins from the header information in the file.
protected abstract  int readByteArrayFrom(long position, byte[] buf, int off, int len)
          Reads a byte array from the given position in the file.
protected abstract  int readByteFrom(long position)
          Reads a byte from the given position in the file.
protected  void reboundArea(long pointer, long[] header, boolean write_headers)
          Rebounds the given area with the given header information.
protected abstract  void setDataAreaSize(long length)
          Sets the size of the data area.
protected  void splitArea(long pointer, long new_boundary)
          Splits an area pointed to by 'pointer' at a new boundary point.
 void statsScan(java.util.HashMap properties)
          Performs an extensive lookup on all the tables in this store and sets a number of properties in the given HashMap (property name(String) -> property description(Object)).
 long totalAllocatedSinceStart()
          Returns the total allocated space since the file was openned.
protected  void writeAllBins()
          Updates all bins to the data area header area.
protected  void writeBinIndex(int index)
          Updates the given bin index to the data area header area.
protected abstract  void writeByteArrayTo(long position, byte[] buf, int off, int len)
          Writes a byte array to the given position in the file.
protected abstract  void writeByteTo(long position, int b)
          Writes a byte to the given position in the file.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface com.mckoi.store.Store
lockForWrite, unlockForWrite
 

Field Detail

free_bin_list

protected long[] free_bin_list
The free bin list contains 128 entries pointing to the first available block in the bin. If the list item contains -1 then there are no free blocks in the bin.


wilderness_pointer

protected long wilderness_pointer
A pointer to the wilderness area (the last deleted area in the store), or -1 if there is no wilderness area.


read_only

protected boolean read_only
True if this is read-only.


total_allocated_space

protected long total_allocated_space
The total amount of allocated space within this store since the store was openned. Note that this could be a negative amount if more space was freed than allocated.


DATA_AREA_OFFSET

protected static final long DATA_AREA_OFFSET
The offset into the file that the data areas start.

See Also:
Constant Field Values

FIXED_AREA_OFFSET

protected static final long FIXED_AREA_OFFSET
The offset into the file of the 64 byte fixed area.

See Also:
Constant Field Values

BIN_AREA_OFFSET

protected static final long BIN_AREA_OFFSET
The offset into the file that the bin area starts.

See Also:
Constant Field Values

MAGIC

protected static final int MAGIC
The magic value.

See Also:
Constant Field Values

header_buf

protected final byte[] header_buf

BIN_ENTRIES

protected static final int BIN_ENTRIES
Constructor Detail

AbstractStore

protected AbstractStore(boolean read_only)
Constructs the store.

Method Detail

open

public boolean open()
             throws java.io.IOException
Opens the data store. Returns true if the store did not close cleanly.

Throws:
java.io.IOException

close

public void close()
           throws java.io.IOException
Closes the store.

Throws:
java.io.IOException

isValidBoundarySize

protected static boolean isValidBoundarySize(long size)
Returns true if the given area size is valid. Currently the criteria for a valid boundary size is (size >= 24) and (size % 8 == 0) and (size < 200 gigabytes)


openScanAndFix

public void openScanAndFix(UserTerminal terminal)
                    throws java.io.IOException
Opens/scans the store looking for any errors with the layout. If a problem with the store is detected, it attempts to fix it.

Throws:
java.io.IOException

statsScan

public void statsScan(java.util.HashMap properties)
               throws java.io.IOException
Performs an extensive lookup on all the tables in this store and sets a number of properties in the given HashMap (property name(String) -> property description(Object)). This should be used for store diagnostics.

Assume the store is open.

Throws:
java.io.IOException

getAllAreas

public java.util.List getAllAreas()
                           throws java.io.IOException
Returns a List of Long objects that contain a complete list of all areas in the store. This is useful for checking if a given pointer is valid or not. The returned list is sorted from start area to end area.

Specified by:
getAllAreas in interface Store
Throws:
java.io.IOException

findAllocatedAreasNotIn

public java.util.ArrayList findAllocatedAreasNotIn(java.util.ArrayList list)
                                            throws java.io.IOException
Scans the area list, and any areas that aren't deleted and aren't found in the given ArrayList are returned as leaked areas. This is a useful method for finding any leaks in the store.

Throws:
java.io.IOException

totalAllocatedSinceStart

public long totalAllocatedSinceStart()
Returns the total allocated space since the file was openned.


internalOpen

protected abstract void internalOpen(boolean read_only)
                              throws java.io.IOException
Internally opens the backing area. If 'read_only' is true then the store is openned in read only mode.

Throws:
java.io.IOException

internalClose

protected abstract void internalClose()
                               throws java.io.IOException
Internally closes the backing area.

Throws:
java.io.IOException

readByteFrom

protected abstract int readByteFrom(long position)
                             throws java.io.IOException
Reads a byte from the given position in the file.

Throws:
java.io.IOException

readByteArrayFrom

protected abstract int readByteArrayFrom(long position,
                                         byte[] buf,
                                         int off,
                                         int len)
                                  throws java.io.IOException
Reads a byte array from the given position in the file. Returns the number of bytes read.

Throws:
java.io.IOException

writeByteTo

protected abstract void writeByteTo(long position,
                                    int b)
                             throws java.io.IOException
Writes a byte to the given position in the file.

Throws:
java.io.IOException

writeByteArrayTo

protected abstract void writeByteArrayTo(long position,
                                         byte[] buf,
                                         int off,
                                         int len)
                                  throws java.io.IOException
Writes a byte array to the given position in the file.

Throws:
java.io.IOException

endOfDataAreaPointer

protected abstract long endOfDataAreaPointer()
                                      throws java.io.IOException
Returns a pointer to the end of the current data area.

Throws:
java.io.IOException

setDataAreaSize

protected abstract void setDataAreaSize(long length)
                                 throws java.io.IOException
Sets the size of the data area.

Throws:
java.io.IOException

checkPointer

protected void checkPointer(long pointer)
                     throws java.io.IOException
Checks the pointer is valid.

Throws:
java.io.IOException

readBins

protected void readBins()
                 throws java.io.IOException
Reads the bins from the header information in the file.

Throws:
java.io.IOException

writeAllBins

protected void writeAllBins()
                     throws java.io.IOException
Updates all bins to the data area header area.

Throws:
java.io.IOException

writeBinIndex

protected void writeBinIndex(int index)
                      throws java.io.IOException
Updates the given bin index to the data area header area.

Throws:
java.io.IOException

getAreaHeader

protected void getAreaHeader(long pointer,
                             long[] header)
                      throws java.io.IOException
Sets the 'header' array with information from the header of the given pointer.

Throws:
java.io.IOException

getPreviousAreaHeader

protected long getPreviousAreaHeader(long pointer,
                                     long[] header)
                              throws java.io.IOException
Sets the 'header' array with information from the previous header to the given pointer, and returns a pointer to the previous area.

Throws:
java.io.IOException

getNextAreaHeader

protected long getNextAreaHeader(long pointer,
                                 long[] header)
                          throws java.io.IOException
Sets the 'header' array with information from the next header to the given pointer, and returns a pointer to the next area.

Throws:
java.io.IOException

reboundArea

protected void reboundArea(long pointer,
                           long[] header,
                           boolean write_headers)
                    throws java.io.IOException
Rebounds the given area with the given header information. If 'write_headers' is true, the header (header[0]) is changed. Note that this shouldn't be used to change the size of a chunk.

Throws:
java.io.IOException

coalescArea

protected void coalescArea(long pointer,
                           long size)
                    throws java.io.IOException
Coalesc one or more areas into a larger area. This alters the boundary of the area to encompass the given size.

Throws:
java.io.IOException

expandDataArea

protected long expandDataArea(long minimum_size)
                       throws java.io.IOException
Expands the data area by at least the minimum size given. Returns the actual size the data area was expanded by.

Throws:
java.io.IOException

splitArea

protected void splitArea(long pointer,
                         long new_boundary)
                  throws java.io.IOException
Splits an area pointed to by 'pointer' at a new boundary point.

Throws:
java.io.IOException

createArea

public AreaWriter createArea(long size)
                      throws java.io.IOException
Description copied from interface: Store
Allocates a block of memory in the store of the specified size and returns an AreaWriter object that can be used to initialize the contents of the area. Note that an area in the store is undefined until the 'finish' method is called in AreaWriter.

Specified by:
createArea in interface Store
Parameters:
size - the amount of memory to allocate.
Returns:
an AreaWriter object that allows the area to be setup.
Throws:
java.io.IOException - if not enough space available to create the area or the store is read-only.

deleteArea

public void deleteArea(long id)
                throws java.io.IOException
Description copied from interface: Store
Deletes an area that was previously allocated by the 'createArea' method by the area id. Once an area is deleted the resources may be reclaimed. The behaviour of this method is undefined if the id doesn't represent a valid area.

Specified by:
deleteArea in interface Store
Parameters:
id - the identifier of the area to delete.
Throws:
java.io.IOException - (optional) if the id is invalid or the area can not otherwise by deleted.

getAreaInputStream

public java.io.InputStream getAreaInputStream(long id)
                                       throws java.io.IOException
Description copied from interface: Store
Returns an InputStream implementation that allows for the area with the given identifier to be read sequentially. The behaviour of this method, and InputStream object, is undefined if the id doesn't represent a valid area.

When 'id' is -1 then a fixed area (64 bytes in size) in the store is returned. The fixed area can be used to store important static static information.

Specified by:
getAreaInputStream in interface Store
Parameters:
id - the identifier of the area to read, or id = -1 is a 64 byte fixed area in the store.
Returns:
an InputStream that allows the area to be read from the start.
Throws:
java.io.IOException - (optional) if the id is invalid or the area can not otherwise be accessed.

getArea

public Area getArea(long id)
             throws java.io.IOException
Description copied from interface: Store
Returns an object that allows for the contents of an area (represented by the 'id' parameter) to be read. The behaviour of this method, and Area object, is undefined if the id doesn't represent a valid area.

When 'id' is -1 then a fixed area (64 bytes in size) in the store is returned. The fixed area can be used to store important static static information.

Specified by:
getArea in interface Store
Parameters:
id - the identifier of the area to read, or id = -1 is a 64 byte fixed area in the store.
Returns:
an Area object that allows access to the part of the store.
Throws:
java.io.IOException - (optional) if the id is invalid or the area can not otherwise be accessed.

getMutableArea

public MutableArea getMutableArea(long id)
                           throws java.io.IOException
Description copied from interface: Store
Returns an object that allows for the contents of an area (represented by the 'id' parameter) to be read and written. The behaviour of this method, and MutableArea object, is undefined if the id doesn't represent a valid area.

When 'id' is -1 then a fixed area (64 bytes in size) in the store is returned. The fixed area can be used to store important static static information.

Specified by:
getMutableArea in interface Store
Parameters:
id - the identifier of the area to access, or id = -1 is a 64 byte fixed area in the store.
Returns:
a MutableArea object that allows access to the part of the store.
Throws:
java.io.IOException - (optional) if the id is invalid or the area can not otherwise be accessed.

lastCloseClean

public boolean lastCloseClean()
Description copied from interface: Store
Returns true if the store was closed cleanly. This is important information that may need to be considered when reading information from the store. This is typically used to issue a scan on the data in the store when it is not closed cleanly.

Specified by:
lastCloseClean in interface Store