com.sleepycat.je.evictor
Class Evictor

java.lang.Object
  extended by com.sleepycat.je.evictor.Evictor
All Implemented Interfaces:
EnvConfigObserver
Direct Known Subclasses:
PrivateEvictor, SharedEvictor

public abstract class Evictor
extends Object
implements EnvConfigObserver

The Evictor is responsible for maintaining the JE cache. Since object sizes are not directly manipulated in a Java application, the cache is actually a collection of in-memory btree nodes, implemented by com.sleepycat.je.dbi.INList. Nodes are selected from the INList for removal, which is done by detaching them from the in-memory tree, and by removing them from the INList. Once all references to them are removed, they can be GC'd by the JVM. There are three main components. Arbiter: queries the memory budget to decide whether eviction is needed TargetSelector : chooses a target node Evictor: does the work of detaching the node. The TargetSelector and Evictor classes are subclassed to provide private/shared cache implementations. A shared cache is used by multiple environments within a single JVM, and is seen logically as a single INList collection, although it is implemented by an umbrella over multiple INLists. The Evictor owns a thread pool which is available to handle eviction tasks. Eviction is carried out by three types of threads: 1. The application thread, in the course of doing critical eviction 2. Daemon threads, such as the cleaner or INCompressor, in the course of doing their respective duties 3. Eviction pool threads We prefer that the eviction pool threads do as much of the eviction as possible, and that the application threads do as little, because eviction adds latency to the perceived application response time. To date, it has been impossible to completely remove eviction responsiblities from the application threads, because the process doesn't have sufficient feedback, and can incur an OutOfMemoryException. The eviction pool is a standard java.util.concurrent thread pool, and can be mutably configured in terms of core threads, max threads, and keepalive times. Since three types of threads can concurrently do eviction, it's important that eviction is both thread safe and as parallel as possible. Memory thresholds are generally accounted for in an unsynchronized fashion, and are seen as advisory. The only point of true synchronization is around the selection of a node for eviction. The act of eviction itself can be done concurrently. The eviction method is not reentrant, and a simple concurrent hash map of threads is used to prevent recursive calls.


Nested Class Summary
(package private) static class Evictor.BackgroundEvictTask
           
(package private) static interface Evictor.DbCache
          Caches DatabaseImpls to reduce DbTree.getDb overhead.
static class Evictor.EvictionSource
           
(package private) static class Evictor.RejectEvictHandler
           
 
Field Summary
(package private)  int dbCacheClearCount
           
(package private)  EnvironmentImpl envImpl
           
 
Constructor Summary
Evictor(EnvironmentImpl envImpl)
           
 
Method Summary
 void addEnvironment(EnvironmentImpl additionalEnvImpl)
           
 void alert()
          Let the eviction pool know there's work to do.
 boolean checkEnv(EnvironmentImpl env)
          Only supported by SharedEvictor.
(package private) abstract  Evictor.DbCache createDbCache()
           
 void doCriticalEviction(boolean backgroundIO)
          Do some eviction before proceeding on with another operation.
 void doDaemonEviction(boolean backgroundIO)
          Do a check on whether synchronous eviction is needed.
(package private)  void doEvict(Evictor.EvictionSource source, boolean backgroundIO)
          Can execute concurrently, called by app threads or by background evictor
 void doEvictOneIN(IN target, Evictor.EvictionSource source)
          Evict a specific IN, used by cache modes.
 void doManualEvict()
           
 void envConfigUpdate(DbConfigManager configManager, EnvironmentMutableConfig ignore)
          Respond to config updates.
(package private)  long evictBatch(Evictor.EvictionSource source, boolean backgroundIO, long maxEvictBytes)
          Each iteration will attempt to evict maxEvictBytes, but will give up after a complete pass over the INList, or if there is nothing more to evict, due to actions by concurrently executing threads.
 AtomicLong getNINCompactKey()
           
 AtomicLong getNINNoTarget()
           
 AtomicLong getNINSparseTarget()
           
(package private)  TargetSelector getSelector()
           
 StatGroup getStatsGroup()
           
 ThreadPoolExecutor getThreadPool()
           
 void incBINEvictStats(Evictor.EvictionSource source)
           
 void incBINFetchStats(boolean isMiss)
           
 void incINEvictStats(Evictor.EvictionSource source)
           
 void incINFetchStats(boolean isMiss)
           
 void incLNFetchStats(boolean isMiss)
          Update the appropriate fetch stat, based on node type.
 boolean isCacheFull()
           
 StatGroup loadStats(StatsConfig config)
          Load stats.
(package private) abstract  TargetSelector makeSelector()
           
 void noteINListChange(int nINs)
          Called whenever INs are added to, or removed from, the INList.
 void removeEnvironment(EnvironmentImpl targetEnvImpl)
           
 void requestShutdownPool()
           
 void setEvictProfileHook(TestHook<IN> hook)
           
 void setPreEvictINHook(TestHook<Object> hook)
           
 void setRunnableHook(TestHook<Boolean> hook)
           
 void shutdown()
          Request and wait for a shutdown of all running eviction tasks.
 boolean wasCacheEverFull()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

envImpl

final EnvironmentImpl envImpl

dbCacheClearCount

int dbCacheClearCount
Constructor Detail

Evictor

Evictor(EnvironmentImpl envImpl)
  throws DatabaseException
Throws:
DatabaseException
Method Detail

envConfigUpdate

public void envConfigUpdate(DbConfigManager configManager,
                            EnvironmentMutableConfig ignore)
                     throws DatabaseException
Respond to config updates.

Specified by:
envConfigUpdate in interface EnvConfigObserver
Throws:
DatabaseException

makeSelector

abstract TargetSelector makeSelector()

loadStats

public StatGroup loadStats(StatsConfig config)
Load stats.


doCriticalEviction

public void doCriticalEviction(boolean backgroundIO)
Do some eviction before proceeding on with another operation. Note that this method is intentionally not synchronized in order to minimize overhead when checking for critical eviction. This method is called from application threads for every cursor operation, and by many daemon threads.


doDaemonEviction

public void doDaemonEviction(boolean backgroundIO)
Do a check on whether synchronous eviction is needed. Note that this method is intentionally not synchronized in order to minimize overhead when checking for critical eviction. This method is called from application threads for every cursor operation.


doManualEvict

public void doManualEvict()
                   throws DatabaseException
Throws:
DatabaseException

doEvictOneIN

public void doEvictOneIN(IN target,
                         Evictor.EvictionSource source)
Evict a specific IN, used by cache modes.


alert

public void alert()
Let the eviction pool know there's work to do.


getThreadPool

public ThreadPoolExecutor getThreadPool()

doEvict

void doEvict(Evictor.EvictionSource source,
             boolean backgroundIO)
       throws DatabaseException
Can execute concurrently, called by app threads or by background evictor

Throws:
DatabaseException

evictBatch

long evictBatch(Evictor.EvictionSource source,
                boolean backgroundIO,
                long maxEvictBytes)
          throws DatabaseException
Each iteration will attempt to evict maxEvictBytes, but will give up after a complete pass over the INList, or if there is nothing more to evict, due to actions by concurrently executing threads. This method is thread safe and may be called concurrently.

Returns:
the number of bytes evicted, or zero if no progress was made. Note that if the INList is completely empty, it's still possible to return a non-zero number of bytes due to special eviction of items such as utilization info, even though no IN eviction happened.
Throws:
DatabaseException

incBINEvictStats

public void incBINEvictStats(Evictor.EvictionSource source)

incINEvictStats

public void incINEvictStats(Evictor.EvictionSource source)

incLNFetchStats

public void incLNFetchStats(boolean isMiss)
Update the appropriate fetch stat, based on node type.


incBINFetchStats

public void incBINFetchStats(boolean isMiss)

incINFetchStats

public void incINFetchStats(boolean isMiss)

getNINSparseTarget

public AtomicLong getNINSparseTarget()

getNINNoTarget

public AtomicLong getNINNoTarget()

getNINCompactKey

public AtomicLong getNINCompactKey()

addEnvironment

public void addEnvironment(EnvironmentImpl additionalEnvImpl)

removeEnvironment

public void removeEnvironment(EnvironmentImpl targetEnvImpl)

setPreEvictINHook

public void setPreEvictINHook(TestHook<Object> hook)

setEvictProfileHook

public void setEvictProfileHook(TestHook<IN> hook)

noteINListChange

public void noteINListChange(int nINs)
Called whenever INs are added to, or removed from, the INList.


checkEnv

public boolean checkEnv(EnvironmentImpl env)
Only supported by SharedEvictor.


getStatsGroup

public StatGroup getStatsGroup()

setRunnableHook

public void setRunnableHook(TestHook<Boolean> hook)

isCacheFull

public boolean isCacheFull()

wasCacheEverFull

public boolean wasCacheEverFull()

getSelector

TargetSelector getSelector()

shutdown

public void shutdown()
Request and wait for a shutdown of all running eviction tasks.


requestShutdownPool

public void requestShutdownPool()

createDbCache

abstract Evictor.DbCache createDbCache()


Copyright (c) 2004-2012 Oracle. All rights reserved.