|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectcom.sleepycat.je.cleaner.Cleaner
public class Cleaner
The Cleaner is responsible for effectively garbage collecting the JE log. It selects the least utilized log file for cleaning (see FileSelector), reads through the log file (FileProcessor) and determines whether each entry is obsolete (no longer relavent) or active (referenced by the Btree). Entries that are active are migrated (copied) to the end of the log, and finally the cleaned file is deleted. The migration of active entries is a multi-step process that can be configured to operate in different ways. Eviction and checkpointing, as well as the cleaner threads (FileProcessor instances) are participants in this process. Migration may be immediate, lazy, or proactive. Active INs are always migrated lazily, which means that they are marked dirty by the FileProcessor, and then logged later by an eviction or checkpoint. Active LNs are either migrated lazily or immediately depending on the EnvironmentConfig.CLEANER_LAZY_MIGRATION setting. If they are migrated lazily, the migrate flag is set in the LN slot by the FileProcessor, and the LN is logged later by an eviction or checkpoint. When the FileProcessor is finished with a file, all lazy migration for that file is normally completed by the end of the next checkpoint, if not sooner via eviction. The checkpoint/recovery mechanism will ensure that obsolete entries will not be referenced by the Btree. At the end of the checkpoint, it is therefore safe to delete the log file. There is one exception to the above paragraph. When attempting to migrate an LN, if the LN cannot be locked then we must retry the migration at a later time. Also, if a database removal is in progress, we consider all entries in the database obsolete but cannot delete the log file until database removal is complete. Such "pending" LNs and databases are queued and processed periodically during file processing and at the start of a checkpoint; see processPending(). In this case, we may have to wait for more than one checkpoint to occur before the log file can be deleted. See FileSelector and the use of the pendingLNs and pendingDBs collections. The last type of migration, proactive migration, is migration of LNs by the evictor or checkpointer for files that are to-be-cleaned, i.e., part of the cleaner's backlog. The idea is to prevent the backlog from growing too large (and potentially filling the disk) by doing more cleaner work during eviction, which will throttle the application threads. By default, proactive migration is disabled, but it can enabled using EnvironmentConfig.CLEANER_BACKGROUND_PROACTIVE_MIGRATION and CLEANER_FOREGROUND_PROACTIVE_MIGRATION.
Field Summary | |
---|---|
(package private) static String |
CLEAN_IN
|
(package private) static String |
CLEAN_LN
|
(package private) static String |
CLEAN_MIGRATE_LN
|
(package private) static String |
CLEAN_PENDING_LN
|
(package private) long |
cleanerBytesInterval
|
(package private) boolean |
clusterAll
|
(package private) boolean |
clusterResident
|
(package private) int |
dbCacheClearCount
|
(package private) static boolean |
DO_CRITICAL_EVICTION
Whether the cleaner should participate in critical eviction. |
(package private) boolean |
expunge
|
(package private) boolean |
fetchObsoleteSize
|
(package private) TestHook |
fileChosenHook
|
(package private) boolean |
lazyMigration
|
(package private) FloatStat |
lnSizeCorrectionFactor
|
(package private) long |
lockTimeout
|
(package private) int |
lookAheadCacheSize
|
(package private) int |
maxBatchFiles
|
(package private) int |
minAge
|
(package private) int |
minFileUtilization
|
(package private) int |
minUtilization
|
(package private) LongStat |
nBINDeltasCleaned
|
(package private) LongStat |
nBINDeltasDead
|
(package private) LongStat |
nBINDeltasMigrated
|
(package private) LongStat |
nBINDeltasObsolete
|
(package private) LongStat |
nCleanerDeletions
|
(package private) LongStat |
nCleanerProbeRuns
|
(package private) LongStat |
nCleanerRuns
|
(package private) LongStat |
nClusterLNsProcessed
|
(package private) long |
nDeadlockRetries
|
(package private) LongStat |
nEntriesRead
|
(package private) LongStat |
nINsCleaned
|
(package private) LongStat |
nINsDead
|
(package private) LongStat |
nINsMigrated
|
(package private) LongStat |
nINsObsolete
|
(package private) LongStat |
nLNQueueHits
|
(package private) LongStat |
nLNsCleaned
|
(package private) LongStat |
nLNsDead
|
(package private) LongStat |
nLNsLocked
|
(package private) LongStat |
nLNsMarked
|
(package private) LongStat |
nLNsMigrated
|
(package private) LongStat |
nLNsObsolete
|
(package private) LongStat |
nMarkedLNsProcessed
|
(package private) LongStat |
nPendingLNsLocked
|
(package private) LongStat |
nPendingLNsProcessed
|
(package private) LongStat |
nRepeatIteratorReads
|
(package private) LongStat |
nToBeCleanedLNsProcessed
|
(package private) int |
readBufferSize
|
(package private) StatGroup |
stats
|
(package private) LongStat |
totalLogSize
|
(package private) AtomicLong |
totalRuns
|
(package private) boolean |
trackDetail
|
(package private) static CacheMode |
UPDATE_GENERATION
Whether to update the IN generation count during searches. |
Constructor Summary | |
---|---|
Cleaner(EnvironmentImpl env,
String name)
|
Method Summary | |
---|---|
(package private) void |
addPendingDB(DatabaseImpl db)
Adds the DB ID to the pending DB set if it is being deleted but deletion is not yet complete. |
void |
addProtectedFileRange(long firstProtectedFile)
Adds a range of log files to be protected from deletion during a backup or similar procedures where log files must not be deleted. |
void |
close()
Release resources and update memory budget. |
(package private) void |
deleteSafeToDeleteFiles()
Deletes all files that are safe-to-delete and which are not protected by a DbBackup or replication. |
int |
doClean(boolean cleanMultipleFiles,
boolean forceCleaning)
Cleans selected files and returns the number of files cleaned. |
void |
envConfigUpdate(DbConfigManager cm,
EnvironmentMutableConfig ignore)
Process notifications of mutable property changes. |
boolean |
getFetchObsoleteSize()
|
FileSelector.CheckpointStartCleanerState |
getFilesAtCheckpointStart()
Returns a copy of the cleaned and processed files at the time a checkpoint starts. |
FileSelector |
getFileSelector()
|
CleanerLogSummary |
getLogSummary()
|
(package private) static ReplicationContext |
getMigrationRepContext(LN ln)
Returns the ReplicationContext to use for migrating the given LN. |
int |
getNWakeupRequests()
|
UtilizationCalculator |
getUtilizationCalculator()
|
UtilizationProfile |
getUtilizationProfile()
|
UtilizationTracker |
getUtilizationTracker()
|
boolean |
isEvictable(BIN bin,
int index,
boolean latched)
Returns whether the given BIN entry may be stripped by the evictor. |
boolean |
isRMWFixEnabled()
|
void |
lazyMigrateLNs(BIN bin,
boolean backgroundIO)
This method should be called just before logging a BIN. |
StatGroup |
loadStats(StatsConfig config)
Load stats. |
(package private) void |
logFine(String action,
Node node,
long logLsn,
boolean completed,
boolean obsolete,
boolean dirtiedMigrated)
Send trace messages to the java.util.logger. |
(package private) void |
processPending()
If any LNs are pending, process them. |
void |
removeProtectedFileRange(long firstProtectedFile)
Removes a range of log files to be protected after calling addProtectedFileRange(long) . |
void |
requestShutdown()
|
void |
runOrPause(boolean run)
|
void |
setExceptionListener(ExceptionListener exceptionListener)
|
void |
setFileChosenHook(TestHook hook)
|
void |
setLogSummary(CleanerLogSummary logSummary)
|
void |
shutdown()
|
void |
updateFilesAtCheckpointEnd(FileSelector.CheckpointStartCleanerState info)
When a checkpoint is complete, update the files that were returned at the beginning of the checkpoint. |
void |
updateReadOnlyFileCollections()
Update the lowUtilizationFiles and toBeCleanedFiles fields with new read-only collections. |
void |
wakeup()
|
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
static final String CLEAN_IN
static final String CLEAN_LN
static final String CLEAN_MIGRATE_LN
static final String CLEAN_PENDING_LN
static final CacheMode UPDATE_GENERATION
static final boolean DO_CRITICAL_EVICTION
StatGroup stats
LongStat nCleanerRuns
LongStat nCleanerProbeRuns
LongStat nCleanerDeletions
LongStat nINsObsolete
LongStat nINsCleaned
LongStat nINsDead
LongStat nINsMigrated
LongStat nBINDeltasObsolete
LongStat nBINDeltasCleaned
LongStat nBINDeltasDead
LongStat nBINDeltasMigrated
LongStat nLNsObsolete
LongStat nLNsCleaned
LongStat nLNsDead
LongStat nLNsLocked
LongStat nLNsMigrated
LongStat nLNsMarked
LongStat nLNQueueHits
LongStat nPendingLNsProcessed
LongStat nMarkedLNsProcessed
LongStat nToBeCleanedLNsProcessed
LongStat nClusterLNsProcessed
LongStat nPendingLNsLocked
LongStat nEntriesRead
LongStat nRepeatIteratorReads
LongStat totalLogSize
FloatStat lnSizeCorrectionFactor
long lockTimeout
int readBufferSize
int lookAheadCacheSize
long nDeadlockRetries
boolean expunge
boolean clusterResident
boolean clusterAll
int maxBatchFiles
long cleanerBytesInterval
boolean trackDetail
boolean fetchObsoleteSize
boolean lazyMigration
int dbCacheClearCount
int minUtilization
int minFileUtilization
int minAge
final AtomicLong totalRuns
TestHook fileChosenHook
Constructor Detail |
---|
public Cleaner(EnvironmentImpl env, String name) throws DatabaseException
DatabaseException
Method Detail |
---|
public void envConfigUpdate(DbConfigManager cm, EnvironmentMutableConfig ignore) throws DatabaseException
envConfigUpdate
in interface EnvConfigObserver
IllegalArgumentException
- via Environment ctor and
setMutableConfig.
DatabaseException
public UtilizationTracker getUtilizationTracker()
public UtilizationProfile getUtilizationProfile()
public UtilizationCalculator getUtilizationCalculator()
public FileSelector getFileSelector()
public boolean getFetchObsoleteSize()
public boolean isRMWFixEnabled()
EnvironmentParams.CLEANER_RMW_FIX
,
FileSummaryLN#postFetchInit
public void setFileChosenHook(TestHook hook)
public CleanerLogSummary getLogSummary()
public void setLogSummary(CleanerLogSummary logSummary)
public void runOrPause(boolean run)
runOrPause
in interface DaemonRunner
public void wakeup()
public void requestShutdown()
requestShutdown
in interface DaemonRunner
public void shutdown()
shutdown
in interface DaemonRunner
public int getNWakeupRequests()
getNWakeupRequests
in interface DaemonRunner
public void setExceptionListener(ExceptionListener exceptionListener)
setExceptionListener
in interface DaemonRunner
setExceptionListener
in interface ExceptionListenerUser
ExceptionListenerUser.setExceptionListener(ExceptionListener)
public int doClean(boolean cleanMultipleFiles, boolean forceCleaning) throws DatabaseException
cleanMultipleFiles
- is true to clean until we're under budget,
or false to clean at most one file.forceCleaning
- is true to clean even if we're not under the
utilization threshold.
DatabaseException
public StatGroup loadStats(StatsConfig config)
void deleteSafeToDeleteFiles() throws DatabaseException
DatabaseException
public void addProtectedFileRange(long firstProtectedFile)
This method is called automatically by the DbBackup
utility and is provided here as a
separate API for advanced applications that may implement a custom
backup procedure.
WARNING: After calling this method, deletion of log files in
the file range by the JE log cleaner will be disabled until removeProtectedFileRange(long)
is called. To prevent unbounded growth of
disk usage, be sure to call removeProtectedFileRange(long)
to
re-enable log file deletion.
firstProtectedFile
- the number of the first file to be protected.
The protected range is from this file number to the last (highest
numbered) file in the log.public void removeProtectedFileRange(long firstProtectedFile)
addProtectedFileRange(long)
.
firstProtectedFile
- the value previously passed to addProtectedFileRange(long)
.
EnvironmentFailureException
- if firstProtectedFile
is not
currently the start of a protected range.public FileSelector.CheckpointStartCleanerState getFilesAtCheckpointStart() throws DatabaseException
If non-null is returned, the checkpoint should flush an extra level, and addCheckpointedFiles() should be called when the checkpoint is complete.
DatabaseException
public void updateFilesAtCheckpointEnd(FileSelector.CheckpointStartCleanerState info) throws DatabaseException
DatabaseException
public void updateReadOnlyFileCollections()
void processPending() throws DatabaseException
DatabaseException
public boolean isEvictable(BIN bin, int index, boolean latched)
latched
- is true if the BIN is latched and an exact answer should
be returned. Is false if the BIN may not be latched; returning the
wrong answer is OK in that case (it will be called again later when
latched), but an exception should not occur.public void lazyMigrateLNs(BIN bin, boolean backgroundIO) throws DatabaseException
On return this method guarantees that no MIGRATE flag will be set on any child entry. If this method is *not* called before logging a BIN, then the addPendingLN method must be called.
bin
- is the latched BIN. The latch will not be released by this
method.
DatabaseException
static ReplicationContext getMigrationRepContext(LN ln)
void addPendingDB(DatabaseImpl db)
void logFine(String action, Node node, long logLsn, boolean completed, boolean obsolete, boolean dirtiedMigrated)
public void close()
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |