Hbase oldWALs directory: what is it, when is it cleaned and who use its ?

The oldWALs folder gets cleaned regularly by a chore in master. When a WAL file is not needed any more for recovery purposes (when HBase can guaratee HBase has flushed all the data in the WAL file), it is moved to the oldWALs
folder for archival. The log stays there until all other references to the
WAL file are finished. There is currently two services which may keep the
files in the archive dir. First is a TTL process, which ensures that the
WAL files are kept at least for 10 min. This is mainly for debugging. You
can reduce this time by setting hbase.master.logcleaner.ttl configuration
property in master. It is by default 600000. The other one is replication.
If you have replication setup, the replication processes will hang on to
the WAL files until they are replicated. Even if you disabled the
replication, the files are still referenced.

Source: http://mail-archives.apache.org/mod_mbox/hbase-user/201502.mbox/%3CCAMUu0w9aOVBo7kGULiM9tXrULirqs9fm-3ra3pQccYpW_17uOw@mail.gmail.com%3E

12 Mart 2015

Posted In: hbase, oldWALs, replication, WAL

Hbase oldWALs directory: what is it, when is it cleaned and who use its ?

The oldWALs folder gets cleaned regularly by a chore in master. When a WAL file is not needed any more for recovery purposes (when HBase can guaratee HBase has flushed all the data in the WAL file), it is moved to the oldWALs
folder for archival. The log stays there until all other references to the
WAL file are finished. There is currently two services which may keep the
files in the archive dir. First is a TTL process, which ensures that the
WAL files are kept at least for 10 min. This is mainly for debugging. You
can reduce this time by setting hbase.master.logcleaner.ttl configuration
property in master. It is by default 600000. The other one is replication.
If you have replication setup, the replication processes will hang on to
the WAL files until they are replicated. Even if you disabled the
replication, the files are still referenced.

Source: http://mail-archives.apache.org/mod_mbox/hbase-user/201502.mbox/%3CCAMUu0w9aOVBo7kGULiM9tXrULirqs9fm-3ra3pQccYpW_17uOw@mail.gmail.com%3E

12 Mart 2015

Posted In: hbase, oldWALs, replication, WAL

WP Twitter Auto Publish Powered By : XYZScripts.com