A technique for parallel I/O on Write Ahead Log (01-Feb-2010)
Disclosed is an invention which removes the serialization of synchronous writes to write ahead log while preserving the strong data integrity semantics. Applications like database systems or log structured file systems employ write ahead logging to ensure strong data integrity. When an update to a persistent data is made in the application, the data may still be updated in memory. However, a log record corresponding to the update is written to a write ahead log which is associated with a logging disk. A log produces better performance than synchronous writes of the persistent data because there are fewer writes and writes have better locality. There are systems that cache the log records in memory and write them asynchronously. However, strong data integrity may require the log records to be written synchronously on the disk. There could be several threads executing and writing log records in a multi-threaded application. Writes to write ahead log from different threads are serialized. If the log records are to be synchronously written to the disk then this serialization impacts the performance of the write ahead log.
A technique for parallel I/O on Write Ahead Log
Assumptions:
a) Parallel I/O for log records delivered to the disk are faster than if the I/O for log records was delivered sequentially. This is generally true if the disk controller has a write cache (which may be backed by NVRAM).
b) No two writes at a given time to the write ahead log overlap on a disk sector. This is also true for all the log based systems.
c) The new technique will not change the log recovery semantics. That is, recovery after a system failure or other wise, should still ensure atomicity and durability of the transactions.
Technique:
Let's call the process of synchronizing log records to the disk as a log flush. Let's first go through a log flush in a general log based system. Then, we will go through the new technique.
Log flush in a General Logging System:
* First a latch is acquired which serializes access to the write ahead log.
* Then the log buffers are removed that need to be written to the disk and calculate the start and end offsets on disk for the log buffers.
* Then the log buffers are written synchronously to the disk at the computed offsets.
* Latch to serialize write ahead log is released.
a
_thread
start transaction update data;
create a log record for the update in the log buffer if synchronous logging then
/* flush
_
buffers */
_executing
_transactions()
{
_log
{
acquire a latch to serialize access to write ahead log
/* Note that the section removed should end at the sector boundary. Any new log records should be created from the last sector removed for writing */
1
remove the section of the log buffer that has new log records
compute the start and end offset for the writing the log buffer.
write the log buffers synchronously to disk
release the latch
}
transaction finished , release locks held by the txn }
Log flush with the new technique:
* First, a latch is acquired which serializes access to the write ahead log.
* We then remove the log buffers that need to be written to the disk and calculate the start and end offsets on disk for the log buffers. After that, we add this thread along with the start and end offsets to the list of threads that are currently writing log buffers synchronously.
* Latch to serialize write ahead log is released.
* Log buffers are written synchronously to the disk using the computed offsets. No locks are held during the disk I/O and several log flushes may do disk I/O at different offsets in the log device in parallel. Any subsequent transactions that need to write log records will create new log buffers. Any subsequent log flushes for the new buffers will happen from the end offset of the last log flush.
* After the log buffers are flushed to the disk, the thread doing the flush will acquire the latch used to serialize write ahead log. Then it checks if the stable offset is the same as its start offset. If it is so then the log flushes at prior offsets have finished...
Find More Documents Like This
Download this Disclosure
Click here if you want to sign-in now.
