leveldb的运行涉及到很多文件,包括manifest文件,WAL log文件,sst文件,日志文件等,为了方便进行文件io,leveldb抽象了几个接口
// A file abstraction for reading sequentially through a file
class SequentialFile {
public:
SequentialFile() { }
virtual ~SequentialFile();
virtual Status Read(size_t n, Slice* result, char* scratch) = 0;
// Skip "n" bytes from the file. This is guaranteed to be no
// slower that reading the same data, but may be faster.
virtual Status Skip(uint64_t n) = 0;
private:
// No copying allowed
SequentialFile(const SequentialFile&);
void operator=(const SequentialFile&);
};
// A file abstraction for randomly reading the contents of a file.
class RandomAccessFile {
public:
RandomAccessFile() { }
virtual ~RandomAccessFile();
// Safe for concurrent use by multiple threads.
virtual Status Read(uint64_t offset, size_t n, Slice* result,
char* scratch) const = 0;
private:
// No copying allowed
RandomAccessFile(const RandomAccessFile&);
void operator=(const RandomAccessFile&);
};
// A file abstraction for sequential writing. The implementation
// must provide buffering since callers may append small fragments
// at a time to the file.
class WritableFile {
public:
WritableFile() { }
virtual ~WritableFile();
virtual Status Append(const Slice& data) = 0;
virtual Status Close() = 0;
virtual Status Flush() = 0;
virtual Status Sync() = 0;
private:
// No copying allowed
WritableFile(const WritableFile&);
void operator=(const WritableFile&);
};
在具体实现这些接口的时候,除了RandomAccessFile类型外,leveldb都是使用标准IO库来实现,也就是<stdio>提供的一套函数,标准IO库有自己的缓存,leveldb没有对磁盘文件做额外的缓存。
对RandomAccessFile 这个类型,leveldb使用了mmap来进行内存映射,以此提高文件IOD的性能。同时控制了能够同时进行内存映射的文件数目。(为了防止内存消耗过度?)
对于日志打印,leveldb的实现比较简单。首先定义了一个接口:
// An interface for writing log messages.
class Logger {
public:
Logger() { }
virtual ~Logger();
// Write an entry to the log file with the specified format.
virtual void Logv(const char* format, va_list ap) = 0;
private:
// No copying allowed
Logger(const Logger&);
void operator=(const Logger&);
};
然后根据具体环境来实现这个接口,在POSIX下,实现的class是:
class PosixLogger : public Logger {
private:
FILE* file_;
uint64_t (*gettid_)(); // Return the thread id for the current thread
public:
PosixLogger(FILE* f, uint64_t (*gettid)()) : file_(f), gettid_(gettid) { }
virtual ~PosixLogger() {
fclose(file_);
}
virtual void Logv(const char* format, va_list ap);
}
打印日志的过程使用了变长参数,这是日志的基本需求。值得注意的是,对于写日志整个过程没有使用额外的并发控制,主要是因为标准IO库<stdio>提供的读写函数都是线程安全的,多个线程可以对同一个FILE同时调用fwrite()。 每次写完一条日志后,立即调用fflush()将缓存刷到操作系统,但是并没有调用fsync来落盘。
网友评论