facebook/rocksdb
1
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2026-03-20 06:24:22 +00:00
Code Issues Projects Releases Packages Wiki Activity
Page: What's new in RocksDB2.7
Pages
A Tutorial of RocksDB SST formats Administration and Data Access Tool Allocating Some Indexes and Bloom Filters using Huge Page TLB Approximate Size Articles about Rocks Asynchronous IO Atomic flush Background Error Handling Basic Operations Benchmarking tools BlobDB Block Cache Block cache analysis and simulation tools Building on Windows Checkpoints Choose Level Compaction Files Column Families Compaction Filter Compaction Stats and DB Status Compaction Trivial Move Compaction Compression Creating and Ingesting SST files CuckooTable Format Daily Off‐peak Time Option Data Block Hash Index Delete A Range Of Keys Delete Stale Files DeleteRange Implementation DeleteRange Developing with an IDE Dictionary Compression Direct IO EventListener External Table (Experimental) FIFO compaction style Features Not in LevelDB Full File Checksum and Checksum Handoff Fuzz Test Home How to ask a performance related question How to backup RocksDB How to persist in memory RocksDB database How we keep track of live SST files IO Tracer and Parser IO Implement Queue Service Using RocksDB Index Block Format Indexing SST Files for Better Lookup Performance Iterator Implementation Iterator JNI Debugging Journal Known Issues Leveled Compaction Logger Logging in RocksJava Low Priority Write MANIFEST Managing Disk Space Utilization Manual Compaction MemTable Memory usage in RocksDB Mempurge (Memtable Garbage Collection) [Experimental] Merge Operator Implementation Merge Operator Multi Column Family Iterator MultiGet Performance Object Registry Online Verification Open Projects Option String and Option Map Partitioned Index Filters Perf Context and IO Stats Context Performance Benchmark 2014 Performance Benchmark 201807 Performance Benchmarks October 2022 Performance Benchmarks Pipelined Write PlainTable Format Platform Requirements Prefix Seek Projects Being Developed Proposal: Unifying Level and Universal Compactions Proposals on Improving Rocksdb's Options Publication Rate Limiter Read Modify Write Benchmarks Read only and Secondary instances Reducing memcpy overhead when using Iterators Remote Compaction Replication Helpers RocksDB Bloom Filter RocksDB Compatibility Between Different Releases RocksDB Configurable Objects RocksDB Contribution Guide RocksDB Extensions RocksDB FAQ RocksDB In Memory Workload Performance Benchmarks RocksDB Options File RocksDB Overview RocksDB Public Communication and Information Channels RocksDB Release Methodology RocksDB Repairer RocksDB Trace, Replay, Analyzer, and Workload Generation RocksDB Troubleshooting Guide RocksDB Tuning Guide RocksDB Users and Use Cases RocksDB version macros RocksJava API TODO RocksJava Basics RocksJava Performance on Flash Storage Rocksdb Architecture Guide Rocksdb BlockBasedTable Format Rocksdb Table Format SST File Manager SST Partitioner SecondaryCache (Experimental) SeekForPrev Setup Options and Basic Tuning Simulation Cache Single Delete Slow Deletion Snapshot Space Tuning Speed Up DB Open Statistics Stress test Subcompaction Tailing Iterator Talks Terminology Tests The Customizable Class Third party language bindings This is a test Thread Pool Tiered Storage (Experimental) Tiered Storage Benchmarking Time to Live Track WAL in MANIFEST Transactions Tuning RocksDB from Java Tuning RocksDB on Spinning Disks Two Phase Commit Implementation Universal Compaction User defined Timestamp WAL Compression WAL Performance WAL Recovery Modes What's new in RocksDB2.7 Wide Columns Write Ahead Log (WAL) Write Ahead Log File Format Write Batch With Index Write Buffer Manager Write Stalls WritePrepared Transactions WriteUnprepared Transactions [To Be Deprecated] Persistent Read Cache log_format.txt poc test poc.html poc testpoc.html unordered_write
16 What's new in RocksDB2.7
Igor Canadi edited this page 2014-01-29 08:50:03 -08:00
Table of Contents

Public API changes

  • Renamed StackableDB::GetRawDB() to StackableDB::GetBaseDB(). The call returns the underlying DB* for a given StackableDB. Before, GetRawDB() returned the raw DB* by stripping away all StackableDB layers.
  • Renamed WriteBatch::Data() const std::string& Data() const.
  • Renamed class TableStats to TableProperties.
  • Deleted class PrefixHashRepFactory. Please use NewHashSkipListRepFactory() instead.
  • Supported multi-threaded EnableFileDeletions() and DisableFileDeletions() - b60c14f
  • Added DB::GetOptions() - 3ce3658
  • Added DB::GetDbIdentity() - 1880268
  • DB::CompactRange() now returns a status aba2acb

New features

  • Added BackupableDB
  • Implemented TailingIterator, a special type of iterator that doesn't create a snapshot (can be used to read newly inserted data) and is optimized for doing sequential reads.
  • Added property block for table, which allows (1) a table to store its metadata and (2) end user to collect and store properties they are interested in.
  • Enabled caching index and filter block in block cache (turned off by default).
  • Supported error report when doing manual compaction.
  • Supported additional Linux platform flavors and Mac OS.
  • Put with SliceParts - Variant of Put() that gathers output like writev(2) -- 8a46ecd
  • Bug fixes and code refactor for compatibility with upcoming Column Family feature.

Performance improvements

  • Huge benchmark performance improvements by multiple efforts. For example, increase in readonly QPS from about 530k in 2.6 release to 1.1 million in 2.7 [1]
  • Speeding up a way RocksDB deleted obsolete files - no longer listing the whole directory under a lock -- decrease in p99
  • Use raw pointer instead of shared pointer for statistics: 5b825d -- huge increase in performance -- shared pointers are slow
  • Optimized locking for get -- 1fdb3f -- 1.5x QPS increase for some workloads
  • Cache speedup - e8d40c3
  • Implemented autovector, which allocates first N elements on stack. Most of vectors in RocksDB are small. Also, we never want to allocate heap objects while holding a mutex. -- c01676e4
  • Lots of efforts to move malloc, memcpy and IO outside of locks

[1] You can see the exact parameters in rocksdb/build_tools/regression_build_test.sh