Skip to content

feat(cubestore): Rockstore - optimize index scanning #9728

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 1, 2025

Conversation

ovr
Copy link
Member

@ovr ovr commented Jun 30, 2025

image

queue_add queues:1, size:64 kb/512:
time: [644.70 ms 648.88 ms 653.45 ms] -> [547.18 ms 558.58 ms 571.78 ms]

queue_add queues:1, size:256 kb/512
time: [334.21 ms 336.78 ms 339.56 ms] -> [286.95 ms 291.70 ms 297.12 ms]

queue_add queues:1, size:512 kb/512
time: [363.73 ms 367.60 ms 371.73 ms] -> [304.42 ms 310.45 ms 317.43 ms]

before

queue_add queues:1, size:64 kb/512
                        time:   [644.70 ms 648.88 ms 653.45 ms]
                        change: [-4.3588% -2.6419% -0.9566%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

Benchmarking queue_add queues:1, size:256 kb/512: Warming up for 3.0000 s
queue_add queues:1, size:256 kb/512
                        time:   [334.21 ms 336.78 ms 339.56 ms]
                        change: [-0.7991% +0.4350% +1.7476%] (p = 0.50 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe

Benchmarking queue_add queues:1, size:512 kb/512: Warming up for 3.0000 s
queue_add queues:1, size:512 kb/512
                        time:   [363.73 ms 367.60 ms 371.73 ms]
                        change: [+2.4181% +3.7749% +5.1900%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

Benchmarking queue_list status_filter: Some(Pending) queues:5, size:128 kb, per_queue:1000/128: Warming up for 3.0000 s
Benchmarking queue_list status_filter: Some(Pending) queues:5, size:128 kb, per_queue:1000/128: Collecting 100 samples in estimated 5.queue_list status_filter: Some(Pending) queues:5, size:128 kb, per_queue:1000/128
                        time:   [967.02 µs 985.48 µs 1.0076 ms]
                        change: [-8.7331% -5.5801% -2.3738%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild
  5 (5.00%) high severe

Benchmarking queue_list status_filter: Some(Active) queues:5, size:128 kb, per_queue:1000/128: Warming up for 3.0000 s
Benchmarking queue_list status_filter: Some(Active) queues:5, size:128 kb, per_queue:1000/128: Collecting 100 samples in estimated 5.6queue_list status_filter: Some(Active) queues:5, size:128 kb, per_queue:1000/128
                        time:   [960.16 µs 976.31 µs 995.10 µs]
                        change: [-5.0910% -0.4810% +4.6747%] (p = 0.84 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe

queue_get queues:5, size:128 kb, per_queue:10000/128
                        time:   [9.9325 ms 10.060 ms 10.209 ms]
                        change: [-5.4308% -3.1552% -0.8813%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  6 (6.00%) high mild
  2 (2.00%) high severe

after

Benchmarking queue_add queues:1, size:64 kb/512: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 28.3s, or reduce sample count to 10.
queue_add queues:1, size:64 kb/512
                        time:   [546.95 ms 553.24 ms 560.41 ms]
                        change: [-18.034% -16.633% -15.105%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  6 (6.00%) high mild
  5 (5.00%) high severe

Benchmarking queue_add queues:1, size:256 kb/512: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 33.2s, or reduce sample count to 10.
queue_add queues:1, size:256 kb/512
                        time:   [291.96 ms 298.85 ms 306.33 ms]
                        change: [-39.325% -34.710% -29.651%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  10 (10.00%) high mild
  2 (2.00%) high severe

Benchmarking queue_add queues:1, size:512 kb/512: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 63.7s, or reduce sample count to 10.

@ovr ovr requested a review from a team as a code owner June 30, 2025 14:37
@ovr ovr force-pushed the chore/optimize-rocksdb-scans branch from 0f5bcc9 to 91bde3e Compare June 30, 2025 14:41
@ovr ovr changed the title feat(cubestore): Rockstore - optize index scanning feat(cubestore): Rockstore - optimize index scanning Jun 30, 2025
@ovr ovr force-pushed the chore/optimize-rocksdb-scans branch from 91bde3e to 6a8a5d9 Compare June 30, 2025 16:20
@ovr ovr merged commit 7987b75 into master Jul 1, 2025
68 of 71 checks passed
@ovr ovr deleted the chore/optimize-rocksdb-scans branch July 1, 2025 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants