Skip to content

fix: persist bitmap index valid bitset for nullable array fields#49008

Merged
sre-ci-robot merged 3 commits intomilvus-io:masterfrom
SpadeA-Tang:fix-bitmap-validbitset
Apr 16, 2026
Merged

fix: persist bitmap index valid bitset for nullable array fields#49008
sre-ci-robot merged 3 commits intomilvus-io:masterfrom
SpadeA-Tang:fix-bitmap-validbitset

Conversation

@SpadeA-Tang
Copy link
Copy Markdown
Contributor

issue: #48901

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
@sre-ci-robot sre-ci-robot added the size/L Denotes a PR that changes 100-499 lines. label Apr 14, 2026
@mergify mergify bot added dco-passed DCO check passed. kind/bug Issues or changes related a bug labels Apr 14, 2026
@sre-ci-robot
Copy link
Copy Markdown
Contributor

[ci-v2-notice]
Notice: New ci-v2 system is enabled for this PR.

To rerun ci-v2 checks, comment with:

  • /ci-rerun-code-check // for ci-v2/code-check
  • /ci-rerun-build // for ci-v2/build
  • /ci-rerun-build-all // for ci-v2/build-all (multi-arch builds)
  • /ci-rerun-buildenv // for ci-v2/build-env (build milvus-env builder images)
  • /ci-rerun-ut-integration // for ci-v2/ut-integration, will rerun ci-v2/build
  • /ci-rerun-ut-go // for ci-v2/ut-go, will rerun ci-v2/build
  • /ci-rerun-ut-cpp // for ci-v2/ut-cpp
  • /ci-rerun-ut // for all ci-v2/ut-integration, ci-v2/ut-go, ci-v2/ut-cpp, will rerun ci-v2/build
  • /ci-rerun-e2e-default // for ci-v2/e2e-default
  • /ci-rerun-e2e-amd // for ci-v2/e2e-amd (e2e pool dispatcher)
  • /ci-rerun-build-ut-cov // for ci-v2/build-ut-cov (build + unit tests in one pipeline)
  • /ci-rerun-gosdk // for ci-v2/go-sdk (Go SDK E2E tests, ARM)

If you have any questions or requests, please contact @zhikunyao.

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
size_t valid_bitset_size = (total_num_rows_ + 7) / 8;
std::shared_ptr<uint8_t[]> valid_bitset_data(
new uint8_t[valid_bitset_size]);
memset(valid_bitset_data.get(), 0, valid_bitset_size);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should be able to directly copy from valid_bitset_.data(), instead of copy bit by bit?

Copy link
Copy Markdown
Contributor Author

@SpadeA-Tang SpadeA-Tang Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can copy directly. But in this way, we assume that the underlying format of TargetBitmap would not change. Otherwise, the format will be corrupt.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, that is fair. we may migrate to roaring one day

"bitmap valid_bitset size mismatch, expect {}, got {}",
expected_size,
data_size);
valid_bitset_ = TargetBitmap(total_num_rows_, false);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we memcpy into valid_bitset_?

@sre-ci-robot sre-ci-robot added the low-code-coverage add test-label from zhikun, diff coverage > 80% label Apr 14, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 14, 2026

Codecov Report

❌ Patch coverage is 91.80328% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.96%. Comparing base (f3aacd1) to head (294f67d).
⚠️ Report is 22 commits behind head on master.

Files with missing lines Patch % Lines
internal/core/src/index/BitmapIndexArrayTest.cpp 93.43% 9 Missing ⚠️
internal/core/src/index/BitmapIndex.cpp 86.95% 6 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #49008      +/-   ##
==========================================
- Coverage   77.97%   77.96%   -0.01%     
==========================================
  Files        2167     2167              
  Lines      356240   356431     +191     
==========================================
+ Hits       277783   277909     +126     
- Misses      69915    69959      +44     
- Partials     8542     8563      +21     
Components Coverage Δ
Client 79.25% <ø> (ø)
Core 84.51% <91.80%> (+0.06%) ⬆️
Go 76.22% <48.64%> (-0.05%) ⬇️
Files with missing lines Coverage Δ
internal/core/src/index/BitmapIndex.h 62.02% <ø> (ø)
internal/core/src/index/BitmapIndex.cpp 85.44% <86.95%> (+8.81%) ⬆️
internal/core/src/index/BitmapIndexArrayTest.cpp 89.21% <93.43%> (+3.41%) ⬆️

... and 36 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: SpadeA <tangchenjie1210@gmail.com>
@sre-ci-robot sre-ci-robot added low-code-coverage add test-label from zhikun, diff coverage > 80% and removed low-code-coverage add test-label from zhikun, diff coverage > 80% labels Apr 14, 2026
@sre-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: SpadeA-Tang, zhengbuqian

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@SpadeA-Tang
Copy link
Copy Markdown
Contributor Author

/ci-rerun-e2e-default

@mergify mergify bot added the ci-passed label Apr 16, 2026
@zhagnlu
Copy link
Copy Markdown
Contributor

zhagnlu commented Apr 16, 2026

/lgtm

@sre-ci-robot sre-ci-robot merged commit ddd1409 into milvus-io:master Apr 16, 2026
18 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved ci-passed dco-passed DCO check passed. kind/bug Issues or changes related a bug lgtm size/L Denotes a PR that changes 100-499 lines.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants