ENH: Buffer API interop - data management update

[samir-nasibli]

Description

The main idea of this PR is to enable host data management without the numpy build dependency. This PR includes changes from PR #1568.

TODO

• Rebase the PR with the latest changes from the main branch.
• Discuss the necessity of these changes.

[samir-nasibli]

/intelci: run

[icfaust]

@syakov-intel I believe that numpy is too integrated into the codebase at a low level (including in serialization/deserialization) that the buffer interop is no longer necessary. I'd like to get @david-cortes-intel @Vika-F 's thoughts on this during my vacation and on #1568 to see if both can be closed.

[david-cortes-intel]

Am I understanding it correctly that the intention is to remove the build-time dependency on NumPy?

I see the initial work was from 2023, and perhaps back then having no numpy dependency would have brought some benefits, but I don't think that matters anymore after the release of NumPy 2.0 which brings a stable ABI that doesn't change across versions.

[david-cortes-intel]

One thing I notice here though: the initial PR was adding support for a chunked array class, which is missing here.

I guess it might have to do with supporting additional formats such as arrow, but the PR didn't add any tests for new packages. Is that somehow connected with the other changes?