ai-sdk-provider: add Genie agent integration

[r-julien-bataille]

Summary

Adds a Databricks Genie integration to @databricks/ai-sdk-provider, exposing both:

• createDatabricksGenieConversationClient(settings) — a typed client over the Databricks Genie Conversation REST API.
• createDatabricksGenieAgent(settings) — a Vercel AI SDK Agent wrapper that turns a Genie space into a streaming agent compatible with streamText / generateText and the AI SDK UI helpers.

The agent surfaces Genie's intermediate states as reasoning-delta parts (e.g. Asking Ai, Executing Query) and emits the final answer as text, so it integrates cleanly with chat UIs that render reasoning streams. An optional debug setting logs requests/responses with Authorization, Cookie, and Set-Cookie redacted.

Validation

Tested end-to-end against a real Databricks workspace + Genie space using vercel/chatbot as the host application. The chat UI:

• Streamed Genie status updates as reasoning, then the final natural-language answer.
• Continued conversations across follow-up turns, including resuming an existing conversation.
• Rendered SQL query attachments correctly as code blocks (via AI SDK message parts) and tabular query results as a table .

API coverage

Genie Conversation API — implemented and covered by unit + live smoke tests:

• start-conversation
• createMessage (follow-up turns)
• getMessage / polling until terminal status
• listConversations — verified live (used to enumerate prior conversations)
• listConversationMessages — verified live (used to resume a conversation)
• deleteConversation — verified live
• getMessageAttachmentQueryResult (tabular results for query attachments)

Genie Conversation API — implemented, not yet tested:

• executeMessageAttachmentQuery (re-execute query attachment)
• generateDownloadFullQueryResult / getDownloadFullQueryResult (full result download)

Genie Space API — partially implemented, not tested:

• getSpace (with include_serialized_space) and getSampleQuestions are wired through but only exercised against mocked responses.

Known limitations

• Attachment ordering: when Genie returns multiple attachments on a single message — particularly follow-up messages that mix text, query, and suggested_questions — the order in which they are emitted from streamConversation does not always match the order rendered in the Databricks UI. The current implementation emits attachments in the order Genie returns them in the API payload, which doesn't appear to carry an explicit display index. Suggestions on the canonical ordering signal would be welcome.

Tests

• Unit tests: pnpm --filter @databricks/ai-sdk-provider test
• Live smoke test against a real Genie space: pnpm --filter @databricks/ai-sdk-provider test:genie:live (requires DATABRICKS_HOST, DATABRICKS_TOKEN, DATABRICKS_GENIE_SPACE_ID)
• End-to-end integration with vercel/chatbot: conversation start, follow-up turns, reasoning stream, SQL rendered as code, query results rendered as a table, resume from listConversationMessages, delete conversation