Skip to content

Commit 31c524c

Browse files
nimarbnimarb
committed
restore 2
1 parent 649da3a commit 31c524c

1 file changed

Lines changed: 72 additions & 132 deletions

File tree

‎CLAUDE.md‎

Lines changed: 72 additions & 132 deletions
Original file line numberDiff line numberDiff line change
@@ -4,175 +4,115 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
44

55
## Project Overview
66

7-
This is the **Langfuse Python SDK**, an observability and analytics platform for AI applications. It provides tracing, evaluation, and analytics for LLM applications through multiple interfaces: decorators, low-level SDK, and integrations with popular AI libraries.
7+
This is the Langfuse Python SDK, a client library for accessing the Langfuse observability platform. The SDK provides integration with OpenTelemetry (OTel) for tracing, automatic instrumentation for popular LLM frameworks (OpenAI, Langchain, etc.), and direct API access to Langfuse's features.
88

99
## Development Commands
1010

11-
### Environment Setup
11+
### Setup
1212
```bash
13-
# Using UV (preferred)
14-
uv venv --python 3.12
15-
source .venv/bin/activate
16-
uv sync
13+
# Install Poetry plugins (one-time setup)
14+
poetry self add poetry-dotenv-plugin
15+
poetry self add poetry-bumpversion
1716

18-
# Using Poetry (legacy)
17+
# Install all dependencies including optional extras
1918
poetry install --all-extras
19+
20+
# Setup pre-commit hooks
2021
poetry run pre-commit install
2122
```
2223

2324
### Testing
2425
```bash
25-
# Run all tests
26+
# Run all tests with verbose output
2627
poetry run pytest -s -v --log-cli-level=INFO
2728

28-
# Run specific test
29+
# Run a specific test
2930
poetry run pytest -s -v --log-cli-level=INFO tests/test_core_sdk.py::test_flush
3031

31-
# Run with UV
32-
uv run pytest -s -v --log-cli-level=INFO
32+
# Run tests in parallel (faster)
33+
poetry run pytest -s -v --log-cli-level=INFO -n auto
3334
```
3435

35-
### Memory for Running Unit Tests
36-
- To run unit tests you must always use the env file, use: `UV_ENV_FILE=.env uv run pytest -s -v --log-cli-level=INFO tests/TESTFILE::TEST_NAME`
37-
3836
### Code Quality
3937
```bash
40-
# Format code
41-
ruff format .
38+
# Format code with Ruff
39+
poetry run ruff format .
40+
41+
# Run linting (development config)
42+
poetry run ruff check .
4243

43-
# Run linter (development config)
44-
ruff check .
44+
# Run type checking
45+
poetry run mypy .
4546

46-
# Run linter (CI config)
47-
ruff check --config ci.ruff.toml .
47+
# Run pre-commit hooks manually
48+
poetry run pre-commit run --all-files
4849
```
4950

50-
### Documentation
51+
### Building and Releasing
5152
```bash
52-
# Generate SDK reference docs
53-
poetry run pdoc -o docs/ --docformat google --logo "https://langfuse.com/langfuse_logo.svg" langfuse
53+
# Build the package
54+
poetry build
5455

55-
# Serve docs locally
56-
poetry run pdoc --docformat google --logo "https://langfuse.com/langfuse_logo.svg" langfuse
56+
# Run release script (handles versioning, building, tagging, and publishing)
57+
poetry run release
58+
59+
# Generate documentation
60+
poetry run pdoc -o docs/ --docformat google --logo "https://langfuse.com/langfuse_logo.svg" langfuse
5761
```
5862

59-
## Architecture Overview
63+
## Architecture
6064

6165
### Core Components
6266

63-
**Main SDK Client** (`langfuse/_client/client.py`)
64-
- Built on OpenTelemetry foundations
65-
- Provides span management for tracing (LangfuseSpan, LangfuseGeneration, LangfuseEvent)
66-
- Thread-safe singleton pattern with multi-project support
67-
- Handles both sync and async operations
68-
69-
**Resource Manager** (`langfuse/_client/resource_manager.py`)
70-
- Central coordination hub implementing thread-safe singleton
71-
- Manages OpenTelemetry setup, API clients, background workers
72-
- Handles media upload and score ingestion queues
73-
- Provides graceful shutdown and resource cleanup
74-
75-
**API Layer** (`langfuse/api/`)
76-
- Auto-generated from OpenAPI spec using Fern
77-
- Provides complete typed client for Langfuse API
78-
- Organized by resources (prompts, datasets, observations, etc.)
79-
80-
### SDK Interfaces
81-
82-
**Three primary interaction patterns:**
83-
84-
1. **Decorator Pattern** (`@observe`)
85-
```python
86-
@observe(as_type="generation")
87-
def my_llm_function():
88-
# Automatically traced
89-
```
90-
91-
2. **Low-level Client API**
92-
```python
93-
langfuse = Langfuse()
94-
span = langfuse.start_span(name="operation")
95-
```
96-
97-
3. **Integration Libraries**
98-
```python
99-
from langfuse.openai import openai # Drop-in replacement
100-
```
101-
102-
### Integration Architecture
103-
104-
**OpenAI Integration** (`langfuse/openai.py`)
105-
- Drop-in replacement supporting both v0.x and v1.x
106-
- Wraps all completion methods (chat, completions, streaming, async)
107-
- Automatic metrics collection (tokens, cost, latency)
108-
109-
**LangChain Integration** (`langfuse/langchain/`)
110-
- CallbackHandler pattern for chain tracing
111-
- Automatic span creation for chains, tools, and agents
112-
- UUID mapping between LangChain runs and Langfuse spans
113-
114-
### Background Processing
115-
116-
**Task Manager** (`langfuse/_task_manager/`)
117-
- Media upload processing with configurable workers
118-
- Score ingestion with batching and retry logic
119-
- Queue-based architecture with backpressure handling
120-
121-
## Key Development Patterns
122-
123-
### Multi-Project Safety
124-
- Thread-safe singleton per public key prevents data leakage
125-
- Project-scoped span processor filters ensure data isolation
126-
- Disabled clients returned when project context is ambiguous
127-
128-
### OpenTelemetry Foundation
129-
- All tracing built on OTel primitives for standards compliance
130-
- Custom span processor for Langfuse-specific export
131-
- Proper context propagation across async boundaries
132-
133-
### Testing Approach
134-
- Comprehensive unit and integration tests
135-
- API mocking through test wrappers
136-
- Concurrency testing with ThreadPoolExecutor
137-
- Proper resource cleanup in test teardown
138-
139-
## File Structure Notes
140-
141-
- `langfuse/_client/` - Core client implementation and tracing logic
142-
- `langfuse/api/` - Auto-generated API client (excluded from linting)
143-
- `langfuse/_utils/` - Utility functions for serialization, error handling, etc.
144-
- `langfuse/_task_manager/` - Background processing workers
145-
- `tests/` - Comprehensive test suite with integration tests
146-
- `static/` - Test assets and sample files
67+
- **`langfuse/_client/`**: Main SDK implementation built on OpenTelemetry
68+
- `client.py`: Core Langfuse client with OTel integration
69+
- `span.py`: LangfuseSpan, LangfuseGeneration, LangfuseEvent classes
70+
- `observe.py`: Decorator for automatic instrumentation
71+
- `datasets.py`: Dataset management functionality
72+
73+
- **`langfuse/api/`**: Auto-generated Fern API client
74+
- Contains all API resources and types
75+
- Generated from OpenAPI spec - do not manually edit these files
76+
77+
- **`langfuse/_task_manager/`**: Background processing
78+
- Media upload handling and queue management
79+
- Score ingestion consumer
80+
81+
- **Integration modules**:
82+
- `langfuse/openai.py`: OpenAI instrumentation
83+
- `langfuse/langchain/`: Langchain integration via CallbackHandler
84+
85+
### Key Design Patterns
86+
87+
The SDK is built on OpenTelemetry for observability, using:
88+
- Spans for tracing LLM operations
89+
- Attributes for metadata (see `LangfuseOtelSpanAttributes`)
90+
- Resource management for efficient batching and flushing
91+
92+
The client follows an async-first design with automatic batching of events and background flushing to the Langfuse API.
14793

14894
## Configuration
14995

150-
### Environment Variables
151-
Key environment variables for development:
152-
- `LANGFUSE_PUBLIC_KEY` / `LANGFUSE_SECRET_KEY` - API credentials
153-
- `LANGFUSE_HOST` - Langfuse instance URL
154-
- `LANGFUSE_DEBUG` - Enable debug logging
155-
- `LANGFUSE_TRACING_ENABLED` - Enable/disable tracing
96+
Environment variables (defined in `_client/environment_variables.py`):
97+
- `LANGFUSE_PUBLIC_KEY` / `LANGFUSE_SECRET_KEY`: API credentials
98+
- `LANGFUSE_HOST`: API endpoint (defaults to https://cloud.langfuse.com)
99+
- `LANGFUSE_DEBUG`: Enable debug logging
100+
- `LANGFUSE_TRACING_ENABLED`: Enable/disable tracing
101+
- `LANGFUSE_SAMPLE_RATE`: Sampling rate for traces
156102

157-
### Test Configuration
158-
- Tests require `.env` file based on `.env.template`
159-
- Some E2E tests are skipped by default (remove decorators to run)
160-
- CI uses different ruff config (`ci.ruff.toml`)
103+
## Testing Notes
161104

162-
## Release Process
105+
- Create `.env` file based on `.env.template` for integration tests
106+
- E2E tests with external APIs (OpenAI, SERP) are typically skipped in CI
107+
- Remove `@pytest.mark.skip` decorators in test files to run external API tests
108+
- Tests use `respx` for HTTP mocking and `pytest-httpserver` for test servers
163109

164-
```bash
165-
# Automated release
166-
poetry run release
110+
## Important Files
167111

168-
# Manual release steps
169-
poetry version patch
170-
poetry build
171-
git commit -am "chore: release v{version}"
172-
git tag v{version}
173-
git push --tags
174-
poetry publish
175-
```
112+
- `pyproject.toml`: Poetry configuration, dependencies, and tool settings
113+
- `ruff.toml`: Local development linting config (stricter)
114+
- `ci.ruff.toml`: CI linting config (more permissive)
115+
- `langfuse/version.py`: Version string (updated by release script)
176116

177117
## API Generation
178118

0 commit comments

Comments
 (0)