Skip to content

CLAUDE.md

Latest commit

 

History

History
505 lines (398 loc) · 23.1 KB

CLAUDE.md

File metadata and controls

505 lines (398 loc) · 23.1 KB

CLAUDE.md - Dawarich Development Guide

This file contains essential information for Claude to work effectively with the Dawarich codebase.

Project Overview

Dawarich is a self-hostable web application built with Ruby on Rails 8.0 that serves as a replacement for Google Timeline (Google Location History). It allows users to track, visualize, and analyze their location data through an interactive web interface.

Key Features

  • Location history tracking and visualization
  • Interactive maps with multiple layers (heatmap, points, lines, fog of war)
  • Import from various sources (Google Maps Timeline, OwnTracks, Strava, GPX, GeoJSON, photos)
  • Export to GeoJSON and GPX formats
  • Statistics and analytics (countries visited, distance traveled, etc.)
  • Public sharing of monthly statistics with time-based expiration
  • Trips management with photo integration
  • Areas and visits tracking
  • Integration with photo management systems (Immich, Photoprism)

Technology Stack

Backend

  • Framework: Ruby on Rails 8.0
  • Database: PostgreSQL with PostGIS extension
  • Background Jobs: Sidekiq with Redis
  • Authentication: Devise
  • Authorization: Pundit
  • API Documentation: rSwag (Swagger)
  • Monitoring: Prometheus, Sentry
  • File Processing: AWS S3 integration

Frontend

  • CSS Framework: Tailwind CSS with DaisyUI components
  • JavaScript: Stimulus, Turbo Rails, Hotwired
  • Maps: Leaflet.js
  • Charts: Chartkick

Conventions

  • Enums over strings: Prefer Rails enums (integer columns) over string columns for status/type fields. Use enum :field_name, { ... }, prefix: :field_name to get scoped predicate methods and avoid name collisions.
  • Turbo first: Follow Rails 8 conventions — use Turbo Frames and Turbo Streams/broadcasts wherever appropriate to avoid full page reloads and provide smooth, in-place UI updates.
  • SVGs as files: Never inline SVG markup in views. Instead, save SVGs to app/assets/svg/icons and use inline_svg_tag "name.svg" to render them. This keeps views clean and SVGs reusable. Use rails_icons to manage SVG assets and ensure consistent styling.

Code Style

  • Follow rubocop conventions (see .rubocop.yml)
  • Rails defaults: convention over configuration
  • Prefer Hotwire (Turbo Frames/Streams + Stimulus) over custom JS
  • Use importmap for JS dependencies — no npm/yarn

Key Gems

  • activerecord-postgis-adapter - PostgreSQL PostGIS support
  • geocoder - Geocoding services
  • rgeo - Ruby Geometric Library
  • gpx - GPX file processing
  • parallel - Parallel processing
  • sidekiq - Background job processing
  • chartkick - Chart generation

Project Structure

├── app/
│   ├── controllers/     # Rails controllers
│   ├── models/         # ActiveRecord models with PostGIS support
│   ├── views/          # ERB templates
│   ├── services/       # Business logic services
│   ├── jobs/           # Sidekiq background jobs
│   ├── queries/        # Database query objects
│   ├── policies/       # Pundit authorization policies
│   ├── serializers/    # API response serializers
│   ├── javascript/     # Stimulus controllers and JS
│   └── assets/         # CSS and static assets
├── config/             # Rails configuration
├── db/                 # Database migrations and seeds
├── docker/             # Docker configuration
├── spec/               # RSpec test suite
└── swagger/            # API documentation

Core Models

Primary Models

  • User: Authentication and user management
  • Point: Individual location points with coordinates and timestamps
  • Track: Collections of related points forming routes
  • Area: Geographic areas drawn by users
  • Visit: Detected visits to areas
  • Trip: User-defined travel periods with analytics
  • Import: Data import operations
  • Export: Data export operations
  • Stat: Calculated statistics and metrics with public sharing capabilities

Geographic Features

  • Uses PostGIS for advanced geographic queries
  • Implements distance calculations and spatial relationships
  • Supports various coordinate systems and projections

Development Environment

Setup

  1. Docker Development: Use docker-compose -f docker/docker-compose.yml up
  2. DevContainer: VS Code devcontainer support available
  3. Local Development:
    • bundle exec rails db:prepare
    • bundle exec sidekiq (background jobs)
    • bundle exec bin/dev (main application)

Default Credentials

  • Username: demo@dawarich.app
  • Password: password

Testing

Test Suite

  • Framework: RSpec
  • System Tests: Capybara + Selenium WebDriver
  • E2E Tests: Playwright
  • Coverage: SimpleCov
  • Factories: FactoryBot
  • Mocking: WebMock

Test Commands

bundle exec rspec                    # Run all specs
bundle exec rspec spec/models/       # Model specs only
npx playwright test                  # E2E tests

Testing Best Practices — Test Behavior, Not Implementation

When writing or modifying tests, always test observable behavior (return values, state changes, side effects) rather than implementation details (which internal methods are called, in what order, with what exact arguments).

Anti-patterns to AVOID:

  1. Never mock the object under test — allow(subject).to receive(:internal_method) makes the test a tautology
  2. Never test private methods via send() — test through the public interface instead; if creating a user triggers a trial, test by creating the user and checking user.trial?, not by calling user.send(:start_trial)
  3. Never use receive_message_chain — allow(x).to receive_message_chain(:a, :b, :c) breaks on any scope reorder; use real data instead
  4. Avoid over-stubbing — if every collaborator is mocked, the test proves nothing; mock only at external boundaries (HTTP, geocoder, external APIs)
  5. Don't test wiring without outcomes — expect(Service).to receive(:new).with(args) only proves a method was called, not that it works; verify the returned data or state change instead
  6. Prefer have_enqueued_job over expect(Job).to receive(:perform_later) — the former tests real ActiveJob integration; the latter just tests a mock
  7. Don't assert on cache key formats or internal metric JSON shapes — test that caching works (2nd call doesn't requery) or that metrics fire, not exact internal formats
  8. Use real factory data over allow(user).to receive(:active?).and_return(true) — set the actual user state: create(:user, status: :active)

Good test pattern:

# Test behavior: creating an export enqueues processing
it 'enqueues processing job' do
  expect { create(:export, file_type: :points) }.to have_enqueued_job(ExportJob)
end

Bad test pattern:

# Tests implementation: mocks the callback interaction
it 'enqueues processing job' do
  expect(ExportJob).to receive(:perform_later)  # mock, not real
  build(:export).save!
end

Background Jobs

Sidekiq Jobs

  • Import Jobs: Process uploaded location data files
  • Calculation Jobs: Generate statistics and analytics
  • Notification Jobs: Send user notifications
  • Photo Processing: Extract EXIF data from photos

Key Job Classes

  • Tracks::ParallelGeneratorJob - Generate track data in parallel
  • Various import jobs for different data sources
  • Statistical calculation jobs

Public Sharing System

Overview

Dawarich includes a comprehensive public sharing system that allows users to share their monthly statistics with others without requiring authentication. This feature enables users to showcase their location data while maintaining privacy control through configurable expiration settings.

Key Features

  • Time-based expiration: Share links can expire after 1 hour, 12 hours, 24 hours, or be permanent
  • UUID-based access: Each shared stat has a unique, unguessable UUID for security
  • Public API endpoints: Hexagon map data can be accessed via API without authentication when sharing is enabled
  • Automatic cleanup: Expired shares are automatically inaccessible
  • Privacy controls: Users can enable/disable sharing and regenerate sharing URLs at any time

Technical Implementation

  • Database: sharing_settings (JSONB) and sharing_uuid (UUID) columns on stats table
  • Routes: /shared/month/:uuid for public viewing, /stats/:year/:month/sharing for management
  • API: /api/v1/maps/hexagons supports public access via uuid parameter
  • Controllers: Shared::StatsController handles public views, sharing management integrated into existing stats flow

Security Features

  • No authentication bypass: Public sharing only exposes specifically designed endpoints
  • UUID-based access: Sharing URLs use unguessable UUIDs rather than sequential IDs
  • Expiration enforcement: Automatic expiration checking prevents access to expired shares
  • Limited data exposure: Only monthly statistics and hexagon data are publicly accessible

Usage Patterns

  • Social sharing: Users can share interesting travel months with friends and family
  • Portfolio/showcase: Travel bloggers and photographers can showcase location statistics
  • Data collaboration: Researchers can share aggregated location data for analysis
  • Public demonstrations: Demo instances can provide public examples without compromising user data

API Documentation

  • Framework: rSwag (Swagger/OpenAPI)
  • Location: /api-docs endpoint
  • Authentication: API key (Bearer) for API access, UUID-based access for public shares

Database Schema

Key Tables

  • users - User accounts and settings
  • points - Location points with PostGIS geometry
  • tracks - Route collections
  • areas - User-defined geographic areas
  • visits - Detected area visits
  • trips - Travel periods
  • imports/exports - Data transfer operations
  • stats - Calculated metrics with sharing capabilities (sharing_settings, sharing_uuid)

PostGIS Integration

  • Extensive use of PostGIS geometry types
  • Spatial indexes for performance
  • Geographic calculations and queries

Configuration

Environment Variables

See .env.template for available configuration options including:

  • Database configuration
  • Redis settings
  • AWS S3 credentials
  • External service integrations
  • Feature flags

Key Config Files

  • config/database.yml - Database configuration
  • config/sidekiq.yml - Background job settings
  • config/schedule.yml - Cron job schedules
  • docker/docker-compose.yml - Development environment

Deployment

Docker

  • Production: docker/docker-compose.production.yml
  • Development: docker/docker-compose.yml
  • Multi-stage Docker builds supported

Procfiles

  • Procfile - Production Heroku deployment
  • Procfile.dev - Development with Foreman
  • Procfile.production - Production processes

Code Quality

Tools

  • Ruby Linting: RuboCop with Rails extensions
  • JS/CSS Linting: Biome (formatting, lint, import sorting)
  • Security: Brakeman, bundler-audit
  • Dependencies: Strong Migrations for safe database changes
  • Performance: Stackprof for profiling

Commands

bundle exec rubocop                  # Ruby linting
npx @biomejs/biome check --write .   # JS/CSS auto-fix (safe fixes)
npx @biomejs/biome check --write --unsafe .  # JS/CSS auto-fix (all fixes)
npx @biomejs/biome ci .              # JS/CSS CI check (read-only)
bundle exec brakeman                 # Security scan
bundle exec bundle-audit             # Dependency security

Lint Rules

  • Always run RuboCop on modified Ruby files before committing: bundle exec rubocop <files>
  • Always run Biome on modified JS/CSS files before committing: npx @biomejs/biome check --write <files>
  • If Biome --write leaves remaining errors, use --write --unsafe to apply fixes like parseInt radix and Number.isNaN
  • CI runs biome ci --changed --since=dev — it compares against the dev branch, not master
  • The noStaticOnlyClass warning is acceptable and does not fail CI
  • Tailwind CSS files (*.tailwind.css) have @import position rules disabled in biome.json because @tailwind directives must come first

Frontend: Hotwire-First Approach

Always prefer Turbo + Stimulus over custom JavaScript. This project uses the Hotwire stack (Turbo Drive, Turbo Frames, Turbo Streams, Stimulus) as its primary frontend architecture. Direct fetch() calls, manual DOM manipulation, and standalone JS modules should only be used when Hotwire cannot handle the use case (e.g., map rendering with Leaflet/MapLibre).

Decision Hierarchy

When adding frontend behavior, follow this order of preference:

  1. Turbo Drive — Default. Links and forms work as SPAs with zero JS.
  2. Turbo Frames — Partial page updates. Wrap a section in <turbo-frame> and target it from links/forms.
  3. Turbo Streams — Server-pushed DOM updates. Use for CRUD operations that need to update multiple page sections. Respond with turbo_stream format from controllers.
  4. Stimulus controller — Client-side behavior that Turbo can't handle (toggles, form validation, UI interactions). Keep controllers thin.
  5. Direct JS — Last resort. Only for complex map interactions, canvas rendering, or third-party library integration (Leaflet, MapLibre, Chartkick).

Turbo Stream Responses

For CRUD actions (create, update, destroy), respond with Turbo Streams instead of redirects or JSON:

# Controller
def create
  @area = current_user.areas.new(area_params)
  if @area.save
    respond_to do |format|
      format.turbo_stream
      format.html { redirect_to areas_path }
    end
  end
end

# app/views/areas/create.turbo_stream.erb
<%= turbo_stream.prepend "areas-list", partial: "areas/area", locals: { area: @area } %>
<%= stream_flash(:notice, "Area created successfully") %>

Use the FlashStreamable concern (included in controllers) to send flash messages via Turbo Streams:

include FlashStreamable

# In turbo_stream responses:
stream_flash(:notice, "Success message")
stream_flash(:error, "Error message")

Flash Messages

  • Server-side (Turbo Stream): Use stream_flash from the FlashStreamable concern. This appends a flash partial to the #flash-messages container.
  • Client-side (Stimulus/JS): Import Flash from flash_controller.js and call Flash.show(type, message): 
    import Flash from "./flash_controller"
Flash.show("notice", "Operation completed")
Flash.show("error", "Something went wrong")
  • Never use raw alert(), console.log for user-facing messages, or create ad-hoc notification DOM elements.

Stimulus Controllers

  • Location: app/javascript/controllers/
  • Naming: <name>_controller.js maps to data-controller="<name>" in HTML
  • Use static targets for DOM references, static values for data from HTML attributes
  • Always clean up in disconnect() (event listeners, timers, subscriptions)
  • Prefer data-action attributes in HTML over addEventListener in JS
  • For forms, prefer this.formTarget.requestSubmit() over manual fetch() calls — this preserves Turbo form handling, CSRF tokens, and Turbo Stream responses

File Uploads

Use the unified upload controller (upload_controller.js) for all file upload forms. Configure via data-upload-*-value attributes:

<%= form_with data: {
  controller: "upload",
  upload_url_value: rails_direct_uploads_url,
  upload_field_name_value: "import[files][]",
  upload_multiple_value: true,
  upload_target: "form"
} do |f| %>

What NOT to Do

  • No fetch() for form submissions — Use form_with with Turbo. If you need custom headers (API key), use Stimulus to submit the form via requestSubmit().
  • No document.getElementById() for updates — Use Turbo Frames/Streams to replace DOM sections server-side.
  • No showFlashMessage() or ad-hoc flash functions — Use Flash.show() (client) or stream_flash (server).
  • No ActionCable subscriptions for CRUD updates — Use Turbo Stream broadcasts from models/controllers instead.
  • No separate upload controllers per form — Use the unified upload controller with value attributes for configuration.

When Direct JS Is Acceptable

  • Map rendering: Leaflet (Maps v1) and MapLibre GL JS (Maps v2) require imperative JS for layers, markers, and interactions.
  • Chart rendering: Chartkick handles its own DOM.
  • Third-party integrations: Libraries that don't have Hotwire adapters.
  • Complex client-side computation: Haversine distance, coordinate transforms, etc.

Even in these cases, wrap the integration in a Stimulus controller and connect it to the DOM via data-controller.

Important Notes for Development

  1. Location Data: Always handle location data with appropriate precision and privacy considerations
  2. PostGIS: Leverage PostGIS features for geographic calculations rather than Ruby-based solutions 2.1 Coordinates: Use lonlat column in points table for geographic calculations
  3. Background Jobs: Use Sidekiq for any potentially long-running operations
  4. Testing: Include both unit and integration tests for location-based features
  5. Performance: Consider database indexes for geographic queries
  6. Security: Never log or expose user location data inappropriately
  7. Migrations: Put all migrations (schema and data) in db/migrate/, not db/data/. Data manipulation migrations use the same ActiveRecord::Migration class and should run in the standard migration sequence.
  8. Public Sharing: When implementing features that interact with stats, consider public sharing access patterns:
    • Use public_accessible? method to check if a stat can be publicly accessed
    • Support UUID-based access in API endpoints when appropriate
    • Respect expiration settings and disable sharing when expired
    • Only expose minimal necessary data in public sharing contexts

Route Drawing Implementation (Critical)

⚠️ IMPORTANT: Unit Mismatch in Route Splitting Logic

Both Map v1 (Leaflet) and Map v2 (MapLibre) contain an intentional unit mismatch in route drawing that must be preserved for consistency:

The Issue:

  • haversineDistance() function returns distance in kilometers (e.g., 0.5 km)
  • Route splitting threshold is stored and compared as meters (e.g., 500)
  • The code compares them directly: 0.5 > 500 = always FALSE

Result:

  • The distance threshold (meters_between_routes setting) is effectively disabled
  • Routes only split on time gaps (default: 60 minutes between points)
  • This creates longer, more continuous routes that users expect

Code Locations:

  • Map v1: app/javascript/maps/polylines.js:390

    • Uses haversineDistance() from maps/helpers.js (returns km)
    • Compares to distanceThresholdMeters variable (value in meters)

  • Map v2: app/javascript/maps_maplibre/layers/routes_layer.js:82-104

    • Has built-in haversineDistance() method (returns km)
    • Intentionally skips /1000 conversion to replicate v1 behavior
    • Comment explains this is matching v1's unit mismatch

Critical Rules:

  1. ❌ DO NOT "fix" the unit mismatch - this would break user expectations
  2. ✅ Keep both versions synchronized - they must behave identically
  3. ✅ Document any changes - route drawing changes affect all users
  4. ⚠️ If you ever fix this bug:
    • You MUST update both v1 and v2 simultaneously
    • You MUST migrate user settings (multiply existing values by 1000 or divide by 1000 depending on direction)
    • You MUST communicate the breaking change to users

Additional Route Drawing Details:

  • Time threshold: 60 minutes (default) - actually functional
  • Distance threshold: 500 meters (default) - currently non-functional due to unit bug
  • Sorting: Map v2 sorts points by timestamp client-side; v1 relies on backend ASC order
  • API ordering: Map v2 must request order: 'asc' to match v1's chronological data flow

Plan System (Lite vs Pro)

Dawarich Cloud has a two-tier plan system. Self-hosted instances bypass all plan restrictions (DawarichSettings.self_hosted? returns true, all users effectively have Pro).

Plans

  • Pro (plan: :pro, enum value 1) — Full access to all features, no data window
  • Lite (plan: :lite, enum value 0) — Free tier with restricted feature set

Plan is stored as an integer enum on the users table. New cloud users start on Lite via trial flow.

Lite Plan Restrictions

Data visibility window (12 months):

  • Lite users only see data from the last 12 months (DawarichSettings::LITE_DATA_WINDOW)
  • Implemented as a query-time filter in PlanScopable concern (app/models/concerns/plan_scopable.rb)
  • Scoped methods: scoped_points, scoped_tracks, scoped_visits, scoped_stats
  • Data is never deleted — only filtered from UI and API reads. Export uses unscoped user.points etc.
  • plan_restricted? returns true only when !self_hosted? && lite?

Disabled map layers (Pro-only):

  • Heatmap, Fog of War, Scratch Map, Globe View
  • Lite users get a 20-second timed preview, then auto-hide with upgrade prompt
  • Gating logic: app/javascript/maps_maplibre/utils/layer_gate.js
  • UI components: Toast (countdown) and UpgradeBanner (post-preview CTA)

API restrictions:

  • Write API returns 403 (require_write_api! in ApiController)
  • Read API scopes results to 12-month window (apply_plan_scope in ApiController)
  • Rate limit: 200 req/hr (Lite) vs 1,000 req/hr (Pro) via rack-attack (config/initializers/rack_attack.rb)

Disabled features:

  • Integrations (Immich, Photoprism)
  • Public sharing of stats
  • Full digest view

Plan endpoint: GET /api/v1/plan returns current plan and feature flags (Api::V1::PlanController)

Archival Warning System

Lite::ArchivalWarningJob runs daily for Lite users and sends warnings at three thresholds:

  1. 11 months — In-app notification warning data will archive in 30 days
  2. 11.5 months — Email notification
  3. 12 months — In-app notification that data has been archived (hidden from view)

Warnings are deduped via settings['archival_warnings'] JSONB on the user record.

Development Guidelines for Plan Gating

  • Use user.plan_restricted? to check if restrictions apply (returns false for self-hosted)
  • Use user.scoped_* methods instead of user.points/user.tracks etc. for plan-aware queries
  • Use require_pro_api! or require_write_api! before_actions in API controllers
  • Use apply_plan_scope(relation) when scoping points that don't start from user.points
  • Frontend: use isGatedPlan(userPlan) and gatedToggle() from layer_gate.js for map layer toggling
  • Export must always use unscoped relations — users can export all their data regardless of plan

Contributing

  • Main Branch: master
  • Development: dev branch for pull requests
  • Issues: GitHub Issues for bug reports
  • Discussions: GitHub Discussions for feature requests
  • Community: Discord server for questions

Resources