Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
9d9cd22
refactor: replace Astro frontend with Vite and Preact
gildesmarais Mar 14, 2026
d6f7fb3
fix: simplify result actions and surface feed errors
gildesmarais Mar 14, 2026
c91c3df
fix: accept escaped public feed tokens
gildesmarais Mar 14, 2026
d11b5d0
fix: label the result feed url field
gildesmarais Mar 14, 2026
bfc0df8
refactor: redesign frontend ui system
gildesmarais Mar 14, 2026
867b62d
style: refine frontend typography and visual noise
gildesmarais Mar 14, 2026
2e4c81c
chore: drop accidental typecheck gate changes
gildesmarais Mar 14, 2026
02f99ab
chore: add frontend typecheck quick-check
gildesmarais Mar 14, 2026
1bd9113
ci: add frontend guardrail checks
gildesmarais Mar 14, 2026
cb16c81
refactor: harden demo contract and simplify ui states
gildesmarais Mar 14, 2026
89abd53
chore: remove dead frontend components
gildesmarais Mar 14, 2026
eed6bc2
style: prettier
gildesmarais Mar 14, 2026
8beb194
chore: refresh generated openapi client
gildesmarais Mar 14, 2026
fa04c2f
chore: update npm deps
gildesmarais Mar 14, 2026
5eb27c3
Refactor feed creation UX flow
gildesmarais Mar 14, 2026
595e7be
Tighten feed result surface
gildesmarais Mar 14, 2026
43d6452
Refine create flow support surfaces
gildesmarais Mar 14, 2026
216352d
Refine feed creation workspace UX
gildesmarais Mar 14, 2026
b8b6ee8
Simplify feed creation page layout
gildesmarais Mar 14, 2026
4a8c1d1
Align result screen with simplified create flow
gildesmarais Mar 14, 2026
1185771
Refine feed preview and utility surfaces
gildesmarais Mar 14, 2026
c5dd861
Refine token prompt interaction flow
gildesmarais Mar 14, 2026
e8bd90c
Polish feed result presentation
gildesmarais Mar 14, 2026
3439bd0
Update frontend contract test for new flow
gildesmarais Mar 14, 2026
ba94bfb
Expose json feed URLs and tighten recovery flow
gildesmarais Mar 14, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
## Overview

- Ruby web app that converts websites into RSS 2.0 feeds.
- Built with **Roda** backend + **Astro** frontend, using the **html2rss** gem (+ `html2rss-configs`).
- **Frontend:** Modern Astro-based UI with component architecture, served alongside Ruby backend.
- Built with **Roda** backend + **Preact** frontend, using the **html2rss** gem (+ `html2rss-configs`).
- **Frontend:** Vite-built Preact UI, served alongside Ruby backend.

## Documentation website of core dependencies

Expand All @@ -14,10 +14,10 @@ Search these pages before using them. Find examples, plugins, UI components, and

1. https://roda.jeremyevans.net/documentation.html

### Astro & Starlight
### Preact & Vite

1. https://docs.astro.build/en/getting-started/
2. https://starlight.astro.build/getting-started/
1. https://preactjs.com/guide/v10/getting-started/
2. https://vite.dev/guide/

### html2rss

Expand All @@ -40,8 +40,8 @@ Fix rubocop `RSpec/MultipleExpectations` adding rspec tag `:aggregate_failures`.
- ✅ Validate all inputs. Pass outbound requests through **SSRF filter**.
- ✅ Add caching headers where appropriate (`Rack::Cache`).
- ✅ Errors: friendly messages for users, detailed logging internally.
- ✅ **Frontend**: Use Astro components in `frontend/src/`. Keep it simple.
- ✅ **CSS**: Use frontend styles provided by Astro Starlight.
- ✅ **Frontend**: Use Preact components in `frontend/src/`. Keep it simple.
- ✅ **CSS**: Use the app-owned frontend styles in `frontend/src/styles/`.
- ✅ **Specs**: RSpec for Ruby, build tests for frontend.
- ✅ When a spec needs to tweak environment variables, wrap the example in `ClimateControl.modify` so state is restored automatically.

Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,9 @@ jobs:
- name: Verify generated OpenAPI client is up to date
run: npm run openapi:verify

- name: Typecheck frontend
run: npm run typecheck

- name: Check formatting
run: npm run format:check

Expand Down Expand Up @@ -127,7 +130,7 @@ jobs:
run: npm ci
working-directory: frontend

- name: Build frontend
- name: Build frontend static assets
run: npm run build
working-directory: frontend

Expand Down Expand Up @@ -162,7 +165,7 @@ jobs:
run: npm ci
working-directory: frontend

- name: Build frontend
- name: Build frontend static assets
run: npm run build
working-directory: frontend

Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,6 @@ USER html2rss

COPY --from=builder /usr/local/bundle /usr/local/bundle
COPY --chown=$USER:$USER . /app
COPY --from=frontend-builder --chown=$USER:$USER /app/frontend/dist ./public/frontend
COPY --from=frontend-builder --chown=$USER:$USER /app/public/frontend ./public/frontend

CMD ["bundle", "exec", "puma", "-C", "./config/puma.rb"]
23 changes: 17 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# frozen_string_literal: true

.PHONY: help test lint lint-js lint-ruby lintfix lintfix-js lintfix-ruby setup dev clean frontend-setup ready yard-verify-public-docs openapi openapi-verify openapi-client openapi-client-verify openapi-lint openapi-lint-redocly openapi-lint-spectral openai-lint-spectral
.PHONY: help test lint lint-js lint-ruby lintfix lintfix-js lintfix-ruby setup dev clean frontend-setup check-frontend quick-check ready yard-verify-public-docs openapi openapi-verify openapi-client openapi-client-verify openapi-lint openapi-lint-redocly openapi-lint-spectral openai-lint-spectral

# Default target
help: ## Show this help message
Expand Down Expand Up @@ -28,7 +28,7 @@ dev: ## Start development server with live reload
dev-ruby: ## Start Ruby server only
@bin/dev-ruby

dev-frontend: ## Start Astro dev server only
dev-frontend: ## Start frontend dev server only
@cd frontend && npm run dev

test: ## Run all tests (Ruby + Frontend)
Expand All @@ -47,6 +47,10 @@ test-frontend-unit: ## Run frontend unit tests only
test-frontend-contract: ## Run frontend contract tests only
@cd frontend && npm run test:contract

check-frontend: ## Run frontend typecheck, format, and test checks
$(MAKE) lint-js
$(MAKE) test-frontend


lint: lint-ruby lint-js ## Run all linters (Ruby + Frontend) - errors when issues found
@echo "All linting complete!"
Expand All @@ -59,6 +63,8 @@ lint-ruby: ## Run Ruby linter (RuboCop) - errors when issues found
@echo "Ruby linting complete!"

lint-js: ## Run JavaScript/Frontend linter (Prettier) - errors when issues found
@echo "Running TypeScript typecheck..."
@cd frontend && npm run typecheck
@echo "Running Prettier format check..."
@cd frontend && npm run format:check
@echo "JavaScript linting complete!"
Expand All @@ -76,10 +82,15 @@ lintfix-js: ## Auto-fix JavaScript/Frontend linting issues
@cd frontend && npm run format
@echo "JavaScript lintfix complete!"

ready: ## Pre-commit gate (RuboCop + RSpec)
quick-check: ## Fast local checks (Ruby lint/docs + frontend format/typecheck)
@echo "Running quick checks..."
$(MAKE) lint-ruby
$(MAKE) lint-js
@echo "Quick checks complete!"

ready: ## Pre-commit gate (quick checks + RSpec)
@echo "Running pre-commit checks..."
bundle exec rubocop -F
bundle exec rake yard:verify_public_docs
$(MAKE) quick-check
bundle exec rspec
@echo "Pre-commit checks complete!"

Expand Down Expand Up @@ -111,7 +122,7 @@ openai-lint-spectral: openapi-lint-spectral ## Alias for openapi-lint-spectral

clean: ## Clean temporary files
@rm -rf tmp/rack-cache-* coverage/
@cd frontend && rm -rf dist/ .astro/ node_modules/
@cd frontend && rm -rf dist/ node_modules/
@echo "Clean complete!"

frontend-setup: ## Setup frontend dependencies
Expand Down
25 changes: 8 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# html2rss-web

html2rss-web converts arbitrary websites into RSS 2.0 feeds with a slim Ruby backend and an Astro-powered frontend.
html2rss-web converts arbitrary websites into RSS 2.0 feeds with a slim Ruby backend and a Preact frontend.

## Links

Expand All @@ -12,25 +12,16 @@ html2rss-web converts arbitrary websites into RSS 2.0 feeds with a slim Ruby bac

## Highlights

- Responsive Astro interface with gallery and custom feed creation.
- Responsive Preact interface for demo, sign-in, conversion, and result flows.
- Automatic source discovery with token-scoped permissions.
- Signed public feed URLs that work in standard RSS readers.
- Built-in SSRF defences, input validation, and HMAC-protected tokens.

## Architecture

- **Backend:** Ruby + Roda, backed by the `html2rss` gem for extraction.
- **Frontend:** Astro static site with progressive enhancement.
- **Frontend:** Preact app built with Vite into `public/frontend`.
- **Distribution:** Docker Compose by default; other deployments require manual wiring.

## Documentation

In-repo docs live under `frontend/src/content/docs/` and are published by Astro.

- [Configuration Guide](frontend/src/content/docs/configuration.md)
- [Security Guide](frontend/src/content/docs/security.md)
- [REST API v1](frontend/src/content/docs/api/v1.md)
- [Testing Overview](frontend/src/content/docs/testing.md)
- [v2 Migration Guide](docs/migrations/v2.md)

## REST API Snapshot
Expand Down Expand Up @@ -73,17 +64,17 @@ bundle exec rspec
make openapi
```

Dev URLs: Ruby app at `http://localhost:4000`, Astro dev server at `http://localhost:4001`.
Dev URLs: Ruby app at `http://localhost:4000`, frontend dev server at `http://localhost:4001`.

## Make Targets

| Command | Purpose |
| -------------------- | ------------------------------------------------------- |
| `make help` | List available shortcuts. |
| `make setup` | Install Ruby and Node dependencies. |
| `make dev` | Run Ruby (port 4000) and Astro (port 4001) dev servers. |
| `make dev` | Run Ruby (port 4000) and frontend (port 4001) dev servers. |
| `make dev-ruby` | Start only the Ruby server. |
| `make dev-frontend` | Start only the Astro dev server (port 4001). |
| `make dev-frontend` | Start only the frontend dev server (port 4001). |
| `make test` | Run Ruby and frontend test suites. |
| `make test-ruby` | Run Ruby specs. |
| `make test-frontend` | Run frontend unit and contract tests. |
Expand All @@ -105,8 +96,8 @@ The OpenAPI file is generated from Ruby request specs only.

| Command | Purpose |
| ----------------------- | --------------------------------------------- |
| `npm run dev` | Astro dev server with hot reload (port 4001). |
| `npm run build` | Production build. |
| `npm run dev` | Vite dev server with hot reload (port 4001). |
| `npm run build` | Build static assets into `public/frontend`. |
| `npm run test:run` | Unit tests (Vitest). |
| `npm run test:contract` | Contract tests with MSW. |

Expand Down
6 changes: 5 additions & 1 deletion app/api/v1/feeds.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# frozen_string_literal: true

require 'html2rss/url'

require_relative 'contract'
require_relative 'feeds/create_feed'
require_relative 'feeds/show_feed'
Expand Down Expand Up @@ -33,7 +35,9 @@ def create(request)
# @param url [String]
# @return [String, nil]
def extract_site_title(url)
CreateFeed.extract_site_title(url)
Html2rss::Url.for_channel(url).channel_titleized
rescue StandardError
nil
end
end
end
Expand Down
69 changes: 16 additions & 53 deletions app/api/v1/feeds/create_feed.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

require 'time'
require 'json'
require 'html2rss/url'

require_relative '../../../auth'
require_relative '../../../auto_source'
Expand All @@ -18,12 +17,11 @@ module Api
module V1
module Feeds
##
# Creates stable feed records from authenticated API requests.
#
# The implementation intentionally keeps parsing, authorization, and
# normalization in a single boundary object so callers can rely on one
# predictable contract instead of coordinating multiple services.
# Creates stable feed records from authenticated API requests with one predictable boundary contract.
module CreateFeed
FEED_ATTRIBUTE_KEYS =
%i[id name url strategy feed_token public_url json_public_url created_at updated_at].freeze

class << self
# Creates a feed and returns a normalized API success payload.
#
Expand All @@ -41,41 +39,19 @@ def call(request)
raise
end

# Extracts a best-effort human-readable title from the URL.
#
# @param url [String] target source URL.
# @return [String, nil] inferred title or nil when unavailable.
def extract_site_title(url)
Html2rss::Url.for_channel(url).channel_titleized
rescue StandardError
nil
end

private

# Enforces feature availability at the API edge to fail fast.
#
# @return [void]
def ensure_auto_source_enabled!
raise ForbiddenError, Contract::MESSAGES[:auto_source_disabled] unless AutoSource.enabled?
end

# Resolves the authenticated account from the request.
#
# @param request [Rack::Request]
# @return [Hash{Symbol=>Object}] authenticated account attributes.
def require_account(request)
account = Auth.authenticate(request)
raise UnauthorizedError, 'Authentication required' unless account

account
end

# Validates and normalizes feed creation parameters.
#
# @param params [Hash{String=>Object}] merged request parameters.
# @param account [Hash{Symbol=>Object}] authenticated account.
# @return [Html2rss::Web::BoundaryModels::FeedCreateParams]
def build_create_params(params, account)
url = params['url'].to_s.strip
raise BadRequestError, 'URL parameter is required' if url.empty?
Expand All @@ -89,10 +65,6 @@ def build_create_params(params, account)
)
end

# Normalizes a strategy value while preserving a default path.
#
# @param raw_strategy [String, nil]
# @return [String]
def normalize_strategy(raw_strategy)
strategy = raw_strategy.to_s.strip
strategy = default_strategy if strategy.empty?
Expand All @@ -112,24 +84,13 @@ def default_strategy
Html2rss::RequestService.default_strategy_name.to_s
end

# Shapes feed attributes into the stable API schema.
#
# @param feed_data [Html2rss::Web::BoundaryModels::FeedMetadata, Hash{Symbol=>Object}] feed record.
# @return [Hash{Symbol=>Object}] response-safe feed attributes.
def feed_attributes(feed_data)
typed_feed = feed_data.is_a?(BoundaryModels::FeedMetadata) ? feed_data : BoundaryModels::FeedMetadata.new(**feed_data)
timestamp = Time.now.iso8601
typed_feed = feed_metadata(feed_data)

typed_feed.to_h.merge(
created_at: timestamp,
updated_at: timestamp
).slice(:id, :name, :url, :strategy, :public_url, :created_at, :updated_at)
typed_feed_attributes(typed_feed, timestamp).slice(*FEED_ATTRIBUTE_KEYS)
end

# Parses params with optional JSON body override.
#
# @param request [Rack::Request]
# @return [Hash{String=>Object}] merged request params.
def request_params(request)
return request.params unless json_request?(request)

Expand All @@ -145,15 +106,11 @@ def request_params(request)
raise BadRequestError, 'Invalid JSON payload'
end

# @param request [Rack::Request]
# @return [Boolean] whether request body should be parsed as JSON.
def json_request?(request)
content_type = request.env['CONTENT_TYPE'].to_s
content_type.include?('application/json')
end

# @param request [Rack::Request]
# @return [Array<(Html2rss::Web::BoundaryModels::FeedCreateParams, Object)>]
def build_feed_from_request(request)
account = require_account(request)
ensure_auto_source_enabled!
Expand All @@ -165,8 +122,6 @@ def build_feed_from_request(request)
[params, feed_data]
end

# @param params [Html2rss::Web::BoundaryModels::FeedCreateParams]
# @return [void]
def emit_create_success(params)
Observability.emit(
event_name: 'feed.create',
Expand All @@ -176,8 +131,6 @@ def emit_create_success(params)
)
end

# @param error [StandardError]
# @return [void]
def emit_create_failure(error)
Observability.emit(
event_name: 'feed.create',
Expand All @@ -186,6 +139,16 @@ def emit_create_failure(error)
level: :warn
)
end

def feed_metadata(feed_data)
return feed_data if feed_data.is_a?(BoundaryModels::FeedMetadata)

BoundaryModels::FeedMetadata.new(**feed_data)
end

def typed_feed_attributes(typed_feed, timestamp)
typed_feed.to_h.merge(created_at: timestamp, updated_at: timestamp)
end
end
end
end
Expand Down
Loading
Loading