Skip to content

Simplify autoShell Action Wiring#2196

Open
TalZaccai wants to merge 50 commits intomainfrom
talzacc/action_json_schema
Open

Simplify autoShell Action Wiring#2196
TalZaccai wants to merge 50 commits intomainfrom
talzacc/action_json_schema

Conversation

@TalZaccai
Copy link
Copy Markdown
Contributor

@TalZaccai TalZaccai commented Apr 14, 2026

Simplify autoShell action wiring

Summary

This PR redesigns how the TypeAgent desktop agent communicates with autoShell, replacing fragile hand-wired plumbing with a schema-driven, type-safe architecture. Adding a new action drops from ~6 file touches to 1–2.

Motivation

  1. Massive boilerplateconnector.ts had a 70-case switch where ~60 cases just forwarded JSON to autoShell. On the C# side, 31 of 46 settings actions followed three simple patterns but each was hand-coded.
  2. No error communication — autoShell wrote results to stdout with no structured response. The TS side had no way to know if an action succeeded, failed, or why.
  3. Silent schema drift — three independent declaration points (TS schema, connector switch, C# SupportedCommands) with no enforcement they agree. 7 C# actions had no TS schema at all.
  4. Unsafe parameter handling — handlers manually extracted parameters from raw JSON with string-keyed lookups and no type safety, broad catch (Exception) blocks, and a hardcoded bin/Debug exe path.

Key changes

1. Structured request-response protocol

  • IActionHandler.Handle now returns ActionResult with Ok()/Fail()/Quit() factories and optional Data payload — errors propagate back to the TS agent
  • connector.ts uses GUID-correlated sendAction instead of fire-and-forget writes
  • The 70-case switch is replaced by a 10-case preprocessor (only actions needing TS-side work)

2. Newtonsoft.Json → System.Text.Json

  • Newtonsoft.Json dependency removed
  • Handler interface simplified from Handle(string key, string value, JToken rawValue) to Handle(string key, JsonElement parameters)

3. Schema-driven validation

  • SchemaValidator parses .pas.json schemas and warns at startup about wiring mismatches between TS schemas and C# handlers
  • connector.ts loads known action names from schemas for runtime validation

4. Registration-driven settings handlers

  • SettingsHandlerBase provides three declarative patterns: AddRegistryToggleAction, AddRegistryMapAction, AddOpenSettingsAction
  • 31 settings actions reduced to one-line registrations; 9 handlers refactored (~785 lines removed)

5. Unified handler hierarchy

  • ActionHandlerBase with AddAction(name, handler) dictionary dispatch replaces static arrays + switch statements across all 17 handlers
  • SettingsHandlerBase extends ActionHandlerBase (single hierarchy)

6. Roslyn source generator for typed parameters

  • autoShell.Generators project reads .pas.json schemas at build time and generates C# record classes with [JsonPropertyName] attributes
  • AddAction<T>(name, handler) auto-deserializes JsonElement to typed records — 33 actions migrated from manual string lookups to compile-time type safety
  • Supports primitives, nullable types, string unions, and arrays

7. TypeScript schema additions

  • Added missing schemas: ApplyThemeAction, SystemThemeModeAction, optional refreshRate on SetScreenResolutionAction
  • Regenerated .pas.json files so the C# generator picks them up automatically

8. Error handling hardening

  • Deserialization: try/catch + null guard in AddAction<T> returns ActionResult.Fail on malformed input
  • Registry/process: targeted exception catches (UnauthorizedAccessException, SecurityException, IOException, Win32Exception) instead of broad catch (Exception)
  • Input validation: negative int-to-uint overflow fixed in SetScreenResolution
  • connector.ts: autoShell.exe resolved from Debug/Release with AUTOSHELL_PATH env var override

Impact

Metric Before After
connector.ts switch cases ~70 10
Hand-coded settings actions 46 15
Files to touch for new action ~6 1–2
Error feedback to TS agent None (stdout only) Structured ActionResult
Parameter type safety Manual string lookups Generated typed records (33 actions)
Test count 119 213

Testing

  • 213 tests passing (0 warnings, 0 failures)
  • Covers: handler actions, registered patterns (toggle/map/open-settings), schema validation, dispatcher routing, typed deserialization, edge cases
  • E2E tests filtered out (Category!=E2E) — require a running desktop

TalZaccai and others added 24 commits April 6, 2026 18:09
- Add ActionSchemaFile.cs with typed models for .pas.json schema format
- Add SchemaValidator.cs that reads .pas.json files and extracts action names
- Wire validation into CommandDispatcher.Create() to warn on mismatches
- Add knownActionNames set in connector.ts loaded from .pas.json at startup
- Add 14 unit tests for SchemaValidator (extraction, file loading, wiring)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Introduce SettingsHandlerBase abstract base class with three registered
action patterns (RegistryToggleAction, RegistryMapAction, OpenSettingsAction)
that eliminate hand-coded boilerplate for 31 of 46 settings actions.

- Created SettingsHandlerBase with AddRegistryToggleAction,
  AddRegistryMapAction, AddOpenSettingsAction registration methods
- Refactored 7 existing handlers to inherit base class
- Recreated PrivacySettingsHandler and SystemSettingsHandler as
  registration-only wrappers
- Removed 76-line centralized config block from CommandDispatcher
- Generalized RegistryToggleConfig to support object values (string
  toggles like StickyKeys/FilterKeys)
- 11 base-class pattern tests + 30 action-specific tests (208 total)
- Net reduction: ~785 lines across settings handlers

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Case-insensitive ValueMap lookup in HandleRegistryMapAction to match
  original OrdinalIgnoreCase behavior (Privacy, Taskbar handlers)
- Fix TaskbarAlignment default from 1 (center) back to 0 (left)
- Restore NotifySettingsChange broadcast after all FileExplorer and
  Taskbar actions by overriding Handle() (was only called for specialized)
- Make SettingsHandlerBase.Handle() virtual to support override pattern
- Add regression test for case-insensitive map lookup (209 tests)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add null guard in AddOpenSettingsAction — throws if no IProcessService
- Add duplicate detection across all registration methods (toggle, map,
  open-settings, specialized) — throws InvalidOperationException
- Add AddSpecializedAction registration method to eliminate two-sources-
  of-truth between static arrays and HandleSpecialized switch
- Make SupportedCommands virtual with default implementation combining
  specialized + registered actions (no longer abstract)
- Remove manual SupportedCommands overrides from all 9 handlers
- Remove static SpecializedActions arrays from 7 handlers
- 4 new guard/registration tests (213 total)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…foreach

AddRegistryMapAction now normalizes the ValueMap dictionary to use
StringComparer.OrdinalIgnoreCase, restoring O(1) TryGetValue lookup
while preserving case-insensitive matching behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Create ActionHandlerBase with AddAction(name, handler) registration and
  dictionary-based dispatch, eliminating duplicate SupportedActions arrays
  and switch statements across all 17 handlers
- SettingsHandlerBase now extends ActionHandlerBase; AddRegistryToggleAction,
  AddRegistryMapAction, and AddOpenSettingsAction delegate to AddAction with
  wrapper lambdas, removing AddSpecializedAction/HandleSpecialized pattern
- Migrate all 8 non-settings handlers to ActionHandlerBase
- Rename SupportedCommands to SupportedActions across interface and both
  base classes
- Rename all Command* classes to Action* for consistent terminology:
  ActionDispatcher, IActionHandler, ActionHandlerBase, ActionResult,
  and all 8 *ActionHandler classes plus their test files
- Make ActionHandlerBase.Handle() virtual so FileExplorer/Taskbar can
  wrap with NotifySettingsChange()
- Update README with new architecture and class names
- 213 tests passing, 0 warnings

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace JsonElement parameters with generated typed records (e.g.,
MuteParams, MaximizeParams, TileParams) using AddAction<T> for
automatic deserialization. Handlers that require JsonElement for
complex extraction (CreateDesktop array, SwitchDesktop string id,
SetScreenResolution refreshRate, SetTextSize non-numeric input,
Volume missing-vs-zero, ConnectWifi null ssid) are left unchanged.

Migrated files:
- AudioActionHandler (Mute, RestoreVolume)
- AppActionHandler (CloseProgram, LaunchProgram)
- WindowActionHandler (Maximize, Minimize, SwitchTo, Tile)
- ThemeActionHandler (SetThemeMode, SetWallpaper)
- NetworkActionHandler (BluetoothToggle, DisconnectWifi,
  EnableMeteredConnections, EnableWifi, ToggleAirplaneMode)
- VirtualDesktopActionHandler (NextDesktop, PreviousDesktop, PinWindow)
- SystemActionHandler (Debug, ToggleNotifications)
- Settings: Accessibility, Display, FileExplorer, Mouse, Power, Taskbar

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add Roslyn IIncrementalGenerator that reads .pas.json schema files at build
time and generates strongly-typed C# record classes for each action's
parameters. Add generic AddAction<T> overload to ActionHandlerBase that
auto-deserializes JsonElement to typed records.

New project: autoShell.Generators (netstandard2.0)
- SchemaParser: extracts action names + parameter fields from .pas.json
- RecordEmitter: generates C# records with [JsonPropertyName] attributes
- ActionParamsGenerator: IIncrementalGenerator wiring

Infrastructure:
- autoShell.csproj references generator as analyzer + includes .pas.json
  as AdditionalFiles from ts/packages/agents/desktop/dist/
- ActionHandlerBase.AddAction<T> deserializes via JsonSerializer with
  PropertyNameCaseInsensitive option

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add comprehensive section covering three action categories:
- Option A: Registry-based settings (AddRegistryToggleAction, one line)
- Option B: Typed action in existing handler (schema + AddAction<T>)
- Option C: New handler with new service (full walkthrough)

Also update Architecture section to document the source generator,
AddAction<T>, and autoShell.Generators project.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…files to Generated/

- Add LinkBase='Schemas' to AdditionalFiles so .pas.json files appear
  under a Schemas/ folder in Solution Explorer instead of the project root
- Enable EmitCompilerGeneratedFiles with output to Generated/ for
  easy inspection of source-generated parameter records
- Exclude Generated/ from compilation (already compiled as generated source)
- Add **/Generated/ to .gitignore (build artifact)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add autoShell.Generators to autoShell.sln (all 3 projects now in sln)
- Show Generated/ files as non-compiled items in Solution Explorer
  so they're browsable without being double-compiled

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add detailed 'Source generator (autoShell.Generators)' section covering:
- How the build pipeline works (AdditionalFiles → SchemaParser → RecordEmitter)
- Project structure and file purposes
- Generated output location and per-schema file mapping
- Type mapping table (.pas.json types → C# types)
- Roslyn constraints (netstandard2.0, analyzer reference)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…launches

- ActionHandlerBase.AddAction<T>: try/catch around JsonSerializer.Deserialize
- SettingsHandlerBase.ExecuteRegistryToggle: try/catch around Registry.SetValue
- SettingsHandlerBase.ExecuteRegistryMap: try/catch around Registry.SetValue
- SettingsHandlerBase.ExecuteOpenSettings: try/catch around Process.StartShellExecute

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rator

- Add array type support (string[]) to SchemaParser for CreateDesktop names
- Migrate Volume, ConnectWifi, SwitchDesktop, MoveWindowToDesktop, SetTextSize,
  CreateDesktop to use generated typed parameter records
- Update tests to match schema types (numeric desktopId, empty string defaults)
- Remaining JsonElement handlers: SetScreenResolution (refreshRate not in schema),
  ApplyTheme/SystemThemeMode (no schema), query-only actions (no params)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add ApplyThemeAction type to actionsSchema.ts (filePath parameter)
- Add SystemThemeModeAction type to personalizationActionsSchema.ts (mode parameter)
- Regenerate .pas.json files with new action definitions
- Migrate ThemeActionHandler.HandleApplyTheme to typed ApplyThemeParams
- Migrate PersonalizationSettingsHandler.HandleSystemThemeMode to typed SystemThemeModeParams

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add optional refreshRate parameter to SetScreenResolutionAction in TS schema
- Regenerate desktopSchema.pas.json with refreshRate field
- Migrate DisplayActionHandler.HandleSetScreenResolution to typed params

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- AddAction<T>: add null check after Deserialize to prevent NullReferenceException
- SetScreenResolution: validate width/height > 0 to prevent uint wraparound
- SetScreenResolution: validate refreshRate > 0 before casting to uint

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- connector.ts: resolve autoShell.exe from Debug/Release with AUTOSHELL_PATH
  env var override, instead of hardcoding bin/Debug
- SettingsHandlerBase: narrow catch blocks to specific expected exceptions
  (UnauthorizedAccessException, SecurityException, IOException, Win32Exception)
  instead of catching all exceptions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- ActionHandlerBaseTests: valid deserialization, null JSON, type mismatch
- DisplayActionHandlerTests: negative dimensions, negative refresh rate

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TalZaccai and others added 3 commits April 14, 2026 16:57
Validate that every .pas.json schema action has a C# handler and
every registered handler has a schema definition. Uses the refactored
SchemaValidator.FindMismatches to detect drift between TypeScript
schemas and C# dispatcher wiring at test time.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- ListAppNames/ListThemes: unwrap ActionResult to assert on .data array
- EmptyLine: fix concurrent stream read race with SemaphoreSlim guard
- Simplify ReadLineAsync to use CancellationToken directly instead of
  Task.WhenAny which left dangling reads

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Aligns the TypeScript interface name with the C# ActionResult class
that it mirrors across the stdin/stdout JSON protocol.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@TalZaccai TalZaccai requested a review from Copilot April 15, 2026 00:09
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 71 out of 72 changed files in this pull request and generated 8 comments.

Comments suppressed due to low confidence (1)

dotnet/autoShell/AutoShell.cs:23

  • The file header remarks and RunFromCommandLine XML comments still describe the old protocol (e.g. { "Volume":50 } and multi-command objects). The implementation now expects { "actionName": "Volume", "parameters": { ... } }, so this documentation is misleading for future maintainers/users. Update the remarks/examples to match the new request/response schema.
/// <summary>
/// Entry point for the autoShell Windows automation console application.
/// Reads JSON commands from stdin (interactive mode) or command-line arguments
/// and dispatches them to the appropriate handler via <see cref="ActionDispatcher"/>.
/// </summary>
/// <remarks>
/// Each JSON command is a single object where property names are command names
/// and values are parameters, e.g. <c>{"Volume":50}</c> or <c>{"Mute":true}</c>.
/// Multiple commands can be batched in one object: <c>{"Volume":50,"Mute":false}</c>.
/// The special command <c>"quit"</c> exits the application.
/// </remarks>

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dotnet/autoShell/Handlers/NetworkActionHandler.cs
Comment thread dotnet/autoShell/Handlers/Settings/TaskbarSettingsHandler.cs
Comment thread dotnet/autoShell/Services/WindowsVirtualDesktopService.cs
Comment thread dotnet/autoShell/Handlers/DisplayActionHandler.cs
Comment thread dotnet/autoShell.Tests/ActionDispatcherTests.cs Outdated
Comment thread dotnet/autoShell/AutoShell.cs
Comment thread dotnet/autoShell/Handlers/DisplayActionHandler.cs
Comment thread dotnet/autoShell/Handlers/NetworkActionHandler.cs
- TaskbarAlignment: default to center (1) matching original behavior
  (was incorrectly defaulting to left after refactor)
- BatterySaverActivationLevel: default to 20% when threshold is 0
  matching the original ?? 20 null-coalescing behavior

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix JsonDocument leaks in DisplayActionHandler and NetworkActionHandler
  by wrapping Parse in using and cloning before dispose
- Validate non-empty SSID in ConnectWifi before calling service
- Return failure from AutoHideTaskbar when registry blob is missing
- Add try/catch to RunFromCommandLine for invalid JSON input
- Guard against null from GetString() in VirtualDesktopService
- Fix XML doc in ActionDispatcherTests (said 'returns null')

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@TalZaccai TalZaccai marked this pull request as ready for review April 15, 2026 23:08
This ensures tsc compiles any TS schema changes before asc generates
the .pas.json files, so dotnet build always picks up the latest schemas
without requiring a separate TS build step.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The pre-build step now runs the full TS desktop build (tsc + asc:all)
instead of asc:all only, so CI needs to build all upstream TS
dependencies (agent-sdk, typechat-utils, etc.) not just the
action-schema-compiler.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use pnpm --filter with ... suffix to build all transitive TS
dependencies, ensuring dotnet build works even without a prior
monorepo-wide TS build.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The pre-build added too much overhead (~18s) and coupling. Instead:
- Developer builds TS desktop package first (generates .pas.json)
- CI explicitly runs asc:all after building the asc compiler
- AdditionalFiles glob moved to static ItemGroup

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@robgruen
Copy link
Copy Markdown
Collaborator

Ship it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants