Skip to content

Latest commit

 

History

History
410 lines (293 loc) · 11 KB

File metadata and controls

410 lines (293 loc) · 11 KB

Deno Development Guide

Table of Contents

Git workflow

Deno uses a GH based standard git workflow. The main branch is main. All development happens in feature branches, which are then merged into main via pull requests.

When the feature is finished and ready to for review, follow these steps:

  • Create a new git branch, if you haven't already, with a descriptive name (e.g., feature/new-cli-command or fix/bug-in-worker-threads).
  • Commit your changes with clear and descriptive commit messages.
  • Push your branch to the remote repository.
  • Open a pull request (PR) against the main branch on GitHub.
  • Before committing, make sure tools/format.js is run to format your code
  • Before committing, if only non-Rust was changed, make sure to run tools/lint.js --js and fix any lint errors before committing
  • If you changed Rust code, make sure to run tools/lint.js and fix any lint errors before committing
  • In the PR description, provide a clear summary of the changes you made, why they were necessary, and any relevant context or links to related issues.
  • When pushing updates to the PR, make sure to never force push. Create as many commits as you need, all of them get squashed when the PR is merged, so there is no need to rewrite history. This also allows reviewers to see the incremental changes you made in response to feedback.
  • Keep your changes minimal, don't do drive-by changes in a PR. If you need to make a change that is not directly related to the PR, create a separate PR for it. This keeps the review process focused and efficient.

High Level Overview

The user visible interface and high level integration is in the deno crate (located in ./cli).

This includes flag parsing, subcommands, package management tooling, etc. Flag parsing is in cli/args/flags.rs. Tools are in cli/tools/<tool>.

The deno_runtime crate (./runtime) assembles the JavaScript runtime, including all "extensions" (native functionality exposed to JavaScript). The extensions themselves are in the ext/ directory, and provide system access to JavaScript – for instance filesystem operations and networking.

Key Directories

  • cli/ - User-facing CLI implementation, subcommands, and tools
  • runtime/ - JavaScript runtime assembly and integration
  • ext/ - Extensions providing native functionality to JS (fs, net, etc.)
  • tests/specs/ - Integration tests (spec tests)
  • tests/unit/ - Unit tests
  • tests/testdata/ - Test fixtures and data files

Quick Start

Before building, install the required prerequisites (Rust, native compilers, cmake, protobuf, etc.) and clone with --recurse-submodules as described in .github/CONTRIBUTING.md.

Building Deno

To compile after making changes:

cargo build

For faster iteration during development (less optimization):

cargo build --bin deno

Execute your development build:

./target/debug/deno eval 'console.log("Hello from dev build")'

Running with your changes

# Run a local file
./target/debug/deno run path/to/file.ts

# Run with permissions
./target/debug/deno run --allow-net --allow-read script.ts

# Run the REPL
./target/debug/deno

Commands

Compilation and Checks

# Check for compilation errors (fast, no binary output)
cargo check

# Check specific package
cargo check -p deno_runtime

# Build release version (slow, optimized)
cargo build --release

Code Quality

# Lint the code
./tools/lint.js

# Format the code
./tools/format.js

# Both lint and format
./tools/format.js && ./tools/lint.js

Testing

Running Tests

# Run all tests (this takes a while)
cargo test

# Filter tests by name
cargo test <nameOfTest>

# Run tests in a specific package
cargo test -p deno_core

# Run just the CLI integration tests
cargo test --bin deno

# Run spec tests only
cargo test specs

# Run a specific spec test
cargo test spec::test_name

Test Organization

  • Spec tests (tests/specs/) - Main integration tests, CLI command execution and output validation
  • Unit tests - Inline with source code in each module
  • Integration tests (cli/tests/) - Additional integration tests
  • WPT (tests/wpt/) - Web Platform Tests for web standards compliance

"spec" tests

The main form of integration test in deno is the "spec" test. These tests can be found in tests/specs. The idea is that you have a __test__.jsonc file that lays out one or more tests, where a test is a CLI command to execute and the output is captured and asserted against.

The name of the test comes from the directory the __test__.jsonc appears in.

Creating a New Spec Test

  1. Create a directory in tests/specs/ with a descriptive name
  2. Add a __test__.jsonc file describing your test steps
  3. Add any input files needed for the test
  4. Add .out files for expected output (or inline in __test__.jsonc)

Example:

tests/specs/my_feature/
  __test__.jsonc
  main.ts
  expected.out

__test__.jsonc schema

The schema for __test__.jsonc can be found in tests/specs/schema.json.

Example test structure:

{
  "tests": {
    "basic_case": {
      "args": "run main.ts",
      "output": "expected.out"
    },
    "with_flag": {
      "steps": [
        {
          "args": "run --allow-net main.ts",
          "output": "[WILDCARD]success[WILDCARD]"
        }
      ]
    }
  }
}

Output assertions

The expected output can be inline in a __test__.jsonc file or in a file ending with .out. For a given test step, the output field tells you either the inline expectation or the name of the file containing the expectation. The expectation uses a small matching language to support wildcards and things like that. A literal character means you expect that exact character, so Foo bar would expect the output to be "Foo bar". Then there are things with special meanings:

  • [WILDCARD] : matches 0 or more of any character, like .* in regex. this can cross newlines
  • [WILDLINE] : matches 0 or more of any character, ending at the end of a line
  • [WILDCHAR] - match the next character
  • [WILDCHARS(5)] - match any of the next 5 characters
  • [UNORDERED_START] followed by many lines then [UNORDERED_END] will match the lines in any order (useful for non-deterministic output)
  • [# example] - line comments start with [# and end with ]

Example .out file:

Check file://[WILDCARD]/main.ts
[WILDCARD]
Successfully compiled [WILDLINE]

Development Workflows

Adding a New CLI Subcommand

  1. Define the command structure in cli/args/flags.rs
  2. Add the command handler in cli/tools/<command_name>.rs or cli/tools/<command_name>/mod.rs
  3. Wire it up in cli/main.rs
  4. Add spec tests in tests/specs/<command_name>/

Example files to reference:

  • Simple command: cli/tools/fmt.rs
  • Complex command: cli/tools/test/

Modifying or Adding an Extension

  1. Navigate to ext/<extension_name>/ (e.g., ext/fs/, ext/net/)
  2. Rust code provides the ops (operations) exposed to JavaScript
  3. JavaScript code in the extension provides the higher-level APIs
  4. Update runtime/worker.rs to register the extension if new
  5. Add tests in the extension's directory

Updating Dependencies

# Update Cargo dependencies
cargo update

# Update to latest compatible versions
cargo upgrade  # Requires cargo-edit: cargo install cargo-edit

# Check for outdated dependencies
cargo outdated  # Requires cargo-outdated

Debugging

Debugging Rust Code

Use lldb directly:

lldb ./target/debug/deno
(lldb) run eval 'console.log("test")'

Debugging JavaScript Runtime Issues

Use println debugging.

Verbose Logging

# Set Rust log level
DENO_LOG=debug ./target/debug/deno run script.ts

# Specific module logging
DENO_LOG=deno_core=debug ./target/debug/deno run script.ts

Debug Prints

In Rust code:

eprintln!("Debug: {:?}", some_variable);
dbg!(some_variable);

In JavaScript/TypeScript code:

console.log("Debug:", value);

Codebase Navigation

Key Files to Understand First

  1. cli/main.rs - Entry point, command routing
  2. cli/args/flags.rs - CLI flag parsing and structure
  3. runtime/worker.rs - Worker/runtime initialization
  4. runtime/permissions.rs - Permission system
  5. cli/module_loader.rs - Module loading and resolution

Common Patterns

  • Ops - Rust functions exposed to JavaScript (in ext/ directories)
  • Extensions - Collections of ops and JS code providing functionality
  • Workers - JavaScript execution contexts (main worker, web workers)
  • Resources - Managed objects passed between Rust and JS (files, sockets, etc.)

Finding Examples

  • Need to add a CLI flag? Look at similar commands in cli/args/flags.rs
  • Need to add an op? Look at ops in relevant ext/ directory (e.g., ext/fs/lib.rs)
  • Need to add a tool? Reference existing tools in cli/tools/

Troubleshooting

Build Failures

Error: linking with cc failed

  • Make sure you have the required system dependencies
  • On macOS: xcode-select --install
  • On Linux: Install build-essential or equivalent

Error: failed to download dependencies

  • Check internet connection
  • Try cargo clean then rebuild
  • Check if behind a proxy, configure cargo accordingly

For other build failures (missing cmake, stdarg.h, etc.), see the full prerequisites in .github/CONTRIBUTING.md.

Test Failures

Spec test failures

  • Check the test output carefully for differences
  • Update .out files if output format changed intentionally
  • Use [WILDCARD] for non-deterministic parts of output

Flaky tests

  • Add [UNORDERED_START]/[UNORDERED_END] for order-independent output
  • Check for race conditions in test code
  • May need to increase timeouts or add retries

Permission Issues

Tests failing with permission errors

  • Ensure test files have correct permissions
  • Check that test setup properly grants necessary permissions

Performance Issues

Slow compile times

  • Use cargo check instead of cargo build when possible
  • Use --bin deno to build only the main binary
  • Use sccache or mold linker for faster builds
  • Consider using cargo-watch for incremental builds

Runtime Debugging

Crashes or panics

  • Run with RUST_BACKTRACE=1 for full backtrace
  • Use RUST_BACKTRACE=full for even more detail
  • Check for unwrap() calls that might panic

Unexpected behavior

  • Add debug prints liberally
  • Check permission grants - many features require explicit permissions

Getting Help

  • Check existing issues on GitHub
  • Look at recent PRs for similar changes
  • Review the Discord community for discussions
  • When in doubt, ask! The maintainers are helpful