feat(copilot): implement token efficiency strategy (#364)

[nanotaboada]

• Optimize .github/copilot-instructions.md to 622 tokens (target: 600, limit: 650)
• Update AGENTS.md with accurate token counts (~2,550 tokens, on-demand)
• Add scripts/count-tokens.sh with auto-install tiktoken feature
• Achieve 80% token savings when AGENTS.md not loaded

Summary by CodeRabbit

• Documentation

  • Updated AI assistant instructions with explicit token budget, clearer quick-context bullets, streamlined architecture overview, concrete quick commands, expanded guidance (error details, testing, DI/logging), and on‑demand loading guidance for large files.
  • Refined reference notes with updated token estimates and loading recommendations.

• Chores

  • Added a utility script to estimate/count tokens for instruction files with fallback behavior and install prompts.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Updated Copilot instruction content to split base vs on-demand guidance and clarified token budgets; adjusted AGENTS.md loading notes; added a new scripts/count-tokens.sh utility that uses Python/tiktoken when available with a word-based fallback and install flow.

Changes

Cohort / File(s)	Summary
Instruction files ​.github/copilot-instructions.md, AGENTS.md	Rewrote and condensed guidance for token-efficiency: explicit Token Budget, reorganized architecture/quick-commands, expanded "Copilot Should" guidance, and changed AGENTS.md to on-demand with updated token counts and load notes.
Token counting utility scripts/count-tokens.sh	Added new shell script that attempts to use Python + tiktoken to compute per-file and total token counts for instruction files, supports AUTO_INSTALL_TIKTOKEN or interactive install prompt, and falls back to a word-based token estimate when tiktoken is unavailable.

Sequence Diagram(s)

mermaid
sequenceDiagram
participant User/CI as User/CI
participant Script as scripts/count-tokens.sh
participant Python as Python (tiktoken)
participant Fallback as Word-based approx
participant Output as Console Output

User/CI->>Script: run script
Script->>Python: check python3 & detect tiktoken
alt tiktoken available
    Script->>Python: execute token-count script
    Python-->>Output: per-file counts + total
else tiktoken missing
    Script->>User/CI: prompt or check AUTO_INSTALL_TIKTOKEN
    alt auto-install or user consents
        Script->>Python: install tiktoken, retry
        Python-->>Output: per-file counts + total
    else
        Script->>Fallback: compute word-based estimate
        Fallback-->>Output: estimated counts + warnings
    end
end

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related issues

• [FEATURE] Implement Token Efficiency Strategy for Copilot Instructions #364: Implements the same token-efficiency split and adds a token-counting script — directly related.
• [FEATURE] Implement Token Efficiency Strategy for Copilot Instructions python-samples-fastapi-restful#477: Mirrors token-budget and on-demand AGENTS.md changes plus scripts/count-tokens.sh.
• [FEATURE] Implement Token Efficiency Strategy for Copilot Instructions java.samples.spring.boot#259: Applies the same instruction-file token-budget pattern and adds a matching token counter script.
• [FEATURE] Implement Token Efficiency Strategy for Copilot Instructions rust-samples-rocket-restful#30: Similar split of copilot instruction content and addition of a token-counting utility.
• [FEATURE] Implement Token Efficiency Strategy for Copilot Instructions go-samples-gin-restful#185: Adds identical token-counting script and token-budget/on-demand adjustments.

Possibly related PRs

• docs: optimize AI agent instructions for token efficiency #361: Restructures .github/copilot-instructions.md and AGENTS.md to reduce auto-loaded tokens — strong overlap with this PR.
• docs: enhance copilot-instructions with commands and workflows #309: Adds concrete command/workflow examples in .github/copilot-instructions.md, overlapping command-section edits.
• chore: add GitHub Copilot instructions file #278: Earlier PR that created the initial Copilot instructions file which this PR refines.

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title follows Conventional Commits format (feat:), is descriptive and specific about implementing token efficiency strategy, is under 80 characters (57), and directly aligns with the changeset's main objective of optimizing Copilot instructions for token efficiency.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feat/copilot-token-efficiency

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 3

🤖 Fix all issues with AI agents

In `@scripts/count-tokens.sh`:
- Around line 94-95: The savings calculation can divide by zero when TOTAL is 0;
update the script around the SAVINGS calculation to guard against this by
checking if TOTAL equals 0 before doing SAVINGS=$((AGENTS_TOKENS * 100 / TOTAL))
and handle that case (e.g., set SAVINGS=0 or print "0% saved"/a clear message)
so AGENTS_TOKENS and TOTAL are not used in a division-by-zero operation; ensure
the change is made where SAVINGS, AGENTS_TOKENS and TOTAL are defined/used so
downstream echo "💡 Savings: ${SAVINGS}% saved when AGENTS.md not needed" always
prints a safe value.
- Around line 64-73: The script assumes files exist; add existence checks before
calling the Python counter so failures are explicit: for each target (the calls
that set COPILOT_TOKENS and AGENTS_TOKENS using "python3 /tmp/count_tokens.py
<file>") test the file exists (e.g. using a shell test like [ -f "<file>" ]) and
if missing, print a clear warning and set the corresponding variable to a safe
default (e.g. "0" or "N/A") or skip counting, otherwise run the python script as
before; ensure messages mirror the current echo lines so behavior is consistent.
- Around line 23-42: The interactive read -p block will hang in CI; modify the
script to detect non-interactive environments (e.g. using [ -t 0 ] or checking
CI/CI_CD env vars) and skip the prompt automatically: if non-interactive, set
USE_APPROX=1 (or respect an env flag like AUTO_INSTALL_TIKTOKEN=1 to
auto-install) and bypass the read -p/install flow; otherwise keep the existing
prompt flow that runs pip3 install tiktoken and reexecs via exec "$0" "$@".
Ensure you change only the prompt surrounding code that references read -p and
USE_APPROX so CI runs don't block.

🧹 Nitpick comments (2)

scripts/count-tokens.sh (2)

49-62: Consider using mktemp for the temporary Python script.

Using a fixed path /tmp/count_tokens.py could cause conflicts if multiple instances run concurrently (e.g., parallel CI jobs on the same runner).

♻️ Proposed improvement

-        cat > /tmp/count_tokens.py << 'PYTHON'
+        TEMP_SCRIPT=$(mktemp /tmp/count_tokens.XXXXXX.py)
+        cat > "$TEMP_SCRIPT" << 'PYTHON'

Then update all references from /tmp/count_tokens.py to "$TEMP_SCRIPT".

---

27-27: Consider --user flag for pip install.

Global pip install may require elevated permissions. Using --user installs to the user's local directory without needing sudo.

♻️ Proposed fix

-            if pip3 install tiktoken --quiet; then
+            if pip3 install tiktoken --user --quiet; then

[codacy-production]

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation	Diff coverage
✅ +0.00%	✅ ∅

Coverage variation details

	Coverable lines	Covered lines	Coverage
Common ancestor commit (5fd85c6)	1721	536	31.14%
Head commit (5096724)	1721 (+0)	536 (+0)	31.14% (+0.00%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details

	Coverable lines	Covered lines	Diff coverage
Pull request (#365)	0	0	∅ (not applicable)

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

[sonarqubecloud]

Quality Gate passed

Issues
11 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud