Skip to content

Commit bc6e4e4

Browse files
nanotaboadananotaboada
committed
feat(copilot): implement token efficiency strategy (#364)
- Optimize .github/copilot-instructions.md to 622 tokens (target: 600, limit: 650) - Update AGENTS.md with accurate token counts (~2,550 tokens, on-demand) - Add scripts/count-tokens.sh with auto-install tiktoken feature - Achieve 80% token savings when AGENTS.md not loaded
1 parent 3b546aa commit bc6e4e4

3 files changed

Lines changed: 165 additions & 28 deletions

File tree

‎.github/copilot-instructions.md‎

Lines changed: 33 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,40 +1,40 @@
11
# GitHub Copilot Instructions
22

3-
> **⚡ Token Efficiency Note**: This is a minimal pointer file (~500 tokens, auto-loaded by Copilot).
4-
> For complete operational details, reference: `#file:AGENTS.md` (~2,500 tokens, loaded on-demand)
5-
> For specialized knowledge, use: `#file:SKILLS/<skill-name>/SKILL.md` (loaded on-demand when needed)
3+
> **Token Budget**: Target 600, limit 650 (auto-loaded)
4+
> Details: `#file:AGENTS.md` (~2,550 tokens, on-demand)
5+
> Skills: `#file:SKILLS/<name>/SKILL.md` (on-demand)
66
7-
## 🎯 Quick Context
7+
## Quick Context
88

9-
**Project**: ASP.NET Core 8 REST API demonstrating layered architecture patterns
10-
**Stack**: .NET 8 (LTS) • EF Core 9 • SQLite • Docker • xUnit
11-
**Pattern**: Repository + Service Layer + AutoMapper + FluentValidation
12-
**Philosophy**: Learning-focused PoC emphasizing clarity and best practices
9+
ASP.NET Core 8 REST API with layered architecture
10+
**Stack**: .NET 8 LTS, EF Core 9, SQLite, Docker, xUnit
11+
**Pattern**: Repository + Service + AutoMapper + FluentValidation
12+
**Focus**: Learning PoC emphasizing clarity and best practices
1313

14-
## 📐 Core Conventions
14+
## Core Conventions
1515

1616
- **Naming**: PascalCase (public), camelCase (private)
1717
- **DI**: Primary constructors everywhere
1818
- **Async**: All I/O operations use async/await
1919
- **Logging**: Serilog with structured logging
2020
- **Testing**: xUnit + Moq + FluentAssertions
21-
- **Formatting**: CSharpier (opinionated)
21+
- **Formatting**: CSharpier
2222

23-
## 🏗️ Architecture at a Glance
23+
## Architecture
2424

2525
```text
2626
Controller → Service → Repository → Database
2727
↓ ↓
2828
Validation Caching
2929
```
3030

31-
- **Controllers**: Minimal logic, delegate to services
32-
- **Services**: Business logic + caching with `IMemoryCache`
33-
- **Repositories**: Generic `Repository<T>` + specific implementations
34-
- **Models**: `Player` entity + Request/Response DTOs
35-
- **Validators**: FluentValidation for input structure (business rules in services)
31+
Controllers: Minimal logic, delegate to services
32+
Services: Business logic + `IMemoryCache` caching
33+
Repositories: Generic `Repository<T>` + specific implementations
34+
Models: `Player` entity + DTOs
35+
Validators: FluentValidation (structure only, business rules in services)
3636

37-
## ✅ Copilot Should
37+
## Copilot Should
3838

3939
- Generate idiomatic ASP.NET Core code with minimal controller logic
4040
- Use EF Core async APIs with `AsNoTracking()` for reads
@@ -44,14 +44,14 @@ Validation Caching
4444
- Use primary constructors for DI
4545
- Implement structured logging with `ILogger<T>`
4646

47-
## 🚫 Copilot Should Avoid
47+
## Copilot Should Avoid
4848

4949
- Synchronous EF Core APIs
5050
- Controller business logic (belongs in services)
5151
- Static service/repository classes
5252
- `ConfigureAwait(false)` (unnecessary in ASP.NET Core)
5353

54-
## ⚡ Quick Commands
54+
## Quick Commands
5555

5656
```bash
5757
# Run with hot reload
@@ -66,12 +66,21 @@ docker compose up
6666
# Swagger: https://localhost:9000/swagger
6767
```
6868

69-
## 📚 Need More Detail?
69+
## Load On-Demand Files
7070

71-
**For operational procedures**: Load `#file:AGENTS.md`
72-
**For Docker expertise**: *(Planned)* `#file:SKILLS/docker-containerization/SKILL.md`
73-
**For testing patterns**: *(Planned)* `#file:SKILLS/testing-patterns/SKILL.md`
71+
**Load `#file:AGENTS.md` when:**
72+
- "How do I run tests with coverage?"
73+
- "CI/CD pipeline setup or troubleshooting"
74+
- "Database migration procedures"
75+
- "Publishing/deployment workflows"
76+
- "Detailed troubleshooting guides"
77+
78+
**Load `#file:SKILLS/<skill-name>/SKILL.md` (planned):**
79+
- Docker optimization: `docker-containerization/SKILL.md`
80+
- Testing patterns: `testing-patterns/SKILL.md`
81+
82+
**Human-readable overview**: See `README.md` (not auto-loaded)
7483

7584
---
7685

77-
💡 **Why this structure?** Copilot auto-loads this file on every chat (~500 tokens). Loading `AGENTS.md` or `SKILLS/` explicitly gives you deep context only when needed, saving 80% of your token budget!
86+
**Why this structure?** Base instructions (~600 tokens) load automatically. On-demand files (~2,550 tokens) load only when needed, saving 80% of tokens per chat.

‎AGENTS.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
# AGENTS.md
22

3-
> **⚡ Token Efficiency Note**: This file contains complete operational instructions (~2,500 tokens).
4-
> **Auto-loaded**: NO (load explicitly with `#file:AGENTS.md` when you need detailed procedures)
5-
> **When to load**: Complex workflows, troubleshooting, CI/CD setup, detailed architecture questions
6-
> **Related files**: See `#file:.github/copilot-instructions.md` for quick context (auto-loaded, ~500 tokens)
3+
> **Token Efficiency**: Complete operational instructions (~2,550 tokens).
4+
> **Auto-loaded**: NO (load explicitly with `#file:AGENTS.md` when needed)
5+
> **When to load**: Complex workflows, troubleshooting, CI/CD setup, detailed architecture
6+
> **Related files**: `#file:.github/copilot-instructions.md` (auto-loaded, ~650 tokens)
77
88
---
99

‎scripts/count-tokens.sh‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
#!/bin/bash
2+
# 📊 Token Counter for Copilot Instruction Files
3+
# Uses tiktoken (OpenAI's tokenizer) for accurate counting
4+
# Approximation: ~0.75 words per token (English text)
5+
6+
set -e
7+
8+
echo "📊 Token Analysis for Copilot Instructions"
9+
echo "=========================================="
10+
echo ""
11+
12+
# Check if tiktoken is available
13+
if command -v python3 &> /dev/null; then
14+
# Try to use tiktoken for accurate counting
15+
if python3 -c "import tiktoken" 2>/dev/null; then
16+
echo "✅ Using tiktoken (accurate Claude/GPT tokenization)"
17+
echo ""
18+
else
19+
# tiktoken not found - offer to install
20+
echo "⚠️ tiktoken not installed"
21+
echo ""
22+
echo "tiktoken provides accurate token counting for Claude/GPT models."
23+
read -p "📦 Install tiktoken now? (y/n): " -n 1 -r
24+
echo ""
25+
if [[ $REPLY =~ ^[Yy]$ ]]; then
26+
echo "📥 Installing tiktoken..."
27+
if pip3 install tiktoken --quiet; then
28+
echo "✅ tiktoken installed successfully!"
29+
echo ""
30+
# Re-run the script after installation
31+
exec "$0" "$@"
32+
else
33+
echo "❌ Installation failed. Using word-based approximation instead."
34+
echo ""
35+
USE_APPROX=1
36+
fi
37+
else
38+
echo "📝 Using word-based approximation instead"
39+
echo " (Install manually: pip3 install tiktoken)"
40+
echo ""
41+
USE_APPROX=1
42+
fi
43+
fi
44+
45+
# Only run tiktoken if it's available and we didn't set USE_APPROX
46+
if [ -z "$USE_APPROX" ] && python3 -c "import tiktoken" 2>/dev/null; then
47+
48+
# Create temporary Python script
49+
cat > /tmp/count_tokens.py << 'PYTHON'
50+
import tiktoken
51+
import sys
52+
53+
# cl100k_base is used by GPT-4, Claude uses similar tokenization
54+
encoding = tiktoken.get_encoding("cl100k_base")
55+
56+
file_path = sys.argv[1]
57+
with open(file_path, 'r', encoding='utf-8') as f:
58+
content = f.read()
59+
60+
tokens = encoding.encode(content)
61+
print(len(tokens))
62+
PYTHON
63+
64+
# Count tokens for each file
65+
echo "📄 .github/copilot-instructions.md"
66+
COPILOT_TOKENS=$(python3 /tmp/count_tokens.py .github/copilot-instructions.md)
67+
echo " Tokens: $COPILOT_TOKENS"
68+
echo ""
69+
70+
echo "📄 AGENTS.md"
71+
AGENTS_TOKENS=$(python3 /tmp/count_tokens.py AGENTS.md)
72+
echo " Tokens: $AGENTS_TOKENS"
73+
echo ""
74+
75+
# Calculate total
76+
TOTAL=$((COPILOT_TOKENS + AGENTS_TOKENS))
77+
echo "📊 Summary"
78+
echo " Base load (auto): $COPILOT_TOKENS tokens"
79+
echo " On-demand load: $AGENTS_TOKENS tokens"
80+
echo " Total (if both): $TOTAL tokens"
81+
echo ""
82+
83+
# Check against target
84+
TARGET=600
85+
LIMIT=650
86+
if [ $COPILOT_TOKENS -le $TARGET ]; then
87+
echo "✅ copilot-instructions.md within target ($TARGET tokens)"
88+
elif [ $COPILOT_TOKENS -le $LIMIT ]; then
89+
echo "⚠️ copilot-instructions.md over target but within limit ($LIMIT tokens)"
90+
else
91+
echo "❌ copilot-instructions.md exceeds limit! Optimization required."
92+
fi
93+
94+
SAVINGS=$((AGENTS_TOKENS * 100 / TOTAL))
95+
echo "💡 Savings: ${SAVINGS}% saved when AGENTS.md not needed"
96+
97+
# Cleanup
98+
rm /tmp/count_tokens.py
99+
fi
100+
else
101+
echo "❌ Python3 not found"
102+
echo " Python 3 is required for token counting"
103+
echo " Install from: https://www.python.org/downloads/"
104+
echo ""
105+
exit 1
106+
fi
107+
108+
# Fallback: word-based approximation
109+
if [ -n "$USE_APPROX" ]; then
110+
echo "📄 .github/copilot-instructions.md"
111+
WORDS=$(wc -w < .github/copilot-instructions.md | tr -d ' ')
112+
APPROX_TOKENS=$((WORDS * 4 / 3))
113+
echo " Words: $WORDS"
114+
echo " Approx tokens: $APPROX_TOKENS"
115+
echo ""
116+
117+
echo "📄 AGENTS.md"
118+
WORDS=$(wc -w < AGENTS.md | tr -d ' ')
119+
APPROX_TOKENS=$((WORDS * 4 / 3))
120+
echo " Words: $WORDS"
121+
echo " Approx tokens: $APPROX_TOKENS"
122+
echo ""
123+
124+
echo "💡 Note: Run script again to install tiktoken for accurate counts"
125+
fi
126+
127+
echo ""
128+
echo "=========================================="

0 commit comments

Comments
 (0)