Skip to content

fix: do not escape underscore inside of words #72

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nikku wants to merge 4 commits into
base: main
Choose a base branch
from
Open

fix: do not escape underscore inside of words #72

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
caae14a
fix: do not escape underscore inside of words
nikku Jan 9, 2026
36591c0
test: verify underscore usage alongside punctuation
nikku Jan 9, 2026
27525c1
test: very against additional standard markdown cases
nikku Jan 9, 2026
30e46e7
test: expand existing underscore roundtrip test
nikku Jan 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion lib/unsafe.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,18 @@ export const unsafe = [
// Caret is not used in markdown for constructs.
// An underscore can start emphasis, strong, or a thematic break.
{atBreak: true, character: '_'},
{character: '_', inConstruct: 'phrasing', notInConstruct: fullPhrasingSpans},
{
character: '_',
before: '[\\s]',
inConstruct: 'phrasing',
notInConstruct: fullPhrasingSpans
},
{
character: '_',
after: '[\\s]',
inConstruct: 'phrasing',
notInConstruct: fullPhrasingSpans
},
// A grave accent can start code (fenced or text), or it can break out of
// a grave accent code fence.
{atBreak: true, character: '`'},
Expand Down
91 changes: 90 additions & 1 deletion test/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3895,6 +3895,39 @@ test('escape', async function (t) {
}
)

await t.test('should escape underscore at word start', async function () {
assert.equal(
to({
type: 'paragraph',
children: [{type: 'text', value: '_WORLD'}]
}),
'\\_WORLD\n'
)
})

await t.test('should escape underscore at word end', async function () {
assert.equal(
to({
type: 'paragraph',
children: [{type: 'text', value: 'HELLO_'}]
}),
'HELLO\\_\n'
)
})

await t.test(
'should not escape underscore inside of word',
async function () {
assert.equal(
to({
type: 'paragraph',
children: [{type: 'text', value: 'HELLO_WORLD'}]
}),
'HELLO_WORLD\n'
)
}
)

await t.test(
'should escape what would otherwise be a heading (atx)',
async function () {
Expand Down Expand Up @@ -4503,6 +4536,40 @@ test('roundtrip', async function (t) {
)
})

await t.test(
'should roundtrip underscore without escaping',
async function () {
const value = `Separate paragraphs:

:heavy_check_mark:

[HELLO_WORLD.md](HELLO_WORLD.md)

HELLO_WORLD https://some/web_site

HELLO_WORLD. HELLO_WORLD, H_W; HELLO_OTHER! HELLO_YES? HELLO_NO

HELLO_WORLD.HELLO_WORLD,H_W;HELLO_OTHER!HELLO_YES?HELLO_NO

HELLO_WORLD - THIS_GOOD

HELLO_WORLD-THIS_GOOD

One Paragraph:

:heavy_check_mark:
[HELLO_WORLD.md](HELLO_WORLD.md)
HELLO_WORLD https://some/web_site
HELLO_WORLD. HELLO_WORLD, H_W; HELLO_OTHER! HELLO_YES? HELLO_NO
HELLO_WORLD.HELLO_WORLD,H_W;HELLO_OTHER!HELLO_YES?HELLO_NO
HELLO_WORLD - THIS_GOOD
HELLO_WORLD-THIS_GOOD
`

assert.equal(to(from(value)), value)
}
)

await t.test(
'should roundtrip a sole blank line in fenced code',
async function () {
Expand Down Expand Up @@ -4781,6 +4848,20 @@ a __\\_ is this emphasis? __\\_

a _\\__ is this emphasis? _\\__

a _ is_this emphasis? _

a __ is_this emphasis? __

a ___ is_this emphasis? ___

a _\\_ is_this emphasis? _\\_

a \\__ is_this emphasis? \\__

a __\\_ is_this emphasis? __\\_

a _\\__ is_this emphasis? _\\__

One paragraph:

a _ is this emphasis? _
Expand All @@ -4789,7 +4870,15 @@ a ___ is this emphasis? ___
a _\\_ is this emphasis? _\\_
a \\__ is this emphasis? \\__
a __\\_ is this emphasis? __\\_
a _\\__ is this emphasis? _\\__`
a _\\__ is this emphasis? _\\__

a _ is_this emphasis? _
a __ is_this emphasis? __
a ___ is_this emphasis? ___
a _\\_ is_this emphasis? _\\_
a \\__ is_this emphasis? \\__
a __\\_ is_this emphasis? __\\_
a _\\__ is_this emphasis? _\\__`
Copy link
Copy Markdown
Member

@wooorm wooorm Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The needed tests are as follows:

  • a matrix of whitespace, punctuation, and rest (such as , ., x)
  • all 4 sides (before open, after open, before close, after close)
  • markers (_ and *)
  • marker count (1, 2)
  • nesting (1, 2, 3 times is probably enough)

I think that adds up to 3*4*2*2*3 == 144. You’d end up with say a.*.b.*.c. Where the .s are the surrounding characters, and * the markers.

You should probably generate those cases! Like what @ChristianMurphy mentioned above, it is important that they are structurally markdown->ast->markdown->ast equal, but the 2 markdown strings do not have to be equal

const tree = from(value)

assert.deepEqual(
Expand Down