Using delimited_by on whitespace doesn't seem to work

[Isaac-Leonard]

Theres a very good chance I'm just making a mistake here and I've not done extensive testing on it yet but it seems like the only cause is a bug in Chomsky.
The below code doesn't seem to be able to successfully parse

{ }

The parser code is:

exp
            .clone()
            .padded()
            .repeated()
            .or(whitespace().map(|_| Vec::new()))
            .delimited_by('{', '}')

The full code is on my GitHub however its not yet commented at all pretty much.
Basically I want to parse a list of zero or more expressions wrapped in braces, If zero expressions then I still want to allow whitespace between the braces.
Am I missing something in the code or is there a bug?
I'm happy to test more if this isn't enough information

[zesterer]

Hello!

The answer to this is subtle and non-obvious, and goes away entirely if you use .at_least(1) after .repeated() (although, this is probably not the best solution).

Why this happens

This happens because repeated is permitted to accept no occurrences of the pattern at all, so there is no possible input for which repeated can fail, because any failure counts as 'the end of the repeating section'.

When parsing your input, { }, the exp.padded().repeated() parser succeeds because no occurrences of exp.padded() were found. It then falls back to the latter part of delimited_by, immediately expecting a } following it (because or is lazy and only tries the second parser if the first failed). Instead, it finds the whitespace inside the brackets.

Solving this

In general, something like .or(whitespace) is not really the way to solve this. Because repeated accepts no input, this is already resolved! However, you will need an extra .padded() to account for any whitespace inside the brackets. The following should work:

exp
    .clone()
    .padded()
    .repeated()
    .padded()
    .delimited_by('{', '}')

I hope this helps!

[Isaac-Leonard]

Hey, that works perfectly thank you and makes complete sense, feel like I should've realised that from the start lol.
Thank you

[zesterer]

That's great to hear! Apologies that it was so unintuitive. I'm hopeful that a future debugging feature I have planned will aid in finding issues like this, but it's not ready yet!