\b doesn't go all the way back on wrapped lines

[DannyDaemonic]

Windows Terminal version

1.16.10261.0

Windows build number

Version 10.0.19045.2965

Other Software

No response

Steps to reproduce

It's easily reproducible in C. This seems to happen with any writing to the console, WriteConsole, printf, putchar, etc. Just write more letters than you have columns, and try to backspace back. You'll get stuck at the front of a row.

    for (int i = 0; i < 100; i++) {
        putchar('-');
    }
    for (int i = 0; i < 100; i++) {
        putchar('\b');
    }

Expected Behavior

I just want \b to take me back to the previous line if I didn't end the previous line with a \n. The console knows the line is wrapped. I can increase the size of the window and the cursor will pop back up to the previous line.

Actual Behavior

The cursor just gets stuck in the first column and refuses to delete what it just wrote.

I don't even care if it stops working when you end your line with a new line, but not letting you unwrap a line you know is wrapped? This is really hard to code around. You can't just write to the terminal, you have to query your position every time you want to backspace or line wrapping will break your program.

[j4james]

This is expected behavior for a standard VT terminal. Quoting from the DEC STD 070 Reference Manual for the Back Space control (page 5-63):

The BS control is not affected in any way by the setting of Auto Wrap Mode. Under no circumstances does the Active Position advance to the previous line on reaching the beginning of the Active Line.

[DannyDaemonic]

It's just really terribly inconvenient to never know if you can use \b. A lot of modern terminals do allow you to unwrap with backspace. I know PuTTY does this and I have a friend who uses iterms2 on macOS that allows it. The Gnome terminal does. I don't know if it really breaks anything since so many modern terminals allow it.

It's too bad the DEC standard made that call, but standards aren't always set in stone and can evolve over time to better serve user needs. I mean, back then they had things like ENQ, ACK, BEL, etc that are all but ignored these days. I took a look real quick and there's lot of stuff that have been left behind from the standard. Here's a direct quote stating double width lines are a required feature:

Page 1-18

1.6.2 Required Functions

The following controls are required for all conforming Level 1 implementations. 
...
SINGLE WIDTH LINE
DOUBLE WIDTH LINE
DOUBLE HEIGHT LINE TOP
DOUBLE HEIGHT LINE BOTTOM

I've never even heard of that before. And it's not just on that one page, double width lines are mentioned 26 times. But I'd be willing to bet no one meets that requirement anymore.

I doubt they ever imagined terminals that were vastly different sizes or that could change size on the fly. I'm not saying the DEC STD 070 standard isn't still influential, but terminal emulator developers do make practical decisions about what aspects to implement based on the needs and constraints of modern computing.

[j4james]

I know PuTTY does this and I have a friend who uses iterms2 on macOS that allows it. The Gnome terminal does.

I've just done some testing, and yes PuTTY does this, and so does iTerm2, but the only other terminal I could find that matches that behavior is Rxvt (but not all versions). Gnome Terminal does NOT wrap back onto the previous line. Nor does XTerm. Nor did any of the other 30 or so terminals I tested. I can't imagine we'd choose to side with PuTTY, iTerm2, and Rxvt over everyone else, in addition to ignoring the standard.

double width lines are mentioned 26 times. But I'd be willing to bet no one meets that requirement anymore.

You'd lose that bet. Windows Terminal and Conhost both support it, as does XTerm, MLTerm, Konsole, and like 15 or so other terminals that I'm aware of. I know not everyone cares about standards, but I certainly do, and I would like to improve our standards compliance - not make it worse.

[DannyDaemonic]

I realize I'm not likely to change your mind, but I'm still interested in the conversation.

The terminals that the DEC STD 070 standards were written for had a fixed width in characters. So you could write your code assuming you had, say, 80 characters. You send a string of text you decided on ahead of time and then safely backspace over it and overwrite it. For example, you could print "Loading..." and then send 10 BS before replacing it with "Successfully loaded."

However, a modern terminal could be just 4 columns wide! Without backspace unwrapping, the previous example is no longer safe. There's no string short enough to safely use backspaces with. You have to check your position every single time you want to backspace and have special code that will take you to the end of the line above your current row instead. I think it's important to keep these interfaces usable and efficient.

I ended up writing my own version of printf/WriteConsole that iterates over each character and handles backspaces differently. So, I guess technically the problem is solved for me, but I don't think the standard would have even introduced a special BS character if they thought it was going to be unreliable.

I can't imagine we'd choose to side with PuTTY, iTerm2, and Rxvt over everyone else

Is it really only those 3? I often run in a screen session so I may have overestimated how common it is since screen supports backspace unwrapping as well. I can't recall screen breaking anything by allowing this, but my point was just that it's already being done. Since some popular terminal emulators allow it, there's already reasonable support for the feature.

You'd lose that bet. Windows Terminal and Conhost both support it

I guess I would lose that bet! I doubt any real terminal supports it but it looks like some virtual terminals do. Even PuTTY supports this:

echo -e "\033#3This is the top half of a double-height line.\n\033#4This is the bottom half of a double-height line."`

I saw screenshots of it in Windows Terminal but I couldn't get it to work myself. How can I test it there? I tried:

Write-Host ([char]27 + "#3This is the top half of a double-height line.`n" + [char]27 + "#4This is the bottom half of a double-height line.")

But perhaps that was a bad example. There's also extensive discussion about printers in DEC STD 070 that I assume no one follows anymore. The word "printer" comes up 852 times. As technology advances and changes, it's only natural that certain parts of these standards become outdated. (Again, a terminal with 4 columns.) I realize there must be some trade off for moving forward but it does seem others have figured out how to do this effectively without breaking backward compatibility with existing command-line tools, scripts, etc.

[j4james]

The terminals that the DEC STD 070 standards were written for had a fixed width in characters.

This isn't strictly true. DEC terminals could at least switch between 80 columns and 132 columns, and the windowing extension potentially allowed for arbitrary column widths to be requested (although I think most terminals of the time would still have restricted that to 80 and 132).

For example, you could print "Loading..." and then send 10 BS before replacing it with "Successfully loaded."
However, a modern terminal could be just 4 columns wide! Without backspace unwrapping, the previous example is no longer safe.

If you're writing that sort of UI, I would recommend disabling wrapping, so your "Loading..." message is simply truncated to "Load", rather than being split over 3 lines ("Load", "ing", "..."). Because once you start working with ridiculous screen sizes, your backspace unwrapping technique will still not prevent things from breaking. Like what happens when the screen height is only 2 lines and you wrap past the bottom? You're just going to end up with a scrollback buffer filled with "Load" messages.

I ended up writing my own version of printf/WriteConsole that iterates over each character and handles backspaces differently. So, I guess technically the problem is solved for me

Surely you would have to do this anyway, even if Windows Terminal changed its implementation, otherwise your app still wouldn't work on almost every other terminal out there.

Is it really only those 3?

I obviously can't test every terminal on the planet, but I've checked 34, and those were the only 3 I found with that behavior, so whatever the number, it seems likely they're going to be in the minority.

I doubt any real terminal supports it but it looks like some virtual terminals do.

What do you mean by "real terminal"? If you mean the hardware terminals, all the DEC hardware terminals supported double-width line renditions as far as I'm aware (at least from VT100 onwards).

I saw screenshots of it in Windows Terminal but I couldn't get it to work myself. How can I test it there?

It looks like the line rendition support for WT was only released in 1.7, so if you're still on 1.6 that explains why it's not working for you.

There's also extensive discussion about printers in DEC STD 070 that I assume no one follows anymore.

The printing functionality is considered an optional extension, so I wouldn't necessarily expect it to be widely supported, but I've still got 15 terminals that at least claim to have that extension.

it does seem others have figured out how to do this effectively without breaking backward compatibility with existing command-line tools

Well I know PuTTY and Rxvt have the bw flag set in their terminfo, which I assume will ensure that at least ncurses-based applications work correctly. And while I doubt it's a common problem, unless you're following their bug trackers closely you can't know for certain that they don't still get bug reports because of this choice. And sometimes people won't even bother complaining - they'll just switch to another terminal that does work for them (which wouldn't be hard to find in this case).

[zadjii-msft]

(I'm going to move this to a discussion - it doesn't seem like there's anything for us to do on our end here, but this has been a really interesting discussion so by all accounts, do go on)