Is there any way to change Qt's ctrl-arrow word tokenizing?

Development
1 
["C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"]

If you ctrl-arrow through this array, it’s… really slow… not any faster than using arrow keys without modifiers.

The reason is because Qt’s word semantics are such that essentially every punctuation mark becomes a “word”… which may be reasonable for English text.

But we aren’t editing English text.

In the browser where I’m editing this message, if the cursor is to the left of "B", ctrl-left-arrow jumps to be between A and # – 4 characters. In SC-IDE, with the same scenario moves only two characters, and then the " and # are treated as separate tokens – so it takes three keystrokes to move the same amount.

I guess it would be necessary to draw up a spec of the kind of word tokens we want, and I admit that would be harder than it sounds at first.

But… am I the only person who is irritated by this? (Live coding depends heavily on efficient cursor movement, and our IDE has paid essentially zero attention to this problem.)

hjh

2

While on the subject… this has always struck me odd.

For each of these, position the cursor to the right of the string or symbol. Then use ctrl-shift-left-arrow to select the entire string, including delimiter(s). There is an inconsistency, in that the backslashed symbol can be selected with one keystroke – the tokenizer considers \ghi to be one “word” – while the other two consist of three tokens. IMO it would be more convenient if "abc" were a single entity.

[
	"abc"  // 3 shift-ctrl-left to select   
	,
	'def'  // 3x
	,
	\ghi   // 1x
]

Ctrl-backspace deletion is similarly affected – except that the backslash case is inconsistent with selection. For ctrl-backspace, it’s \ and ghi … ? Why?

[
	"abc"  // 3 ctrl-backspace to delete
	,
	'def'  // 3x
	,
	\ghi   // 2x - inconsistent with selection
]

Yeah, sure, it’s a small thing. But over the years, I’ve been tripped up momentarily by these, oh, it must be thousands of times now. It does wear on the nerves… software should adapt to us, instead of requiring us to adapt to it.

Oh, here’s one more:

0.1

A single ctrl-shift-left to select, but three ctrl-backspaces to delete :face_with_raised_eyebrow: which… go ahead, I’d love to hear the logical rationale for that :laughing:

hjh

3

It is annoying indeed.

At first sight the current behaviour is implemented in void ScCodeEditor::keyPressEvent(QKeyEvent* e) (file editors/sc-ide/widgets/code_editor/sc_editor.cpp, line 102) which calls ScCodeEditor::moveToNextToken and ScCodeEditor::moveToPreviousToken. Since these two functions seem to be called only on the context of handling keyPressEvents, maybe these are the ones where some modifications could alleviate the pain.

4

Great tip, thanks!

I was never able to figure out how to get visibility into the tokenizer’s object structure… In sclang, I would stick in printing statements, but Qt seems to have its own way to handle printing, and I didn’t have time to research it carefully :worried: so I’ll be honest that I’m not sure how far I’ll get.

hjh

5

To be honest I haven’t tried to modify this part of the code either, but it seems interesting enough that I’ll probably do some small experiment one of the coming days.

6

For fun and giggles, I replaced the

if (tokenIt.valid()) {
...
}

code fragment in void ScCodeEditor::moveToNextToken with

 if (tokenIt.isValid()) {
        ScIDE::Token::Type tokenType = tokenIt.type();
        if (skipTwiceIfPossible(tokenType)) {
            TokenIterator helper {tokenIt};
            if (helper.next().type() == tokenType)
                tokenIt = helper.next();
        }
         positionInBlock = tokenIt->positionInBlock + tokenIt->length;
}

and I replaced the same fragment in void ScCodeEditor::moveToPreviousToken with

if (tokenIt.isValid()) {
       ScIDE::Token::Type tokenType = tokenIt.type();
       if (skipTwiceIfPossible(tokenType)) {
           TokenIterator helper {tokenIt};
           if (helper.previous().type() == tokenType)
               tokenIt = helper.previous();
       }
       cursor.setPosition( tokenIt.position(), mode );
}

and I added a new method declaration in sc_editor.hpp (immediately under that of moveToPreviousToken)

...
void moveToPreviousToken( QTextCursor &, QTextCursor::MoveMode );
bool skipTwiceIfPossible(ScIDE::Token::Type &) const;
...

with implementation in sc_editor.cpp

bool ScCodeEditor::skipTwiceIfPossible(ScIDE::Token::Type & tokenType) const
{
    return tokenType == ScIDE::Token::SymbolMark 
        || tokenType == ScIDE::Token::StringMark;
}

And it seems to improve (if not fix) the “ctrl ->” and “ctrl <-” navigation and selection.

For ctrl-backspace, it seems there’s currently no special handling implemented, the keypress event is delegated to QPlainTextEdit’s default implementation, but I guess something could be added in the same regions.

I’ve pushed the experiment in my github repo in branch navigation_improvements, but I guess I made the wrong decision by basing it on develop, which doesn’t seem to have been updated in 3 years or so GitHub - shimpe/supercollider at navigation_improvements.

Anyway if this can be considered an improvement, I can spend some time making a proper pull request (but I will have to reread the contribution guidelines first :slight_smile: )

7

Ok, new attempt at improved ctrl + navigation that should be more relevant:

In my github repo GitHub - shimpe/supercollider: An audio server, programming language, and IDE for sound synthesis and algorithmic composition., in branch topic/ctrlarrow_navigation based on branch 3.12, I’ve added what I mentioned before as well as an implementation for ctrl+backspace (linting may still fail - I need to set up clang-format).