Ignore NULLs (if desired) while scanning keys during index navigation

[dyemanov]

See also #8291. This improvement completes the solution by extending the "ignore null" checks to the scan phase. The logic mostly matches btr.cpp (BTR_evaluate() and scan()).

With the same test case, results are:

before this PR:

SELECT count(*) from (SELECT ID FROM T WHERE ID < 3 ORDER BY ID);

Select Expression
    -> Aggregate
        -> Filter
            -> Table "T" Access By ID
                -> Index "IT" Range Scan (upper bound: 1/1)

                COUNT 
===================== 
                    1 

-- Fetches = 1775

after this PR:

-- Fetches = 7

[hvlad]

Accordingly to the compress():
// Further a NULL state is always returned as 1 byte 0xFF (descending index).
So, why check for >= 1, not == 1 ?
And, for consistency, please use 0xFF, not 255.

[dyemanov]

This is a copy-paste from btr.cpp :) where such a check is used twice. I don't mind changing it, but I'd suggest this to be done in btr.cpp too.

[dyemanov]

I'm also wondering whether we really should check for (node.prefix == 0 && node.length == 1 && node.data[0] == 0xFF) -- i.e. the first NULL encountered -- or better for (node.prefix + node.length == 1 && node.data[0] == 0xFF) -- i.e. any NULL encountered, as more reliable. Is it theoretically possible that we somehow jump to the bucket in the middle of the NULL duplicates chain and start scanning from there, thus skipping the first NULL node?

[hvlad]

I'm also wondering whether we really should check for
(node.prefix == 0 && node.length == 1 && node.data[0] == 0xFF) -- i.e. the first NULL encountered -- or better
for
(node.prefix + node.length == 1 && node.data[0] == 0xFF) -- i.e. any NULL encountered, as more reliable.

Second way is not correct: when node.length == 0 we should not access node.data[0] at all.
Instead, we could look at key contents, if necessary.

Is it theoretically possible that we somehow jump to the bucket in the middle of the NULL duplicates chain and start scanning from there, thus skipping the first NULL node?

The position is based on the last key processed, so, we should be safe.

[hvlad]

This is a copy-paste from btr.cpp :) where such a check is used twice. I don't mind changing it, but I'd suggest this to be done in btr.cpp too.

Ok, then, let it be this way

[pavel-zotov]

QA note: ticket issue is covered by test for 8291 (it was enough to reduce MAX_ALLOWED_IDX_READS threshold, see notes there).