Add rar file pattern as requested in #258

[targodan]

Pattern

The RAR file format as defined in the official documentation (https://www.rarlab.com/technote.htm). This pattern was requested in #258 .

There are still some issues I'd like to resolve, but I'm not versed enough in the pattern language to do so. I need some help here.

Issue 1: I have multiple cases where they have used a variable-length integer with certain bits indicating certain things. In some cases I got away with just using a fixed size bitfield, because the fields effectively were fixed width (at least with the linux rar implementation). In case of line 295 I did observe the field having variable width. I know of no way to cast the computed value of a vint into a bitfield. Maybe I'm missing a feature of the pattern language or it simply isn't possible.

Issue 2: As stated in line 360 I'd prefer to read headers until I reach a specific header, rather than end of file. This would allow for successful extraction of embedded rar files. Again, I don't know how to do this. Sadly, there is no absolute archive size anywhere that I could use instead.

Checklist

• A pattern for this format doesn't exist yet (or this PR improves the existing one)
• The new pattern has been added to the relevant table in the Readme
• The new pattern has a description pragma (#pragma description My pattern Description here)
• The pattern was associated with all relevant MIME types (using #pragma MIME mime-type in the source code)
  • Make sure to never use application/octet-stream here as that means "Unidentifiable binary data"
• A test file for this pattern has been added to /tests/patterns/test_data
  • Try to keep this file below ~ 1 MB

PS

FYI: I'm not sure why, but Github didn't load the PR template, I copy-pasted it manually.

[targodan]

Not sure why unit tests of other patterns suddenly fail, my rar.hexpat passed the unit tests. I did not modify any existing files except the README.md

[WerWolv]

Thanks a lot! The CI errors were my fault, sorry. I restarted the CI again now and it should hopefully pass

[WerWolv]

About your questions,

Issue 1:
It's possible to write bitfield fields whose length depend on other fields or other values in the pattern. For example:

bitfield Test {
    field1 : 3;
    field2 : field1;
};

Issue 2:
Using while(std::mem::eof()) is fine in my opinion. If you want to stop the evaluation of the array sooner though, you can use a break statement inside one of the structs that are placed by that array and it will terminate the array immediately. That way you should be able to stop once you hit that header

[targodan]

Thanks for the suggestions :)

Issue 1:

I don't think this approach will work. The vint implementation is a little more complex. The MSBit is basically a continue indicator. As long as that bit is set, we need to read another byte. Then for the actual integer value we have to mask out the continuation bits and do some shifting. Only then, does this computed value act as a bitfield.

You can see the decoding implementation in line 33.

So the problem is that I have a computed value (currently managed via both a transform and a format attribute) and the resulting value needs to be interpreted as a bit field. Is this possible?

Issue 2:

That's useful, I think. :) Implemented it.

[WerWolv]

There's not a very nice way to do it currently but here is one way. This uses the uLEB128 type since it works similar to your vint type.

The std::mem::Reinterpreter template is basically a templated union where you can assign a value to the from_value field and then read back the bits as a different type from the to_value field. Here I'm reinterpreting the decoded u64 value as a MyBitfield type and then print it. This works but it's not exactly pretty. We should probably add some nicer syntax for this

import type.leb128;

type::uLEB128 value @ 0x00;

bitfield MyBitfield {
    a : 1;
    b : 2;
    c : 3;
    padding : 58;
};

std::mem::Reinterpreter<u64, MyBitfield> reinterpreter;
reinterpreter.from_value = value;
std::print("{}", reinterpreter.to_value);

[paxcut]

The vint defined in the rar documentation is really uLEB128. For some reason they don't call it that but the definition is identical. Also the rar template in 010 uses uLEB128 instead of vint.

[Cynosphere]

This will complicate things, but the proposed pattern only works for newer RAR5 archives, which even though it was introduced in 2013, there is still a lot of RAR files floating around using the older format.

After a bit of digging to find any documentation on the older format I found this document that was listed on the format description from the Library of Congress.

[paxcut]

That document seems to be for versions 4 and app. The links to earlier versions are dead but with a bit of searching I found the version 2 and one of the version 3 specs. there are no records for version 1.

[WerWolv]

I don't know the spec super well but if it's too much work to implement full support right now, feel free to just add an std::assert to abort evaluation with a nice error message if the version isn't supported. There can always be improved upon that then :)