Evaluate integration of Yara-forge

[mlodic]

https://cyb3rops.medium.com/introducing-yara-forge-a77cbb77dcab. We could re-evaluate how we manage the rules.

They update the list once in a week. This would remove the efforts of maintaining a list of decent yara rules repositories. We could just download these rules and remove the other repos by default

[alpeshjamgade]

hi @mlodic , can i work on this as my first contribution.

[mlodic]

yep you are welcome. What we need is a new analyzer that download periodically these rules and execute them.

[decodingafterlife]

Hey @mlodic I went through the medium article as well as read some Yara-forge documentation. What I get is that we want to create an analyser that frequently updates the YARA rules as they are updated in YARA forge and a function to execute these rules.

I would like to work on this. As this would be my first contribution, I'll appreciate it if you could guide me.

[mlodic]

hey @decodingafterlife, feel free to start. Please follow the guidelines in the documentation and you won't have any problem. Then, if some questions come up, please open a draft PR so we can have a conversation there

[decodingafterlife]

Great Ill start working then

[decodingafterlife]

Hey @mlodic do we want this to be a completely new analyzer wherein it automatically downloads the rules from yara-forge when the analyzer is used by a user (so it locally downloads them whenever the user uses the specific analyzer) or we want the rules to be uploaded and stored in this repo itself in a new directory, updated as new releases are released and maybe integrate it with already existing analyzer such as 'yara' (by default if user doesn't give specific yara rules)

[mlodic]

I think we should create a completely new analyzer so the users can choose between the new and the old one.

so it locally downloads them whenever the user uses the specific analyzer)

This should happen only if there is no local cache otherwise the analysis would be too slow. Ideally, the Yara forge rules should be downloaded periodically with a cronjob, like we do for the old analyzer.

we want the rules to be uploaded and stored in this repo itself in a new directory

Not in the repo itself, this could violate licenses and is generally problematic. We should download and update them with a cronjob and save them in the file storage of the principal container

[mlodic]

as mentioned here it would make sense to use Yara-X for this task

[AnshSinghal]

Hi @mlodic,
I’d like to take up the task of integrating Yara-forge (#2035). After reviewing the related documentation and prior comments, I understand that the objective is to create a new analyzer that will periodically download YARA rules from Yara-forge and execute them.

I have already started exploring and testing the issue to understand it further. Here's the approach I plan to follow:

1. New Analyzer: Create a completely new analyzer that integrates with Yara-forge.
2. Download & Cache: Ensure the analyzer fetches rules only if they aren’t already cached locally, to optimize performance.
3. Periodic Updates: Set up a cron job to periodically update the YARA rules so the analyzer always uses the latest versions.
4. User Choice: Allow users to choose this new Yara-forge analyzer or continue using the previous one, based on their needs.

Could you assign this issue to me? I’m excited to contribute!

[fgibertoni]

Hi @AnshSinghal,
thank you for your interest in solving this issue!
As @mlodic mentioned, this issue should be solved together with #2592. If you're encountering any compatibility issue (maybe it's still not fully compatible, as said here) during your work please open a draft PR so that you can get help to solve the issue. 😄

[mlodic]

Hey guys, don't bother too much about Yara-X at this time to avoid unnecesary additional complexity. @AnshSinghal any update on this? After 2 weeks without any activity, we have the rule of removing the assignment as you can see in the Contribute's docs.

[AnshSinghal]

Hi I created the analyzer but having issues to test because I work on windows. Need some time to test on other device

[mlodic]

I strongly suggest to use an Ubuntu VM if you have to use your Windows Workstation