Cloud spring cleaning 2024 04 30

README.md

@@ -27,8 +27,8 @@ Here are some of the most-visited sections:
 - [How to get started as a Code4rena warden](roles/wardens#joining-an-audit)
 - [Submission policy](roles/wardens/submission-policy.md) and [reporting guidelines](roles/wardens/submission-guidelines.md)
 - [Becoming Certified (KYC’d): benefits and process](roles/certified-contributors)
-    - [+Backstage warden role: overview, criteria and process](roles/certified-contributors/backstage-wardens.md)
-    - [Lookout role: overview, criteria and process](roles/certified-contributors/lookouts.md)
+    - [Certified Security Researchers: overview, criteria and process](roles/certified-contributors/sr-backstage-wardens.md)
+    - [Validator role: overview, criteria and process](roles/certified-contributors/validators.md)
     - [Scout role: overview and selection process](roles/certified-contributors/scouts.md)
 - Awarding [model](awarding/incentive-model-and-awards) and [process](awarding/incentive-model-and-awards/awarding-process.md)
 - [Judging criteria](awarding/judging-criteria) and [severity categorization](awarding/judging-criteria/severity-categorization.md)

awarding/incentive-model-and-awards/README.md

@@ -1,6 +1,6 @@
 # Incentive model and awards
 
-To incentivize **wardens**, C4 uses a unique scoring system with two primary goals: reward contestants for finding unique bugs and also to make the audit resistant to Sybil attack. A secondary goal of the scoring system is to encourage contestants to form teams and collaborate.
+To incentivize **wardens**, C4 uses a unique scoring system with two primary goals: reward participants for finding unique bugs and also to make the audit resistant to Sybil attack. A secondary goal of the scoring system is to encourage participants to form teams and collaborate.
 
 **Judges** are incentivized to review findings and decide their severity, validity, and quality by receiving a share of the prize pool themselves.
 
@@ -11,7 +11,7 @@ To incentivize **wardens**, C4 uses a unique scoring system with two primary goa
 
 ## High and Medium Risk bugs
 
-Contestants are given shares for bugs discovered based on severity, and those shares give the owner a pro rata piece of the pie:
+Wardens are given shares for bugs discovered based on severity, and those shares give the owner a pro rata piece of the pie:
 
 `Med Risk Slice: 3 * (0.9 ^ (split - 1)) / split`\
 `High Risk Slice: 10 * (0.9 ^ (split - 1)) / split`
@@ -40,6 +40,14 @@ The resulting awards are:
 | 'Warden B'  | 'H-02'      | '3'      |         8.91       |   3       |         2.70        |  1000                  |
 | 'Warden C'  | 'H-02'      | '3'      |         8.91       |   3       |         2.70        |  1000                  |
 
+### Bonuses for top competitors
+For audits starting on or after April 30, 2024, there are two bonuses for top-performing wardens:
+
+1. **Hunter bonus:** 10% of the HM pool will be awarded to the warden or team who identifies the greatest number of unique HMs.
+2. **Gatherer bonus:** 10% of the HM pool will be awarded to the warden or team who identifies the greatest number of valid HMs.
+
+Both bonuses weigh Highs more heavily than Mediums, similarly to Code4rena's standard awarding mechanism.
+
 ### Duplicates getting partial credit
 
 All issues which identify the same functional vulnerability will be considered duplicates regardless of effective rationalization of severity or exploit path.
@@ -103,66 +111,46 @@ We can see here that the logic behind the `partial-` labels only impacts the awa
 
 Only the award amounts for "partial" findings have been reduced, in line with expectations. The aim of this adjustment is to recalibrate the rewards allocated for these specific findings. Meanwhile, the awards for full-credit findings remain unchanged.
 
-## Bot races
-
-The first hour of each Code4rena audit is devoted to a bot race, to incentivize high quality automated findings as the first wave of the audit.
-
-- The winning bot report is selected and shared with all wardens within 24 hours of the audit start time.
-- The full set of issues identified by the best automated tools are considered out of scope for the audit and ineligible for awards.
-
-Doing this eliminates the enormous overlapping effort of all wardens needing to document common low-hanging issues And because the best bot report is shared with auditors at the start of the audit, these findings serve as a thorough starting place for understanding the codebase and where weaknesses may exist.
-
-**Ultimately, the bot race ensures human auditors are focused on things humans can do.**
-
-By designating a portion of the pool in this direction, Code4rena creates a separate lane for the significant investment of effort that many auditors already make in automated tooling -- and rather than awarding 100 people for identifying the same issue, we award the best automated tools.
-
-## Analyses
-
-Each warden is encouraged to submit an Analysis alongside their findings for each audit, to share high-level advice and insights from their review of the code.
-
-Where individual findings are the "trees" in an audit, the Analysis is a "forest"-level view.
+### Validator-improved submissions
 
-Advanced-level Analyses compete for a portion of each audit's award pool, and are graded and awarded similarly to QA and Gas Optimization reports.
+[Validators](https://docs.code4rena.com/roles/certified-contributors/validators.md) may enhance submissions (add PoC, increase quality of report, etc.) in exchange for a % of the finding’s payout. 
 
+For Validator-improved submissions: if the judge believes the validator added a measurable enhancement, they get a split of the value of the issue:
+- 25% cut → small enhancement
+- 50% cut → med enhancement
+- 75% cut → large enhancement
 
 ## QA and Gas Optimization Reports
 
-In order to incentivize wardens to focus efforts on high and medium severity findings while also ensuring quality coverage, the pool’s allocation is capped for low severity, non-critical, and gas optimization findings.
+In order to incentivize wardens to focus efforts on high and medium severity findings while also ensuring quality coverage, the pool’s allocation is capped for low severity, governance/centralization risk, and gas optimization findings.
 
-Low and non-critical findings are submitted as a **single** QA report. Similarly, gas optimizations are submitted as a single gas report. For more on reports, see [Judging criteria](/awarding/judging-criteria/README.md).
+Low severity and governance/centralization risk findings are submitted as a **single** QA report. Similarly, gas optimizations are submitted as a single gas report. For more on reports, see [Judging criteria](/awarding/judging-criteria/README.md).
 
 QA and gas optimization reports are awarded on a curve based on the judge’s score.
 
-- QA reports compete for a share of 2.5% of the prize pool (e.g. $1,250 for a $50,000 audit);
-- The gas optimization pool varies from audit to audit, but is typically 2.5% of the total prize pool (e.g. $1,250 for a $50,000 audit);
-- QA and Gas optimization reports are scored by judges using A/B/C grades (with C = unsatisfactory), and awarded on a curve.
+- QA reports compete for a share of 4% of the prize pool (e.g. $2,000 for a $50,000 audit);
+- The gas optimization pool varies from audit to audit;
+- QA and Gas optimization reports are awarded on a curve.
 
 There is a very high burden of quality and value provided for QA and gas optimization reports. Only submissions that demonstrate full effort worthy of consideration for inclusion in the report will be eligible for rewards.
 
-It is highly recommended to clearly spell out the impact of proposed gas optimizations.
-
-Historically, Code4rena valued non-critical findings at 0; the intent of the QA report is not to increase the value of non-criticals, but rather to allow them to be consolidated in reports alongside low severity issues.
-
 **Note:** Audits pre-dating February 3, 2022 awarded low risk and gas optimization shares as: `Low Risk Shares: 1 * (0.9 ^ (findingCount - 1)) / findingCount`
 
-In the unlikely event that zero high- or medium-risk vulnerabilities are found, the HM award pool will be divided based on the QA Report curve.
-
-## Grades for Analyses, QA and Gas reports
+### Ranks for QA and Gas reports
 
-Analyses, QA reports and Gas reports are graded A, B, or C.
+_These guidelines apply to all audits starting on or after April 30, 2024._ 
 
-C scores are unsatisfactory and ineligible for awards.
+After post-judging QA is complete, the Judge and Validators vote to select the top 3 QA reports and Gas reports. (In the case of a tie vote, there may be a 4th place report.)
 
-All A-grade reports receive a score of 2; All B-grade reports get a 1. Awarding for QA and Gas reports is on a curve that's described [here](https://docs.code4rena.com/awarding/incentive-model-and-awards/curve-logic).
+The 1st, 2nd, and 3rd place winners are awarded using a curve model that will be documented here ASAP. 
 
-### Bonus for best / selected for report
-Judges choose the best report in each category (Analysis, QA report, and Gas report), each of which earns the same 30% share bonus described under "High and Medium Risk bugs."
+Satisfactory reports not among the winning reports will not be awarded -- but will count towards wardens' accuracy scores.
 
-**Note:** if the `selected for report` submission has a B-grade label, it will still be treated as A-grade and given proportionally more than B-grade, plus the 30% bonus for being `selected for report`.
+In the unlikely event that zero high- or medium-risk vulnerabilities are found, the HM award pool will be divided among all satisfactory QA reports based on the QA Report curve, **unless otherwise stated in the audit repo.** 
 
 ## Satisfactory / unsatisfactory submissions
 
-Any submissions deemed unsatisfactory are ineligible for awards.
+Any submissions deemed unsatisfactory are ineligible for awards, and count against wardens' accuracy scores.
 
 The bar for satisfactory submissions is that they are roughly at a level that could be found in a draft report by a professional auditor: specifically on the merits of technical substance, with writing quality considered only where it interferes with comprehension of the technical message.
 
@@ -176,3 +164,39 @@ It is possible for a submission to be *technically* valid and still unsatisfacto
 - approach is disrespectful of sponsors’ and judges’ time in some way
 
 Any submissions that appear to be direct copies of other reports in the current audit will be collectively deemed unsatisfactory.
+
+## Other submission types
+
+As of April 30, 2024, the following submission types are paused:
+
+### Bot reports
+
+The first hour of each Code4rena audit is devoted to a bot race, to incentivize high quality automated findings as the first wave of the audit.
+
+- The winning bot report is selected and shared with all wardens within 24 hours of the audit start time.
+- The full set of issues identified by the best automated tools are considered out of scope for the audit and ineligible for awards.
+
+Doing this eliminates the enormous overlapping effort of all wardens needing to document common low-hanging issues And because the best bot report is shared with auditors at the start of the audit, these findings serve as a thorough starting place for understanding the codebase and where weaknesses may exist.
+
+**Ultimately, the bot race ensures human auditors are focused on things humans can do.**
+
+By designating a portion of the pool in this direction, Code4rena creates a separate lane for the significant investment of effort that many auditors already make in automated tooling -- and rather than awarding 100 people for identifying the same issue, we award the best automated tools.
+
+### Analyses
+
+Analyses share high-level advice and insights from wardens' review of the code.
+
+Where individual findings are the "trees" in an audit, the Analysis is a "forest"-level view.
+
+Analyses compete for a portion of each audit's award pool, and are graded and awarded similarly to QA and Gas Optimization reports.
+
+### Understanding historical grading for QA, Gas, and Analysis reports
+
+For audits that started before April 30, 2024: 
+
+- Analyses, QA reports and Gas reports in this time period were graded A, B, or C.
+- C scores are unsatisfactory and ineligible for awards.
+- All A-grade reports receive a score of 2; All B-grade reports get a 1. Awarding for QA and Gas reports is on a curve that's described [here](https://docs.code4rena.com/awarding/incentive-model-and-awards/curve-logic).
+- Judges choose the best report in each category (Analysis, QA report, and Gas report), each of which earns the same 30% share bonus described under "High and Medium Risk bugs."
+
+**Note:** if the `selected for report` submission has a B-grade label, it will still be treated as A-grade and given proportionally more than B-grade, plus the 30% bonus for being `selected for report`.

awarding/incentive-model-and-awards/awarding-process.md

@@ -5,18 +5,18 @@ description: >-
 
 # Awarding process
 
-At the conclusion of an audit, sponsors review wardens’ findings and express their opinions with regard to severity of issues. Judges evaluate input from both and make the ultimate decision in terms of severity and validity of issues. (See [How to judge an audit](../../roles/judges/how-to-judge-a-contest.md) for more detail.)
+At the conclusion of an audit, sponsors review wardens’ findings and express their opinions with regard to severity of issues. Judges evaluate input from both and make the ultimate decision in terms of severity and validity of issues. (See [How to judge an audit](https://docs.code4rena.com/roles/judges/how-to-judge-a-contest.md) for more detail.)
 
 In making their determination, judges add labels to Github issues, while the original submission data (including the warden's proposed severity rating) is preserved via a JSON data file. 
 
-The judge's decisions are reviewed by the sponsoring project team and by [+backstage wardens](https://docs.code4rena.com/roles/certified-contributors/backstage-wardens) via a 48-hour QA process, to ensure fairness and quality. 
+The judge's decisions are reviewed by the sponsoring project team and by [certified Security Researchers](https://docs.code4rena.com/roles/certified-contributors/backstage-wardens) via a 48-hour QA process, to ensure fairness and quality. 
 
 Judging data is used to generate the awards using Code4rena's award calculation script, which factors in:
 
 - Risk level
 - Validity
 - Number of duplicates
-- Grade (A, B, C; Satisfactory/Unsatisfactory)
+- Rank (1st, 2nd, 3rd; Satisfactory/Unsatisfactory)
 - In some cases, "partial duplicate" status
 
  It should be possible to reverse engineer awards using a combination of two CSV files:
@@ -40,7 +40,7 @@ If you still don’t see the award in your wallet, please [open a help desk tick
 
 We are occasionally asked how wardens should declare Code4rena earnings for tax (or other financial/legal) purposes. Due to the nascent nature of DAOs, we are unable to provide reliable information in this area. You must assess and determine your own best course of action.
 
-Audit contest rewards are distributed by the DAO, which does not have a legal personality.
+Audit rewards are distributed by the DAO, which does not have a legal personality.
 
 The DAO has designated Code4rena Foundation as its agent via [a governance action](https://github.com/code-423n4/org/discussions/13) [approved by DAO members](https://polygonscan.com/tx/0x8fbe178e34a7ae03a5e0d1f49f23e38f3a1c0d1186a67920d33196a89f79da98) for purposes of entering into contractual agreements. However, wardens are not in any contractual agreement with the Foundation [unless they are certified](https://code4rena.com/certified-contributor-summary/).