ArmoRM + README.md update

.gitmodules

@@ -1,9 +1,9 @@
 [submodule "src/metametrics/metrics/GEMBA"]
-	path = metametrics/metrics/GEMBA
+	path = src/metametrics/metrics/GEMBA
 	url = https://github.com/MicrosoftTranslator/GEMBA.git
 [submodule "src/metametrics/metrics/BARTScore"]
-	path = metametrics/metrics/BARTScore
+	path = src/metametrics/metrics/BARTScore
 	url = https://github.com/neulab/BARTScore.git
 [submodule "src/metametrics/metrics/bleurt"]
-	path = metametrics/metrics/bleurt
+	path = src/metametrics/metrics/bleurt
 	url = https://github.com/google-research/bleurt.git

README.md

@@ -1,18 +1,60 @@
-# MetaMetrics
+# MetaMetrics V0.0.1
+[MetaMetrics](https://github.com/meta-metrics/metametrics) is a tuneable, easily extensible, and explainable metric for human evaluation alignment in generation tasks.
+The repository is the open-source implementation for MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences https://arxiv.org/pdf/2410.02381. We will release the code soon.
 
-The repository is the open-source implementation for MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences https://arxiv.org/pdf/2410.02381.
-We will release the code soon.
+## Supported Tasks
+The current version supports the following tasks:
+1. Question Answering
+2. Machine Translation
+3. Image Captioning
+4. Text Summarization
+5. Reward Modeling
 
-## Contents
+You can clone and adapt the codes to support other generation tasks.
 
-+ [Environment](#environment)
-+ [Setup Instruction](#setup-instruction)
+## Supported Metrics
+The current version supports the following metrics:
+1. BLEU
+2. BARTScore
+3. BERTScore
+4. BLEURT20
+5. chrF
+6. comet
+7. MetricX
+8. METEOR
+9. ROUGE
+10. ROUGEWE
+11. SummaQA
+12. YiSi
+13. GEMBA_MQM
+14. ClipScore
+15. ArmoRM
 
-## Environment
+## Installation Guide
+Requires `Python 3.10+`
+```
+PENDING
+```
 
-Python 3.10 or higher. Details of dependencies are in `setup.py`.
+## How To Use
+Example use-case with MetaMetrics library:
+```
+CODE PENDING
+```
 
-## Setup Instruction
+## How To Extend New Metrics
+Extending MetaMetrics to support other metrics is done by creating a Subclass of `metametrics.metrics.base_metric.BaseMetric` (Text Only Metric)
+or `metametrics.metrics.base_metric.VisionToTextBaseMetric` (Vision to Text Metric)
+and placing the file in `metametrics/src/metametrics/metrics/`.
 
-1. Run `pip install .` as it will automatically install required dependencies.
-2. Provide hugging-face token by setting as an environment variable in HF_TOKEN.
+
+The new metric must contain the following functions:
+1. `NewMetric.score(self, predictions: List[str], references: Union[None,List[List[str]]]=None, sources: Union[None, List[str]]=None) -> List[float]`.
+
+
+Checklist To Integrate Custom Metrics:
+1. [ ] `metametrics/src/metametrics/metrics/__init__.py` | Import your new metric
+2. [ ] `metametrics/src/metametrics/metrics/__init__.py` | Extend the `__all__` variable
+3. [ ] `metametrics/src/metametrics/metametrics.py` | Import your metric
+4. [ ] `metametrics/src/metametrics/metametrics.py` | Update variable `MetaMetrics.normalization_config`
+5. [ ] `metametrics/src/metametrics/metametrics.py` | update function `MetaMetrics.get_metric()`

src/metametrics/__init__.py

@@ -1,2 +1 @@
 from .version import __version__
-from .metametrics import MetaMetrics