Moving monorepo to single repo (#1347)

This PR changes the directory structure to move away from the monorepo
format, and instead be a single installable agent0 repo. This likely
includes many breaking changes.

- Updating installation instructions.
- Support for using [uv](https://github.com/astral-sh/uv) for
installation.
- Renaming `lib` to `src`
- Renaming `lib/agent0/agent0` to `lib/agent0/core`
- Moving all packages to be under agent0 (e.g., `lib/ethpy/ethpy` to
`src/agent0/ethpy`)
- Moving examples to outer directory (e.g., `lib/agent0/examples` to
`examples/`)
- Moving bin directories to scripts (e.g., `lib/agent0/bin` to
`scripts/`)
- Fixing imports
- Adding `src/agent0/__init__.py` that shortcuts imports (e.g., `from
agent0 import ILocalHyperdrive`)
- General cleanup of outdated examples and readmes.

.github/workflows/lint.yml

@@ -22,17 +22,20 @@ jobs:
           cache: "pip"
           token: ${{github.token}}
 
-      - name: upgrade pip
-        run: python -m pip install --upgrade pip
+      - name: install uv
+        run: curl -LsSf https://astral.sh/uv/install.sh | sh
 
       - name: install requirements
         run: |
-          python -m pip install --upgrade -r requirements.txt
-          python -m pip install --upgrade -r requirements-rl.txt
-          python -m pip install --upgrade -r requirements-dev.txt
+          uv venv .venv -p 3.10
+          source .venv/bin/activate
+          uv pip install --upgrade -e .[all]
+          uv pip install --upgrade -r requirements-hyperdrivepy.txt 
 
       - name: run black
-        run: python -m black --config pyproject.toml --check --diff .
+        run: |
+          source .venv/bin/activate
+          python -m black --config pyproject.toml --check --diff .
 
       - name: get all Python files
         id: list_files
@@ -41,6 +44,7 @@ jobs:
 
       - name: run Pylint on files
         run: |
+          source .venv/bin/activate
           files="${{ steps.list_files.outputs.files }}"
           if [ -n "$files" ]; then
             pylint --rcfile=.pylintrc $files

.github/workflows/static.yml

@@ -22,14 +22,17 @@ jobs:
           cache: "pip"
           token: ${{github.token}}
 
-      - name: upgrade pip
-        run: python -m pip install --upgrade pip
+      - name: install uv
+        run: curl -LsSf https://astral.sh/uv/install.sh | sh
 
       - name: install requirements
         run: |
-          python -m pip install --upgrade -r requirements.txt
-          python -m pip install --upgrade -r requirements-rl.txt
-          python -m pip install --upgrade -r requirements-dev.txt
+          uv venv .venv -p 3.10
+          source .venv/bin/activate
+          uv pip install --upgrade -e .[all]
+          uv pip install --upgrade -r requirements-hyperdrivepy.txt 
 
       - name: analysing code with pyright
-        run: python -m pyright $(git ls-files '*.py' '*.pyi')
+        run: |
+          source .venv/bin/activate
+          python -m pyright $(git ls-files '*.py' '*.pyi')

.github/workflows/test.yml

@@ -41,21 +41,24 @@ jobs:
           cache: "pip"
           token: ${{github.token}}
 
-      - name: upgrade pip
-        run: python -m pip install --upgrade pip
+      - name: install uv
+        run: curl -LsSf https://astral.sh/uv/install.sh | sh
 
       - name: install requirements
         run: |
-          python -m pip install --upgrade -r requirements.txt
-          python -m pip install --upgrade -r requirements-rl.txt
-          python -m pip install --upgrade -r requirements-dev.txt
+          uv venv .venv -p 3.10
+          source .venv/bin/activate
+          uv pip install --upgrade -e .[all]
+          uv pip install --upgrade -r requirements-hyperdrivepy.txt 
 
       - name: run pytest with coverage
         run: |
+          source .venv/bin/activate
           IN_CI=true coverage run -m pytest
 
       - name: generate coverage report
         run: |
+          source .venv/bin/activate
           coverage xml -i
           coverage html -i
 

.pylintrc

@@ -48,7 +48,7 @@ ignore=CVS
 # ignore-list. The regex matches against paths and can be in Posix or Windows
 # format. Because '\' represents the directory delimiter on Windows systems, it
 # can't be used as an escape character.
-ignore-paths=docs,.venv,lib/hypertypes/hypertypes/types/.*.py
+ignore-paths=docs,.venv,src/agent0/hypertypes/types/.*.py
 
 # Files or directories matching the regular expression patterns are skipped.
 # The regex matches against base names, not paths. The default value ignores

CONTRIBUTING.md

@@ -33,7 +33,7 @@ Please provide a brief description of the PR in the summary, as opposed to a lis
 
 ## Contributing a new bot policy
 
-We welcome contributions of new bot policies, alongside those in `lib/agent0/agent0/hyperdrive/policies`.
+We welcome contributions of new bot policies, alongside those in `src/agent0/core/hyperdrive/policies`.
 
 Submit a pull request that meets the following criteria:
 

Dockerfile

@@ -15,8 +15,8 @@ COPY . ./
 RUN apt-get update && \
   apt-get install -y --no-install-recommends gcc python3-dev libssl-dev git && \
   python -m pip install --no-cache-dir --upgrade pip && \
-  python -m pip install --no-cache-dir -r requirements.txt && \
-  python -m pip install --no-cache-dir -r requirements-dev.txt && \
+  python -m pip install --no-cache-dir . && \
+  python -m pip install --no-cache-dir -r requirements-hyperdrivepy.txt && \
   apt-get remove -y gcc python3-dev libssl-dev && \
   apt-get autoremove -y && \
   pip uninstall pipenv -y

INSTALL.md

@@ -1,124 +1,36 @@
-# Install -- overview
+# Install
 
-All `agent0` packages should be accessed from Python 3.10.
-The following outlines our recommended install workflow, although alternatives are listed below.
+## 1. Prerequisites
+While not strictly required when connecting to a remote chain, we recommend that you install these tools to use the full suite of Agent0 simulation tools.
 
-## 1. Clone the `agent0` monorepo
+- [Foundry](https://book.getfoundry.sh/getting-started/installation) for running a local Anvil testnnet node.
+- [Docker](https://docs.docker.com/engine/install/) for hosting the underlying data tools.
 
-Clone the repo into a <repo_location> of your choice.
+## 2. Clone
+Clone the repo to <repo_location>, then enter that directory.
 
 ```bash
 git clone https://github.com/delvtech/agent0.git <repo_location>
-```
-
-## 2. Install Pyenv
-
-Follow [Pyenv install instructions](https://github.com/pyenv/pyenv#installation).
-
-## 3. Set up a virtual environment
-
-You can use any environment; we use [venv](https://docs.python.org/3/library/venv.html):
-
-```bash
 cd <repo_location>
-pyenv install 3.10
-pyenv local 3.10
-python -m venv .venv
-source .venv/bin/activate
-```
-
-## 4. Install `agent0` packages and requirements
-
-All of the agent0 packages can be installed from `requirements.txt`:
-
-```bash
-python -m pip install --upgrade pip
-python -m pip install --upgrade -r requirements.txt
-```
-
-If you want to use the CI tools, such as run tests, check linting or types, build containers or documentation, then you must also [install Foundry](https://book.getfoundry.sh/getting-started/installation) and the dev packages:
-
-```bash
-python -m pip install --upgrade -r requirements-dev.txt
-```
-
-Finally, you can test that everything is working by calling: `python -m pytest .`
-
-# Alternate install paths
-
-The default installation directions above should automatically install all local sub-packages, and should be sufficient for development.
-We also support these installation options:
-
-## Installing each subpackage independently
-
-After you have cloned the repository you can install each package independently.
-For example:
-
-```bash
-python -m pip install --upgrade lib/agent0[with-dependencies]
-```
-
-Internally, the above installation calls
-
-```bash
-pip install agent0[base] # Install with base packages only (this is what's called in requirements.txt)
-pip install agent0[lateral] # Installs dependent sub-packages from git (e.g., ethpy)
-```
-
-# Working with smart contracts
-
-We run tests and offer utilities that depend on executing bytecode compiled from Hyperdrive solidity contracts.
-This is not required to use the agent0 libraries.
-
-NOTE: The Hyperdrive solidity implementation is currently under security review, and thus is not available publicly.
-The following instructions will not work for anyone who is not a member of Delv.
-
-## 1. Set up smart contracts
-
-Clone the hyperdrive repo:
-
-```bash
-git clone [email protected]:delvtech/hyperdrive.git ../hyperdrive
-```
-
-## 2. Install and Build Hyperdrive
-
-Complete the steps in Hyperdrive's [Pre-requisites](https://github.com/delvtech/hyperdrive#pre-requisites) and [Build](https://github.com/delvtech/hyperdrive#build) sections.
-
-## 3. Copy ABI & bytecode files
-
-Copy the contract `sol` folders from the generated `out` directory in the `hyperdrive` repository root.
-These folder should contain the ABI JSON and bytecode files for each contract.
-Paste the folders into `agnet0/packages/hyperdrive/src/abis/`.
-
-```bash
-cp -R ../hyperdrive/out/*.sol packages/hyperdrive/src/abis/
 ```
 
-Our codebase uses the following contracts:
+## 3. Setup environment
+We use [uv](https://github.com/astral-sh/uv) for package management and virtual environments.
 
 ```bash
-ERC20Mintable.sol/
-ERC4626DataProvider.sol/
-ERC4626HyperdriveDeployer.sol/
-ERC4626HyperdriveFactory.sol/
-ForwarderFactory.sol/
-IERC4626Hyperdrive.sol/
-IHyperdrive.sol/
-MockERC4626.sol/
+curl -LsSf https://astral.sh/uv/install.sh | sh
+uv venv .venv -p 3.10
+source .venv/bin/activate
 ```
 
-You then can update the generated `hypertypes` python package by running `pypechain` on this folder:
-
+## 4. Install `agent0`
 ```bash
-pip install --upgrade pip && pip install --upgrade pypechain
-pypechain packages/hyperdrive/src/abis/ --output-dir lib/hypertypes/hypertypes/types/
+uv pip install -e .
+# uv pip install agent0@. # For non-editable install
+# uv pip install -e .[dev] # For CI tools
+# uv pip install -e .[all] # Install everything
+# TODO: remove once hyperdrivepy is public
+uv pip install -r requirements-hyperdrivepy.txt 
 ```
 
-# Additional useful applications
-
-You can test against a local testnet node using [Anvil](https://book.getfoundry.sh/reference/anvil/).
-
-We use [Docker](docs.docker.com/get-docker) for building images.
-Some tests also use Docker to launch a local postgres server and Anvil chain.
-If there are Docker issues when running tests, ensure "Allow the default Docker socket to be used" is set under Advanced settings (typically when running Docker Desktop on Macs).
+You can test that everything is working by calling `python -m pytest .`

README.md

@@ -9,25 +9,35 @@
   <img alt="Agent 0" src="icons/agent0-light.svg">
 </picture>
 
-# [DELV](https://delv.tech) monorepo for market simulation and analysis
+# [DELV](https://delv.tech) repo for market simulation and analysis
 
 This project is a work-in-progress. All code is provided as is and without guarantee.
 The language used in this code and documentation is not intended to, and does not, have any particular financial, legal, or regulatory significance.
 
 This docs page can be found via [https://agent0.readthedocs.io/en/latest/](https://agent0.readthedocs.io/en/latest/).
 
-## Packages
-
-This monorepo houses internal packages that are still under development. They are:
-
-- agent0 ([README](lib/agent0/README.md))
-- chainsync ([README](lib/chainsync/README.md))
-- ethpy ([README](lib/ethpy/README.md))
-
-We also utilize internal packages that are "in production," which is to say they live in their own repo:
+```python
+import datetime
+from fixedpointmath import FixedPoint
+from agent0 import ILocalHyperdrive, ILocalChain
+
+# Initialize
+chain = ILocalChain()
+interactive_hyperdrive = ILocalHyperdrive(chain)
+hyperdrive_agent0 = interactive_hyperdrive.init_agent(base=FixedPoint(100_000))
+
+# Run trades
+chain.advance_time(datetime.timedelta(weeks=1))
+open_long_event = hyperdrive_agent0.open_long(base=FixedPoint(100))
+chain.advance_time(datetime.timedelta(weeks=5))
+close_event = hyperdrive_agent0
+
+# Analyze
+pool_state = interactive_hyperdrive.get_pool_state(coerce_float=True)
+pool_state.plot(x="block_number", y="longs_outstanding", kind="line")
+```
 
-- pypechain ([README](https://github.com/delvtech/pypechain/tree/main#readme), [pypi](https://pypi.org/project/pypechain/))
-- fixedpointmath ([README](https://github.com/delvtech/fixedpointmath#readme), [pypi](https://pypi.org/project/fixedpointmath/))
+See our [tutorial notebook](examples/tutorial.ipynb) for more information.
 
 ## Install
 
@@ -55,38 +65,10 @@ coverage html
 
 then just open `htmlcov/index.html` to view the report!
 
-## Examples
-
-Python files in the `examples/` folder should be executable from the repository root.
-Run them with the -h flag to see argument options.
-The Jupyter notebooks contained in `examples/notebooks/` should be run locally using [Jupyter](https://jupyter.org/install), [VS Code](https://code.visualstudio.com/docs/datascience/jupyter-notebooks), or something equivalent.
-
 ## Contributions
 
 Please refer to [CONTRIBUTING.md](CONTRIBUTING.md).
 
-## Modifying configuration for agent deployment
-
-Follow [`lib/agent0/README.md`](lib/agent0/README.md) for agent deployment.
-
-## Data pipeline
-
-The data pipeline queries the running chain and exports data to a postgres database. The `infra` repository spins up a local postgres instance via Docker, and the data pipeline will point to this by default. Optionally, you can also configure the backend database by specifying the following environmental variables (for example, in a `.env` file in the base of the repo):
-
-```bash
-POSTGRES_USER="admin"
-POSTGRES_PASSWORD="password"
-POSTGRES_DB="postgres_db"
-POSTGRES_HOST="localhost"
-POSTGRES_PORT=5432
-```
-
-The data script can be then ran using the following command:
-
-```bash
-python lib/chainsync/chainsync/exec/acquire_data.py
-```
-
 ## Number format
 
 We frequently use 18-decimal [fixed-point precision numbers](https://github.com/delvtech/fixedpointmath#readme) for arithmetic.

conftest.py

@@ -6,18 +6,18 @@
 import os
 
 import pytest
-from chainsync.test_fixtures import database_engine, db_api, db_session, dummy_session, psql_docker
-from ethpy.test_fixtures import (
+
+from agent0.chainsync.test_fixtures import database_engine, db_api, db_session, dummy_session, psql_docker
+from agent0.core.test_fixtures import chain
+from agent0.core.test_utils import cycle_trade_policy
+from agent0.ethpy.test_fixtures import (
     hyperdrive_read_interface,
     hyperdrive_read_write_interface,
     init_local_hyperdrive_pool,
     local_chain,
     local_hyperdrive_pool,
 )
 
-from agent0.test_fixtures import chain
-from agent0.test_utils import cycle_trade_policy
-
 # Hack to allow for vscode debugger to throw exception immediately
 # instead of allowing pytest to catch the exception and report
 # Based on https://stackoverflow.com/questions/62419998/how-can-i-get-pytest-to-not-catch-exceptions/62563106#62563106

docs/README.md

@@ -1,2 +1 @@
-agent0 documentation and notebook examples can be built locally using the provided Makefile using `make html`.
-This also includes notebook examples built using CI from `https://github.com/delvtech/elf-simulations-examples`.
+agent0 documentation and notebook examples can be built locally using the provided Makefile using `make html`.