Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dependency caching #10

Closed
wants to merge 15 commits into from
28 changes: 23 additions & 5 deletions .github/workflows/train_workflow.yml
Original file line number Diff line number Diff line change
@@ -5,17 +5,35 @@ on:
script_content:
description: 'Content of train.py'
required: true
type: string # Explicitly specify the type
type: string

jobs:
train:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'

- name: Cache PyTorch
uses: actions/cache@v3
with:
path: |
~/.cache/pip
${{ runner.tool_cache }}/Python
key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-

- name: Install dependencies
run: |
pip install numpy
# pip install torch - need to find a way to cache this otherwise it will take a long time to install

python -m pip install --upgrade pip
pip install -r requirements.txt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need most these requirements to run the file, we mostly need them to run the bot which is not yet properly tested in CI

- name: Create and run training script
run: |
echo "${{ inputs.script_content }}" > train.py
@@ -24,7 +42,7 @@ jobs:

- name: Upload logs
uses: actions/upload-artifact@v3
if: always() # Upload logs whether the job succeeds or fails
if: always()
with:
name: training-logs
path: training.log
4 changes: 2 additions & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
PyGithub
aiohttp
discord.py
audioop-lts # discord.py imports using * syntax
python-dotenv
requests
requests
pytest
58 changes: 58 additions & 0 deletions test_workflow.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import subprocess
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the github action should report how long it takes to run which will be more indicative of actual performance. see my caching prs i noticed caching would be slower than without

import time
import os
import logging

logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s',
datefmt='%Y-%m-%d %H:%M:%S'
)
logger = logging.getLogger(__name__)

def test_pytorch_install():
cache_dir = os.path.expanduser("~/.cache/pip")

if os.path.exists(cache_dir):
subprocess.run(["rm", "-rf", cache_dir])

start_time = time.time()
result = subprocess.run(
["pip", "install", "-r", "requirements.txt"],
capture_output=True,
text=True
)

if result.returncode != 0:
logger.error(f"pip install failed with output:\n{result.stdout}\n{result.stderr}")
raise AssertionError("First pip install failed")

first_run_time = time.time() - start_time
logger.info(f"First run (no cache) took: {first_run_time:.2f} seconds")

start_time = time.time()
result = subprocess.run(
["pip", "install", "-r", "requirements.txt"],
capture_output=True,
text=True
)

if result.returncode != 0:
logger.error(f"Second pip install failed with output:\n{result.stdout}\n{result.stderr}")
raise AssertionError("Second pip install failed")

second_run_time = time.time() - start_time
logger.info(f"Second run (with cache) took: {second_run_time:.2f} seconds")

improvement = (first_run_time - second_run_time) / first_run_time * 100
logger.info(f"Cache improved installation time by {improvement:.1f}%")

assert improvement > 0, "Cache did not improve installation time"
assert improvement > 10, f"Cache improvement ({improvement:.1f}%) was less than 10%"

if __name__ == "__main__":
try:
import pytest
pytest.main([__file__])
except ImportError:
test_pytorch_install()