Q-Learning for Sensor Tasking Decisions for Space Domain Awareness

Structure

.
├── SSN_RL                              # Core Python Library 
│   ├── agent                           # - Agent classes, algorithms, functions
│   ├── debug                           # - Logger for debugging and counting events
│   ├── environment                     # - Environment classes: Satellite, Sensors, etc.
│   ├── scenarioBuilder                 # - Functions and classes for building scenarios
│   └── utils                            # - Misc utilities
├── data                                # 
│   └── geo_catalog_14Nov2024.txt       # GEO Catalog from 11/14/2024 
└── scripts                             # Test, Training, Plotting, Drafting 
    ├── drafts                          # 
    ├── experiments                     # All Experiments
    │   ├── DQN                         # - DQN training and testing scripts
    │   ├── Heuristic                   # - Heuristic algorithm testing script
    │   ├── Linear                      # - Linear Q-Learning training and testing scripts
    │   ├── QLearning                   # - Q-Table training and testing scripts
    │   ├── QTable                      # - Q-Table first draft
    │   └── plots                       # - Plot generation and test results
    └── tests                           # Tests for Environment

Developer Environment Setup

1. create conda environment with micromamba

micromamba create -n 184-final-project-rl -c conda-forge -y python=3.10

it used to be

micromamba create -n 184-final-project -c conda-forge -y python=3.12.4

2. activate environment

micromamba activate 184-final-project-rl

Note: sometimes I need to run the following command prior to reactivating

eval "$(micromamba shell hook --shell zsh)"

3. install dependencies

pip install -r requirements.txt

running files

Instructions

Run files from here (.)

python -m scripts.experiments.(Algorithm).(file_name)

test files

python -m scripts.tests.basic-maneuver-sim

python -m scripts.tests.optical-sensing-test

python -m scripts.experiments.DQN.toy1-dqn-train

drafts

python -m scripts.drafts.driver1

python -m scripts.drafts.driver2

Library Documentation

skyfield

skyfield: earth satellites

Astrodynamics Checklist

check the following capabilities

• load states from file or string
• have ground observer
• compute visibility from ground to satellite
• do propagation to create ephemerides
• convert states to pos/vel
• make modifications to states after loading (maneuver simulation)

generate the following scripts

• GEO visibility opportunity plot
• Impulsive maneuver simulation
• Covariance growth plot simulation

Data Documentation

space-track.org (requires an account)

current GEO catalog from space-track