Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cooper dev #5

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
__pycache__/
*.py[cod]

*.log

# Distribution / packaging
bin/
build/
Expand Down
17 changes: 17 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
FROM python:3.9.16
EXPOSE 8501
WORKDIR /src
COPY requirements_3916_docker.txt ./requirements_3916_docker.txt
COPY src/haarcascade_frontalface_default.xml ./haarcascade_frontalface_default.xml
COPY src/final_model.h5 ./final_model.h5
RUN pip3 install -r requirements_3916_docker.txt
RUN apt-get update && apt-get install ffmpeg libsm6 libxext6 -y
COPY . .
# CMD ls
# CMD pwd
CMD streamlit run src/app.py \
--server.headless true \
--browser.serverAddress="0.0.0.0" \
--server.enableCORS false \
--server.enableXsrfProtection=false \
--browser.gatherUsageStats false
81 changes: 75 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ Interactive Chatbot with Real-Time Emotional Analysis/Integration

## Usage

Option 1 (Run locally):

- Clone this repository
- Enter the commands:

Expand All @@ -21,9 +23,74 @@ Interactive Chatbot with Real-Time Emotional Analysis/Integration
- `cd src/`
- `streamlit run app.py`

Option 2 (Docker):

- Download Docker Desktop
- `docker pull rcsnyder/streamlitapp:0.0.1`
- Enter into the app: `sudo docker run --rm -it -v /$(pwd)/src -p 8501:8501 --env OPENAI_API_KEY=xxxx --env OPENAI_ORG_NAME=xxxx --entrypoint bash rcsnyder/streamlitapp:0.0.1`
- (Optional) Add a .env file to the top level directory with the following environment variables.

```
OPENAI_API_KEY=xxxx
OPENAI_ORG_NAME=xxxxx

```

- Start it: `streamlit run app.py --server.headless true --browser.serverAddress="0.0.0.0" --server.enableCORS false --server.enableXsrfProtection=false --browser.gatherUsageStats false`
- Go to `localhost:8501`
- Choose camera device
- Click Start

Option 3 (Deploy on AWS EC2):

- Spin up an EC2 instance on AWS with Ubuntu 22.04. 16gb of RAM, 128gb EBS
- Set the security group for port 80 and 443 to all IP addresses.
- Set a pem key
- Set up an elastic IP Address and attach it to the EC2 Instance.
- Set up a domain in Route53 or Google Domains to point to that Ec2 instance.
- Windows Cmds to get into the ec2 instance:
- `icacls.exe filename.pem /reset`
- `icacls.exe filename.pem /grant:r %username%:(R)`
- `icacls.exe filename.pem /inheritance:r`
- `ssh -i "filename.pem" ubuntu@ec2instancepublicaddress`
- Linux Commands:
- `chmod filename/pem 400`
- `ssh -i "filename.pem" ubuntu@ec2instancepublicaddress`
- EC2 App commands:

- ```
sudo apt-get update
sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt-get install \
ca-certificates \
curl \
gnupg
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo docker pull rcsnyder/streamlitapp:0.0.2
sudo apt-get install -y debian-keyring
sudo apt-get install -y debian-archive-keyring
sudo apt-get install -y apt-transport-https
sudo keyring_location=/usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf \
'https://dl.cloudsmith.io/public/caddy/stable/setup.deb.sh' \
| sudo -E bash
sudo apt-get install caddy=2.6.4
echo -e "DOMAINNAME.com {\n\treverse_proxy localhost:8501\n}" > Caddyfile | echo -e "PUBLIC_EC2IPADDRESS.nip.io {\n\treverse_proxy localhost:8501\n}" > Caddyfile
sudo caddy start
sudo docker run -it --rm -p 8501:8501 --env OPENAI_API_KEY=xxx --env OPENAI_ORG_NAME=xxx rcsnyder/streamlitapp:0.0.2
```

- (Optional to get into the docker container)
- `sudo docker run --env OPENAI_API_KEY=xxxxx --env OPENAI_ORG_NAME=xxxx --rm -it -v /$(pwd) -p 8501:8501 --entrypoint bash rcsnyder/streamlitapp:0.0.2`
- `streamlit run app.py --server.headless true --browser.serverAddress="0.0.0.0" --server.enableCORS false --server.enableXsrfProtection=false --browser.gatherUsageStats false`

- Go to `PUBLIC_EC2IPADDRESS.nip.io` or `DOMAINNAME.com` and choose your camera device and click start.

### User Stories

- [ ] A user would like to be able to start up an app and talk to a chatbot that can percieve their emotions in real-time.
- [ x ] A user would like to be able to start up an app and talk to a chatbot that can percieve their emotions in real-time.
- [ x ] A user would like to be able to choose their initial meta prompt when talking to the chatbot that can percieve their emotions in real-time.

### Technology Stack

Expand All @@ -32,7 +99,7 @@ Interactive Chatbot with Real-Time Emotional Analysis/Integration
- Streamlit or Gradio
- Huggingface Endpoints for emotion detecting pre-trained models
- OpenAI large language models (Chat GPT)
- Hypermodern Python Coding Standards (<https://cookiecutter-hypermodern-python.readthedocs.io/en/2021.3.14/guide.html>)
- Hypermodern Python Coding Standards (<https://cookiecutter-hypermodern-python.readthedocs.io/en/2021.3.14/guide.html>) (Eventually)

### Inspiration Repos/Articles

Expand All @@ -49,6 +116,8 @@ Interactive Chatbot with Real-Time Emotional Analysis/Integration
- <https://stackoverflow.com/questions/69439390/streamlit-image-processing>
- <https://github.com/whitphx/streamlit-fesion>
- <https://github.com/whitphx/streamlit-webrtc/blob/56d715a76247cd6303c9ddade5bc2a8e430ddc2f/app.py#L344>
- <https://bansalanuj.com/https-aws-ec2-without-custom-domain>
- <https://github.com/nicolasmetallo/deploy-streamlit-on-fargate-with-aws-cdk>

### Huggingface

Expand All @@ -75,7 +144,7 @@ Interactive Chatbot with Real-Time Emotional Analysis/Integration

# TODO

- [ ] Flip the conversation history to show most recent messages first
- [ ] Make the form input chat text clear on change.
- [x] figure out how to continuously grab the video image to pipe into another emotion detection api
- [ ] figure out how to continuously update state with the emotion dictionary and wire it up to the chatbot prompt with the average since last updated.
- [ x ] Flip the conversation history to show most recent messages first
- [ x ] Make the form input chat text clear on change.
- [ x ] figure out how to continuously grab the video image to pipe into another emotion detection api
- [ x ] figure out how to continuously update state with the emotion dictionary and wire it up to the chatbot prompt with the normalized average since last updated.
4 changes: 2 additions & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -126,8 +126,8 @@ python-decouple==3.8
python-json-logger==2.0.7
pytz==2022.7.1
pytz-deprecation-shim==0.1.0.post0
pywin32==305
pywinpty==2.0.10
# pywin32==305
# pywinpty==2.0.10
PyYAML==6.0
pyzmq==25.0.2
requests==2.28.2
Expand Down
202 changes: 202 additions & 0 deletions requirements_3916_docker.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
absl-py==1.4.0
aiohttp==3.8.4
aioice==0.9.0
aiortc==1.5.0
aiosignal==1.3.1
altair==4.2.2
altgraph==0.17.3
anyio==3.6.2
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
arrow==1.2.3
asttokens==2.2.1
astunparse==1.6.3
async-timeout==4.0.2
attrs==22.2.0
av==10.0.0
backcall==0.2.0
beautifulsoup4==4.12.2
bleach==6.0.0
blinker==1.6.2
build==0.10.0
cachetools==5.3.0
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==3.1.0
click==8.1.3
colorama==0.4.6
comm==0.1.3
contourpy==1.0.7
cryptography==40.0.2
cycler==0.11.0
debugpy==1.6.7
decorator==5.1.1
deepface==0.0.79
defusedxml==0.7.1
dnspython==2.3.0
docutils==0.19
entrypoints==0.4
executing==1.2.0
fastjsonschema==2.16.3
filelock==3.11.0
fire==0.5.0
Flask==2.2.3
flatbuffers==23.3.3
fonttools==4.39.3
fqdn==1.5.1
frozenlist==1.3.3
gast==0.4.0
gdown==4.7.1
gitdb==4.0.10
GitPython==3.1.31
google-auth==2.17.3
google-auth-oauthlib==1.0.0
google-crc32c==1.5.0
google-pasta==0.2.0
grpcio==1.53.0
gunicorn==20.1.0
h5py==3.8.0
idna==3.4
ifaddr==0.2.0
importlib-metadata==6.3.0
importlib-resources==5.12.0
ipykernel==6.22.0
ipython==8.12.0
ipython-genutils==0.2.0
isoduration==20.11.0
itsdangerous==2.1.2
jaraco.classes==3.2.3
jax==0.4.8
jedi==0.18.2
jeepney==0.8.0
Jinja2==3.1.2
jsonpointer==2.3
jsonschema==4.17.3
jupyter-events==0.6.3
jupyter_client==8.2.0
jupyter_core==5.3.0
jupyter_server==2.5.0
jupyter_server_terminals==0.4.4
jupyterlab-pygments==0.2.2
keras==2.12.0
keyring==23.13.1
kiwisolver==1.4.4
libclang==16.0.0
Markdown==3.4.3
markdown-it-py==2.2.0
MarkupSafe==2.1.2
matplotlib==3.7.1
matplotlib-inline==0.1.6
mdurl==0.1.2
mistune==2.0.5
ml-dtypes==0.1.0
more-itertools==9.1.0
mtcnn==0.1.1
multidict==6.0.4
nbclassic==0.5.5
nbclient==0.7.3
nbconvert==7.3.1
nbformat==5.8.0
nest-asyncio==1.5.6
netifaces==0.11.0
notebook==6.5.4
notebook_shim==0.2.2
numpy==1.23.5
oauthlib==3.2.2
openai==0.27.4
opencv-python==4.7.0.72
opt-einsum==3.3.0
packaging==23.1
pandas==1.5.3
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.5.0
pkginfo==1.9.6
platformdirs==3.2.0
prometheus-client==0.16.0
prompt-toolkit==3.0.38
protobuf==3.20.3
psrecord==1.2
psutil==5.9.4
ptyprocess==0.7.0
pure-eval==0.2.2
pyarrow==11.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.21
pydeck==0.8.0
pyee==9.0.4
Pygments==2.15.0
pyinstaller==5.10.1
pyinstaller-hooks-contrib==2023.2
pylibsrtp==0.8.0
Pympler==1.0.1
pyOpenSSL==23.1.1
pyparsing==3.0.9
pyproject_hooks==1.0.0
pyrsistent==0.19.3
PySocks==1.7.1
python-dateutil==2.8.2
python-decouple==3.8
python-json-logger==2.0.7
pytz==2023.3
pytz-deprecation-shim==0.1.0.post0
PyYAML==6.0
pyzmq==25.0.2
readme-renderer==37.3
requests==2.28.2
requests-oauthlib==1.3.1
requests-toolbelt==0.10.1
retina-face==0.0.13
rfc3339-validator==0.1.4
rfc3986==2.0.0
rfc3986-validator==0.1.1
rich==13.3.4
rsa==4.9
scipy==1.10.1
SecretStorage==3.3.3
semver==3.0.0
Send2Trash==1.8.0
six==1.16.0
smmap==5.0.0
sniffio==1.3.0
soupsieve==2.4
stack-data==0.6.2
streamlit==1.21.0
streamlit-chat==0.0.2.2
streamlit-webrtc==0.45.0
tensorboard==2.12.2
tensorboard-data-server==0.7.0
tensorboard-plugin-wit==1.8.1
tensorflow==2.12.0
tensorflow-cpu==2.12.0
tensorflow-estimator==2.12.0
tensorflow-intel==0.0.1
tensorflow-io-gcs-filesystem==0.32.0
termcolor==2.2.0
terminado==0.17.1
tinycss2==1.2.1
toml==0.10.2
tomli==2.0.1
toolz==0.12.0
tornado==6.2
tqdm==4.65.0
traitlets==5.9.0
twine==4.0.2
typing_extensions==4.5.0
tzdata==2023.3
tzlocal==4.3
uri-template==1.2.0
urllib3==1.26.15
validators==0.20.0
watchdog==3.0.0
wcwidth==0.2.6
webcolors==1.13
webencodings==0.5.1
websocket-client==1.5.1
Werkzeug==2.2.3
wrapt==1.14.1
yarl==1.8.2
zipp==3.15.0
Loading