ash/artemis
1
0
Fork 0
forked from Hay1tsme/artemis
Code Pull Requests Activity

- prettier docker guide

Browse Source 
- update docker-compose to reflect 1 level higher config folder
This commit is contained in:
phantomlan 2023-10-21 17:47:23 +02:00
parent 5420076c8e
commit 12d0a3f927
2 changed files with 53 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docker-compose.yml
View File

@ -5,7 +5,7 @@ services:
build: .
volumes:
- ./aime:/app/aime
- ./config:/app/config
- ./configs/config:/app/config
environment:
CFG_DEV: 1

70
docs/INSTALL_DOCKER.md
View File

@ -1,15 +1,18 @@
# ARTEMiS - Docker Installation Guide
This step-by-step guide will allow you to install a Contenerized Version of ARTEMiS inside Docker, some steps can be skipped assuming you already have pre-requisite components and modules installed.
This guide assumes using Debian 12(bookworm-stable) as a Host Operating System for most of packages and modules.
## Pre-Requisites:
- Linux-Based Operating System (e.g. Debian, Ubuntu)
- Docker (https://get.docker.com)
- Python 3.9+
- (optional) Git
## Install Python3.9+ and Docker
```
(if this is a fresh install of the system)
sudo apt update && sudo apt upgrade
@ -26,6 +29,7 @@ sudo apt install git
```
## Get ARTEMiS
If you installed git, clone into your choice of ARTEMiS git repository, e.g.:
```
git clone <ARTEMiS Repo> <folder>
@ -33,6 +37,7 @@ git clone <ARTEMiS Repo> <folder>
If not, download the source package, and unpack it to the folder of your choice.
## Prepare development/home configuration
To build our Docker setup, first we need to create some folders and copy some files around
- Create 'aime', 'configs', 'AimeDB', and 'logs' folder in ARTEMiS root folder (where all source files exist)
- Inside configs folder, create 'config' folder, and copy all .yaml files from example_config to config (thats all files without nginx_example.conf)
@ -76,18 +81,23 @@ docker compose logs
add '-f' to the end if you want to follow logs.
## Running commands
If you need to execute python scripts supplied with the application, use `docker compose exec app python3 <script> <command>`, for example `docker compose exec app python3 dbutils.py version`
## Persistent DB
By default, in development mode, ARTEMiS database is stored temporarily, if you wish to keep your database saved between restarts, we need to bind the database inside the container to actual storage/folder inside our server, to do this we need to make a few changes:
First off, edit docker-compose.yml, and uncomment 2 lines:
- First off, edit docker-compose.yml, and uncomment 2 lines:
```
(uncomment these two)
#volumes:
# - ./AimeDB:/var/lib/mysql
```
After that, start up the server, this time Database will be saved in AimeDB folder we created in our configuration steps.
If you wish to save it in another folder and/or storage device, change the "./AimeDB" target folder to folder/device of your choice
- After that, start up the server, this time Database will be saved in AimeDB folder we created in our configuration steps.
- If you wish to save it in another folder and/or storage device, change the "./AimeDB" target folder to folder/device of your choice
NOTE (NEEDS FIX): at the moment running development mode with persistent DB will always run database creation script at the start of application, while it doesn't break database outright, it might create some issues, a temporary fix can be applied:
@ -101,11 +111,13 @@ NOTE (NEEDS FIX): at the moment running development mode with persistent DB will
To add data using importer, we can do that a few ways:
### Use importer locally on server
For that we need actual GameData and Options supplied somehow to the server system, be it wsl2 mounting layer, a pendrive with data, network share, or a direct copy to the server storage
With python3 installed on system, install requirements.txt directly to the system, or through python3 virtual-environment (python3-venv)
Default mysql/mariadb client development packages will also be required
In the system:
- In the system:
```
sudo apt install default-libmysqlclient-dev build-essential pkg-config libmemcached-dev
sudo apt install mysql-client
@ -113,12 +125,14 @@ OR
sudo apt install libmariadb-dev
```
In the root ARTEMiS folder
- In the root ARTEMiS folder
```
python3 -m pip install -r requirements.txt
```
If we wish to layer that with python3 virtual-environment, install required system packages, then:
- If we wish to layer that with python3 virtual-environment, install required system packages, then:
```
sudo apt install python3-venv
python3 -m venv /path/to/venv
@ -126,52 +140,66 @@ cd /path/to/venv/bin
python3 -m pip install -r /path/to/artemis/requirements.txt
```
depending on how you installed, now you can run read.py using:
For direct installation, from root ARTEMiS folder:
- Depending on how you installed, now you can run read.py using:
- For direct installation, from root ARTEMiS folder:
```
python3 read.py <args>
```
Or from python3 virtual environment, from root ARTEMiS folder:
- Or from python3 virtual environment, from root ARTEMiS folder:
```
/path/to/python3-venv/bin/python3 /path/to/artemis/read.py <args>
```
We need to expose database container port, so that read.py can communicate with the database, inside docker-compose.yml, uncomment 2 lines in the database container declaration (db):
- We need to expose database container port, so that read.py can communicate with the database, inside docker-compose.yml, uncomment 2 lines in the database container declaration (db):
```
#ports:
# - "3306:3306"
```
Now, `docker compose down && docker compose build && docker compose up -d` to restart containers
- Now, `docker compose down && docker compose build && docker compose up -d` to restart containers
Now to insert the data, by default, docker doesn't expose container hostnames to root system, when trying to run read.py against a container, it will Error that hostname is not available, to fix that, we can add database hostname by hand to /etc/hosts:
```
sudo <editor of your choice> /etc/hosts
add '127.0.0.1 ma.db' to the table
save and close
```
You can remove the line in /etc/hosts and de-expose the database port after successful import (this assumes you're using Persistent DB, as restarting the container without it will clear imported data).
- You can remove the line in /etc/hosts and de-expose the database port after successful import (this assumes you're using Persistent DB, as restarting the container without it will clear imported data).
### Use importer on remote Linux system
Follow the system and python portion of the guide, installing required packages and python3 modules, Download the ARTEMiS source.
Edit core.yaml and insert it into config catalog:
- Edit core.yaml and insert it into config catalog:
```
database:
host: "<hostname of target system>"
```
Expose port 3306 from database docker container to system, and allow port 3306 through system firewall to expose port to the system from which you will be importing data. (Remember to close down the database ports after finishing!)
Import data using read.py
- Expose port 3306 from database docker container to system, and allow port 3306 through system firewall to expose port to the system from which you will be importing data. (Remember to close down the database ports after finishing!)
- Import data using read.py
### Use importer on remote Windows system
Follow the [windows](docs/INSTALL_WINDOWS.md) guide for installing python dependencies, download the ARTEMiS source.
Edit core.yaml and insert it into config catalog:
- Edit core.yaml and insert it into config catalog:
```
database:
host: "<hostname of target system>"
```
Expose port 3306 from database docker container to system, and allow port 3306 through system firewall to expose port to the system from which you will be importing data.
For Windows, also allow port 3306 outside the system so that read.py can communicate with remote database. (Remember to close down the database ports after finishing!)
- Expose port 3306 from database docker container to system, and allow port 3306 through system firewall to expose port to the system from which you will be importing data.
- For Windows, also allow port 3306 outside the system so that read.py can communicate with remote database. (Remember to close down the database ports after finishing!)
# Troubleshooting
@ -186,19 +214,25 @@ Make sure you have a proper AimeDB Key added to configuration.
## Memcached Error in ARTEMiS application causes errors in loading data
Currently when running ARTEMiS from master branch, there is a small bug that causes app to always configure memcached service to 127.0.0.1, to fix that, locate cache.py file in core/data, and edit:
```
memcache = pylibmc.Client([hostname]), binary=True)
```
to:
```
memcache = pylibmc.Client(["ma.memcached"], binary=True)
```
And build the containers again.
This will fix errors loading data from server.
(This is fixed in development branch)
## read.py "Can't connect to local server through socket '/run/mysqld/mysqld.sock'"
sqlalchemy by default reads any ip based connection as socket, thus trying to connect locally, please use a hostname (such as ma.db as in guide, and do not localhost) to force it to use a network interface.
### TODO:
- Production environment