pokken: add additional logging

Reviewed-on: #147

LICENSE.txt

@@ -0,0 +1,13 @@
+           DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
+                   Version 2, December 2004
+ 
+Copyright (C) 2004 Sam Hocevar <sam@hocevar.net>
+
+Everyone is permitted to copy and distribute verbatim or modified
+copies of this license document, and changing it is allowed as long
+as the name is changed.
+ 
+           DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
+  TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+ 0. You just DO WHAT THE FUCK YOU WANT TO.

changelog.md

@@ -1,6 +1,54 @@
 # Changelog
 Documenting updates to ARTEMiS, to be updated every time the master branch is pushed to.
 
+## 20240530
+### DIVA
++ Fix reader for when dificulty is not a int
+
+## 20240526
+### DIVA
++ Fixed missing awaits causing coroutine error
+
+## 20240524
+### DIVA
++ Fixed new profile start request causing coroutine error
+
+## 20240523
+### DIVA
++ Fixed binary handler & render_POST errors
+
+## 20240408
+### System
++ Modified the game specific documentation
+
+## 20240407
+### Maimai
++ Support maimai DX International [#118](https://gitea.tendokyu.moe/Hay1tsme/artemis/pulls/118) (Thanks beerpsi!)
++ Fixed the maimai DX reboot time from config [#120](https://gitea.tendokyu.moe/Hay1tsme/artemis/pulls/120) (Thanks topty!)
+
+## 20240318
+### CXB
++ Fixing handle_data_shop_list_detail_request for Sunrise S1
+
+## 20240302
+### SAO
++ Fixing new profile creation with right heroes and start VP
++ Fix to the Unanalyzed Log responses returning the wrong rewards
++ Documentation revised
+
+## 20240226
+### CXB
++ Fixing paths for rev.py
++ Changed encoding for handle_data_item_list_icon_request
+
+## 20240202
+### SAO
++ Added reader assets and edited the game specific documentation
+
+## 20240118
+### System
++ Added game version names to the readme
+
 ## 20240109
 ### System
 + Removed `ADD config config` from dockerfile [#83](https://gitea.tendokyu.moe/Hay1tsme/artemis/pulls/83) (Thanks zaphkito!)

contributing.md

@@ -1,8 +1,182 @@
 # Contributing to ARTEMiS
-If you would like to contribute to artemis, either by adding features, games, or fixing bugs, you can do so by forking the repo and submitting a pull request [here](https://gitea.tendokyu.moe/Hay1tsme/artemis/pulls). Please make sure, if you're submitting a PR for a game or game version, that you're following the n-0/y-1 guidelines, or it will be rejected.
+If you would like to contribute to artemis, either by adding features, games, or fixing bugs, you can do so by forking the repo and submitting a pull request [here](https://gitea.tendokyu.moe/Hay1tsme/artemis/pulls). This guide assume you're familiar with both git, python, and the libraries that artemis uses.
+
+This document is a work in progress. If you have any questions or notice any errors, please report it to the discord.
 
 ## Adding games
-Guide WIP
+### Step 0
++ Follow the "n-1" rule of thumb. PRs for game versions that are currently active in arcades will be deleted. If you're unsure, ask!
++ Always PR against the `develop` branch.
++ Check to see if somebody else is already PRing the features/games you want to add. If they are, consider contributing to them rather then making an entirely new PR.
++ We don't technically have a written code style guide (TODO) but try to keep your code consistant with code that's already there where possible.
+
+### Step 1 (Setup)
+1) Fork the gitea repo, clone your fork, and checkout the develop branch.
+2) Make a new folder in the `titles` folder, name it some recogniseable shorthand for your game (Chunithm becomes chuni, maimai dx is mai2, etc)
+3) In this new folder, create a file named `__init__.py`. This is the first thing that will load when your title module is loaded by the core system, and it acts as sort of a directory for where everything lives in your module. This file will contain the following required items:
+    + `index`: must point to a subclass of `BaseServlet` that will handle setup and dispatching of your game.
+    + `game_codes`: must be a list of 4 letter SEGA game codes as strings.
+
+    It can also contain the following optional fields:
+    + `database`: points to a subclass of `Data` that contains one or more subclasses of `BaseData` that act as database transaction handlers. Required for the class to store and retrieve data from the database.
+    + `reader`: points to a subclass of `BaseReader` that handles importing static data from game files into the database.
+    + `frontend`: points to a subclass of `FE_Base` that handles frontend routes for your game.
+
+    The next step will focus on `index`
+
+### Step 2 (Index)
+1) Create another file in your game's folder. By convention, it should be called `index.py`.
+
+2) Inside `index.py`, add the following code, replacing {Game name here} with the name of your game, without spaces or special characters. Look at other titles for examples.
+```py
+from core.title import BaseServlet
+from core import CoreConfig
+
+class {Game name here}Servlet(BaseServlet):
+    def __init__(self, core_cfg: CoreConfig, cfg_dir: str) -> None:
+        pass
+```
+3) The `__init__` function should acomplish the following:
+    + Reading your game's config
+    + Setting up your games logger
+    + Instancing your games versions
+    
+    It's usually safe to copy and paste the `__init__` functions from other games, just make sure you change everything that needs to be changed!
+
+4) Go back to the `__init__.py` that you created and add the following:
+```py
+from .index import {Game name here}Servlet
+
+index = {Game name here}Servlet
+```
+
+5) Going back to `index.py`, within the Servlet class, define the following functions from `BaseServlet` as needed (see function documentation):
+    + `is_game_enabled`: Returns true if the game is enabled and should be served, false otherwise. Returns false by default, so override this to allow your game to be served.
+    + `get_routes`: Returns a list of Starlette routes that your game will serve.
+    + `get_allnet_info`: Returns a tuple of strings where the first is the allnet uri and the second is the allnet host. The function takes the game ID, version and keychip ID as parameters, so you can send different responses if need be.
+    + `get_mucha_info`: Only used by games that use Mucha as authentication. Returns a tuple where the first is a bool that is weather or not the game is enabled, the 2nd is a list of game CDs as strings that this servlet should handle, and the 3rd is a list of netID prefixes that each game CD should use. If your game does not use mucha, do not define this function.
+    + `setup`: Preforms any setup your servlet requires, such as spinning up matching servers. It is run once when the server starts. If you don't need any setup, do not define.
+
+6) Make sure any functions you specify to handle routes in `get_routes` are defined as async, as follows: `async def handle_thing(self, request: Request) -> Response:` where Response is whatever kind of Response class you'll be returning. Make sure all paths in this function return some subclass of Response, otherwise you'll get an error when serving.
+
+### Step 3 (Constants)
+1) In your game's folder, create a file to store static values for your game. By convention, we call this `const.py`
+
+2) Inside, create a class called `{Game name here}Constants`. Do not define an `__init__` function.
+
+3) Put constants related to your game here. A good example of something to put here is game codes.
+```py
+class {Game name here}Constants:
+    GAME_CODE = "SBXX"
+    CONFIG_NAME = "{game name}.yaml"
+```
+
+4) If you choose to put game codes in here, add this to your `__init__.py` file:
+```py
+from .const import {Game name here}Constants
+...
+game_codes = [{Game name here}Constants.GAME_CODE]
+```
+
+### Step 4 (Config)
+1) Make a file to store your game's config. By convention, it should be called `config.py`
+
+2) Inside that file, add the following:
+```py
+from core.config import CoreConfig
+
+class {game name}ServerConfig:
+    def __init__(self, parent_config: "{game name}Config") -> None:
+        self.__config = parent_config
+
+    @property
+    def enable(self) -> bool:
+        return CoreConfig.get_config_field(
+            self.__config, "{game name}", "server", "enable", default=True
+        )
+
+    @property
+    def loglevel(self) -> int:
+        return CoreConfig.str_to_loglevel(
+            CoreConfig.get_config_field(
+                self.__config, "{game name}", "server", "loglevel", default="info"
+            )
+        )
+
+class {game name}Config(dict):
+    def __init__(self) -> None:
+        self.server = {game name}ServerConfig(self)
+```
+
+3) In the `example_config` folder, create a yaml file for your game. By convention, it should be called `{game folder name}.ymal`. Add the following:
+```yaml
+server:
+    enable: True
+    loglevel: "info"
+```
+
+4) Add any additional config options that you feel the game needs. Look to other games for config examples.
+
+5) In `index.py` import your config and instance it in `__init__` with:
+```py
+self.game_cfg = {game folder name}Config()
+if path.exists(f"{cfg_dir}/{game folder name}Constants.CONFIG_NAME}"):
+    self.game_cfg.update(
+        yaml.safe_load(open(f"{cfg_dir}/{game folder name}Constants.CONFIG_NAME}"))
+    )
+```
+This will attempt to load the config file you specified in your constants, and if not, go with the defaults specified in `config.py`. This game_cfg object can then be passed down to your handlers when you create them.
+
+At this stage your game should be loaded by allnet, and serve whatever routes you put in `get_routes`. See the next section about adding versions and handlers.
+
+### Step 5 (Database)
+TODO
+
+### Step 6 (Frontend)
+TODO
+
+### Step 7 (Reader)
+TODO
 
 ## Adding game versions
-Guide WIP
+See the above section about code expectations and how to PR.
+1) In the game's folder, create a python file to contain the version handlers. By convention, the first version is version 0, and is stored in `base.py`. Versions following that increment the version number, and are stored in `{short version name}.py`. See Wacca's folder for an example of how to name versions.
+
+2) Internal version numbers should be defined in `const.py`. The version should change any time the game gets a major update (i.e. a new version or plus version.)
+```py
+# in const.py
+VERSION_{game name} = 0
+VERSION_{game name}_PLUS = 1
+```
+
+3) Inside `base.py` (or whatever your version is named) add the following:
+```py
+class {game name}Base:
+    def __init__(self, cfg: CoreConfig, game_cfg: {game name}Config) -> None:
+        self.game_config = game_cfg
+        self.core_config = cfg
+        self.version = {game name}Constants.VERSION_{game name}
+        self.data = {game name}Data(cfg)
+        # Any other initialization stuff
+```
+
+4) Define your handlers. This will vary wildly by game, but best practice is to keep the naming consistant, so that the main dispatch function in `index.py` can use `getattr` to get the handler, rather then having a static list of what endpoint or request type goes to which handler. See Wacca's `index.py` and `base.py` for examples of how to do this.
+
+5) If your version is not the base version, make sure it inherits from the base version:
+```py
+class {game name}Plus({game name}Base):
+    def __init__(self, cfg: CoreConfig, game_cfg: {game name}Config) -> None:
+        super().__init__(cfg, game_cfg)
+        self.version = {game name}Constants.VERSION_{game name}_PLUS
+```
+
+6) Back in `index.py` make sure to import your new class, and add it to `__init__`. Some games may opt to just a single list called `self.versions` that contains all the version classes at their internal version's index. Others may simply define them as seperate members. See Wacca for an example of `self.versions`
+
+7) Add your version to your game's dispatching logic.
+
+8) Test to make sure your game is being handled properly.
+
+9) Submit a PR.
+
+## Adding/improving core services
+If you intend to submit improvements or additions to core services (allnet, mucha, billing, aimedb, database, etc) please get in touch with a maintainer.

core/aimedb.py

@@ -66,7 +66,8 @@ class AimedbServlette():
     
     def start(self) -> None:
         self.logger.info(f"Start on port {self.config.aimedb.port}")
-        asyncio.create_task(asyncio.start_server(self.dataReceived, self.config.server.listen_address, self.config.aimedb.port))
+        addr = self.config.aimedb.listen_address if self.config.aimedb.listen_address else self.config.server.listen_address
+        asyncio.create_task(asyncio.start_server(self.dataReceived, addr, self.config.aimedb.port))
     
     async def dataReceived(self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter):
         self.logger.debug(f"Connection made from {writer.get_extra_info('peername')[0]}")
@@ -190,6 +191,9 @@ class AimedbServlette():
         self.logger.info(
             f"access_code {req.access_code} -> user_id {ret.user_id}"
         )
+        
+        if user_id and user_id > 0:
+            await self.data.card.update_card_last_login(req.access_code)
         return ret
 
     async def handle_lookup_ex(self, data: bytes, resp_code: int) -> ADBBaseResponse:
@@ -219,6 +223,8 @@ class AimedbServlette():
                 self.logger.debug(f"Generated auth token {auth_key}")
                 ret.auth_key = auth_key_full
 
+        if user_id and user_id > 0:
+            await self.data.card.update_card_last_login(req.access_code)
         return ret
 
     async def handle_felica_lookup(self, data: bytes, resp_code: int) -> bytes:
@@ -267,6 +273,8 @@ class AimedbServlette():
                 f"Registration blocked!: access code {ac} (IDm: {req.idm} PMm: {req.pmm})"
             )
 
+        if user_id > 0:
+            await self.data.card.update_card_last_login(ac)
         return ADBFelicaLookupResponse.from_req(req.head, ac)
 
     async def handle_felica_lookup_ex(self, data: bytes, resp_code: int) -> bytes:
@@ -290,7 +298,9 @@ class AimedbServlette():
                 auth_key_full = auth_key.encode() + (b"\0" * auth_key_extra_len)
                 self.logger.debug(f"Generated auth token {auth_key}")
                 resp.auth_key = auth_key_full
-
+        
+        if user_id and user_id > 0:
+            await self.data.card.update_card_last_login(access_code)
         return resp
 
     async def handle_campaign_clear(self, data: bytes, resp_code: int) -> ADBBaseResponse:
@@ -332,6 +342,9 @@ class AimedbServlette():
         if resp.user_id <= 0:
             resp.head.status = ADBStatus.BAN_SYS # Closest we can get to a "You cannot register"
 
+        else:
+            await self.data.card.update_card_last_login(req.access_code)
+
         return resp
 
     # TODO: Save these in some capacity, as deemed relevant

core/allnet.py

@@ -94,6 +94,7 @@ class DLI_STATUS(Enum):
             return cls.UNKNOWN
 
 class AllnetServlet:
+    allnet_registry: Dict[str, Any] = {}
     def __init__(self, core_cfg: CoreConfig, cfg_folder: str):
         self.config = core_cfg
         self.config_folder = cfg_folder
@@ -125,11 +126,13 @@ class AllnetServlet:
 
     def startup(self) -> None:
         self.logger.info(f"Ready on port {self.config.allnet.port if self.config.allnet.standalone else self.config.server.port}")
+        if not TitleServlet.title_registry:
+            TitleServlet(self.config, self.config_folder)
 
     async def handle_poweron(self, request: Request):
         request_ip = Utils.get_ip_addr(request)
         pragma_header = request.headers.get('Pragma', "")
-        is_dfi = pragma_header is not None and pragma_header == "DFI"
+        is_dfi = pragma_header == "DFI"
         data = await request.body()
         
         try:
@@ -168,7 +171,7 @@ class AllnetServlet:
         if machine is None and not self.config.server.allow_unregistered_serials:
             msg = f"Unrecognised serial {req.serial} attempted allnet auth from {request_ip}."
             await self.data.base.log_event(
-                "allnet", "ALLNET_AUTH_UNKNOWN_SERIAL", logging.WARN, msg
+                "allnet", "ALLNET_AUTH_UNKNOWN_SERIAL", logging.WARN, msg, {"serial": req.serial}, None, None, None, request_ip, req.game_id, req.ver
             )
             self.logger.warning(msg)
 
@@ -180,9 +183,9 @@ class AllnetServlet:
             arcade = await self.data.arcade.get_arcade(machine["arcade"])
             if self.config.server.check_arcade_ip:
                 if arcade["ip"] and arcade["ip"] is not None and arcade["ip"] != req.ip:
-                    msg = f"Serial {req.serial} attempted allnet auth from bad IP {req.ip} (expected {arcade['ip']})."
+                    msg = f"{req.serial} attempted allnet auth from bad IP {req.ip} (expected {arcade['ip']})."
                     await self.data.base.log_event(
-                        "allnet", "ALLNET_AUTH_BAD_IP", logging.ERROR, msg
+                        "allnet", "ALLNET_AUTH_BAD_IP", logging.ERROR, msg, {}, None, arcade['id'], machine['id'], request_ip, req.game_id, req.ver
                     )
                     self.logger.warning(msg)
 
@@ -191,9 +194,9 @@ class AllnetServlet:
                     return PlainTextResponse(urllib.parse.unquote(urllib.parse.urlencode(resp_dict)) + "\n")
                 
                 elif (not arcade["ip"] or arcade["ip"] is None) and self.config.server.strict_ip_checking:
-                    msg = f"Serial {req.serial} attempted allnet auth from bad IP {req.ip}, but arcade {arcade['id']} has no IP set! (strict checking enabled)."
+                    msg = f"{req.serial} attempted allnet auth from bad IP {req.ip}, but arcade {arcade['id']} has no IP set! (strict checking enabled)."
                     await self.data.base.log_event(
-                        "allnet", "ALLNET_AUTH_NO_SHOP_IP", logging.ERROR, msg
+                        "allnet", "ALLNET_AUTH_NO_SHOP_IP", logging.ERROR, msg, {}, None, arcade['id'], machine['id'], request_ip, req.game_id, req.ver
                     )
                     self.logger.warning(msg)
 
@@ -201,7 +204,17 @@ class AllnetServlet:
                     resp_dict = {k: v for k, v in vars(resp).items() if v is not None}
                     return PlainTextResponse(urllib.parse.unquote(urllib.parse.urlencode(resp_dict)) + "\n")
 
+            if machine['game'] and machine['game'] != req.game_id:
+                msg = f"{req.serial} attempted allnet auth with bad game ID {req.game_id} (expected {machine['game']})."
+                await self.data.base.log_event(
+                    "allnet", "ALLNET_AUTH_BAD_GAME", logging.ERROR, msg, {}, None, arcade['id'], machine['id'], request_ip, req.game_id, req.ver
+                )
+                self.logger.warning(msg)
 
+                resp.stat = ALLNET_STAT.bad_game.value
+                resp_dict = {k: v for k, v in vars(resp).items() if v is not None}
+                return PlainTextResponse(urllib.parse.unquote(urllib.parse.urlencode(resp_dict)) + "\n")
+            
             country = (
                 arcade["country"] if machine["country"] is None else machine["country"]
             )
@@ -209,7 +222,7 @@ class AllnetServlet:
                 country = AllnetCountryCode.JAPAN.value
 
             resp.country = country
-            resp.place_id = arcade["id"]
+            resp.place_id = f"{arcade['id']:04X}"
             resp.allnet_id = machine["id"]
             resp.name = arcade["name"] if arcade["name"] is not None else ""
             resp.nickname = arcade["nickname"] if arcade["nickname"] is not None else ""
@@ -233,11 +246,14 @@ class AllnetServlet:
                 arcade["timezone"] if arcade["timezone"] is not None else "+0900" if req.format_ver == 3 else "+09:00"
             )
         
+        else:
+            arcade = None
+
         if req.game_id not in TitleServlet.title_registry:
             if not self.config.server.is_develop:
                 msg = f"Unrecognised game {req.game_id} attempted allnet auth from {request_ip}."
                 await self.data.base.log_event(
-                    "allnet", "ALLNET_AUTH_UNKNOWN_GAME", logging.WARN, msg
+                    "allnet", "ALLNET_AUTH_UNKNOWN_GAME", logging.WARN, msg, {}, None, arcade['id'] if arcade else None, machine['id'] if machine else None, request_ip, req.game_id, req.ver
                 )
                 self.logger.warning(msg)
 
@@ -260,27 +276,45 @@ class AllnetServlet:
 
         
         int_ver = req.ver.replace(".", "")
-        resp.uri, resp.host = TitleServlet.title_registry[req.game_id].get_allnet_info(req.game_id, int(int_ver), req.serial)
+        try:
+            resp.uri, resp.host = TitleServlet.title_registry[req.game_id].get_allnet_info(req.game_id, int(int_ver), req.serial)
+        except Exception as e:
+            self.logger.error(f"Error running get_allnet_info for {req.game_id} - {e}")
+            resp.stat = ALLNET_STAT.bad_game.value
+            resp_dict = {k: v for k, v in vars(resp).items() if v is not None}
+            return PlainTextResponse(urllib.parse.unquote(urllib.parse.urlencode(resp_dict)) + "\n")
 
-        msg = f"{req.serial} authenticated from {request_ip}: {req.game_id} v{req.ver}"
-        await self.data.base.log_event("allnet", "ALLNET_AUTH_SUCCESS", logging.INFO, msg)
+        if machine and arcade:
+            msg = f"{req.serial} authenticated from {request_ip}: {req.game_id} v{req.ver}"
+            await self.data.base.log_event(
+                "allnet", "ALLNET_AUTH_SUCCESS", logging.INFO, msg, {}, None, arcade['id'], machine['id'], request_ip, req.game_id, req.ver
+                )
+        else:
+            msg = f"Allow unregistered serial {req.serial} to authenticate from {request_ip}: {req.game_id} v{req.ver}"
+            await self.data.base.log_event(
+                "allnet", "ALLNET_AUTH_SUCCESS_UNREG", logging.INFO, msg, {"serial": req.serial}, None, None, None, request_ip, req.game_id, req.ver
+                )
+        
         self.logger.info(msg)
 
         resp_dict = {k: v for k, v in vars(resp).items() if v is not None}
-        resp_str = urllib.parse.unquote(urllib.parse.urlencode(resp_dict))
+        resp_str = urllib.parse.unquote(urllib.parse.urlencode(resp_dict)) + "\n"
         self.logger.debug(f"Allnet response: {resp_dict}")        
-        resp_str += "\n"
 
-        """if is_dfi:
-            request.responseHeaders.addRawHeader('Pragma', 'DFI')
-            return self.to_dfi(resp_str)"""
+        if is_dfi:
+            return PlainTextResponse(
+                content=self.to_dfi(resp_str) + b"\r\n",
+                headers={
+                    "Pragma": "DFI",
+                },
+            )
 
         return PlainTextResponse(resp_str)
 
     async def handle_dlorder(self, request: Request):
         request_ip = Utils.get_ip_addr(request)
         pragma_header = request.headers.get('Pragma', "")
-        is_dfi = pragma_header is not None and pragma_header == "DFI"
+        is_dfi = pragma_header == "DFI"
         data = await request.body()
         
         try:
@@ -317,7 +351,11 @@ class AllnetServlet:
         ):
             return PlainTextResponse(urllib.parse.unquote(urllib.parse.urlencode(vars(resp))) + "\n")
 
-        else:  # TODO: Keychip check
+        else:
+            machine = await self.data.arcade.get_machine(req.serial)
+            if not machine or not machine['ota_enable'] or not machine['is_cab'] or machine['is_blacklisted']:
+                return PlainTextResponse(urllib.parse.unquote(urllib.parse.urlencode(vars(resp))) + "\n")
+
             if path.exists(
                 f"{self.config.allnet.update_cfg_folder}/{req.game_id}-{req.ver.replace('.', '')}-app.ini"
             ):
@@ -328,25 +366,38 @@ class AllnetServlet:
             ):
                 resp.uri += f"|http://{self.config.server.hostname}:{self.config.server.port}/dl/ini/{req.game_id}-{req.ver.replace('.', '')}-opt.ini"
 
-            self.logger.debug(f"Sending download uri {resp.uri}")
-            await self.data.base.log_event("allnet", "DLORDER_REQ_SUCCESS", logging.INFO, f"{Utils.get_ip_addr(request)} requested DL Order for {req.serial} {req.game_id} v{req.ver}")
+            if resp.uri:
+                self.logger.info(f"Sending download uri {resp.uri}")
+                await self.data.base.log_event(
+                    "allnet", "DLORDER_REQ_SUCCESS", logging.INFO, f"Send download URI to {req.serial} for {req.game_id} v{req.ver} from {Utils.get_ip_addr(request)}", {"uri": resp.uri}, None,
+                    machine['arcade'], machine['id'], request_ip, req.game_id, req.ver
+                )
+            # Maybe add a log event for checkin but no url sent?
 
             res_str = urllib.parse.unquote(urllib.parse.urlencode(vars(resp))) + "\n"
-            """if is_dfi:
-                request.responseHeaders.addRawHeader('Pragma', 'DFI')
-                return self.to_dfi(res_str)"""
+            
+            if is_dfi:
+                return PlainTextResponse(
+                    content=self.to_dfi(res_str) + b"\r\n",
+                    headers={
+                        "Pragma": "DFI",
+                    },
+                )
 
             return PlainTextResponse(res_str)
 
     async def handle_dlorder_ini(self, request: Request) -> bytes:
         req_file = request.path_params.get("file", "").replace("%0A", "").replace("\n", "")
+        request_ip = Utils.get_ip_addr(request)
 
         if not req_file:
             return PlainTextResponse(status_code=404)
 
         if path.exists(f"{self.config.allnet.update_cfg_folder}/{req_file}"):
-            self.logger.info(f"Request for DL INI file {req_file} from {Utils.get_ip_addr(request)} successful")
-            await self.data.base.log_event("allnet", "DLORDER_INI_SENT", logging.INFO, f"{Utils.get_ip_addr(request)} successfully recieved {req_file}")
+            self.logger.info(f"Request for DL INI file {req_file} from {request_ip} successful")
+            await self.data.base.log_event(
+                "allnet", "DLORDER_INI_SENT", logging.INFO, f"{request_ip} successfully recieved {req_file}", {"file": req_file}, ip=request_ip
+            )
             
             return PlainTextResponse(open(
                 f"{self.config.allnet.update_cfg_folder}/{req_file}", "r", encoding="utf-8"
@@ -384,7 +435,13 @@ class AllnetServlet:
         msg = f"{rep.serial} @ {client_ip} reported {rep.rep_type.name} download state {rep.rf_state.name} for {rep.gd} v{rep.dav}:"\
               f" {rep.tdsc}/{rep.tsc} segments downloaded for working files {rep.wfl} with {rep.dfl if rep.dfl else 'none'} complete."
         
-        await self.data.base.log_event("allnet", "DL_REPORT", logging.INFO, msg, dl_data)
+        machine = await self.data.arcade.get_machine(rep.serial)
+        if machine:
+            await self.data.base.log_event("allnet", "DL_REPORT", logging.INFO, msg, dl_data, None, machine['arcade'], machine['id'], client_ip, rep.gd, rep.dav)
+        
+        else:
+            msg = "Unknown serial " + msg
+            await self.data.base.log_event("allnet", "DL_REPORT_UNREG", logging.INFO, msg, dl_data, None, None, None, client_ip, rep.gd, rep.dav)
         self.logger.info(msg)
 
         return PlainTextResponse("OK")
@@ -404,14 +461,24 @@ class AllnetServlet:
         if serial is None or num_files_dld is None or num_files_to_dl is None or dl_state is None:
             return PlainTextResponse("NG")
 
-        self.logger.info(f"LoaderStateRecorder Request from {ip} {serial}: {num_files_dld}/{num_files_to_dl} Files download (State: {dl_state})")
+        msg = f"LoaderStateRecorder Request from {ip} {serial}: {num_files_dld}/{num_files_to_dl} Files download (State: {dl_state})"
+        machine = await self.data.arcade.get_machine(serial)
+        if machine:
+            await self.data.base.log_event("allnet", "LSR_REPORT", logging.INFO, msg, req_dict, None, machine['arcade'], machine['id'], ip)
+        
+        else:
+            msg = "Unregistered " + msg
+            await self.data.base.log_event("allnet", "LSR_REPORT_UNREG", logging.INFO, msg, req_dict, None, None, None, ip)
+        
+        self.logger.info(msg)
         return PlainTextResponse("OK")
     
     async def handle_alive(self, request: Request) -> bytes:
         return PlainTextResponse("OK")
 
     async def handle_naomitest(self, request: Request) -> bytes:
-        self.logger.info(f"Ping from {Utils.get_ip_addr(request)}")
+        # This could be spam-able, removing
+        #self.logger.info(f"Ping from {Utils.get_ip_addr(request)}")
         return PlainTextResponse("naomi ok")
 
     def allnet_req_to_dict(self, data: str) -> Optional[List[Dict[str, Any]]]:
@@ -541,18 +608,35 @@ class BillingServlet:
         if machine is None and not self.config.server.allow_unregistered_serials:
             msg = f"Unrecognised serial {req.keychipid} attempted billing checkin from {request_ip} for {req.gameid} v{req.gamever}."
             await self.data.base.log_event(
-                "allnet", "BILLING_CHECKIN_NG_SERIAL", logging.WARN, msg
+                "allnet", "BILLING_CHECKIN_NG_SERIAL", logging.WARN, msg, ip=request_ip, game=req.gameid, version=req.gamever
             )
             self.logger.warning(msg)
 
             return PlainTextResponse(f"result=1&requestno={req.requestno}&message=Keychip Serial bad\r\n")
+        
+        log_details = {
+            "playcount": req.playcnt,
+            "billing_type": req.billingtype.name,
+            "nearfull": req.nearfull,
+            "playlimit": req.playlimit,
+        }
+        
+        if machine is not None:
+            await self.data.base.log_event("billing", "BILLING_CHECKIN_OK", logging.INFO, "", log_details, None, machine['arcade'], machine['id'], request_ip, req.gameid, req.gamever)
+            
+            self.logger.info(
+                f"Unregistered Billing checkin from {request_ip}: game {req.gameid} ver {req.gamever} keychip {req.keychipid} playcount "
+                f"{req.playcnt} billing_type {req.billingtype.name} nearfull {req.nearfull} playlimit {req.playlimit}"
+            )
+        else:
+            log_details['serial'] = req.keychipid
+            await self.data.base.log_event("billing", "BILLING_CHECKIN_OK_UNREG", logging.INFO, "", log_details, None, None, None, request_ip, req.gameid, req.gamever)
+        
+            self.logger.info(
+                f"Unregistered Billing checkin from {request_ip}: game {req.gameid} ver {req.gamever} keychip {req.keychipid} playcount "
+                f"{req.playcnt} billing_type {req.billingtype.name} nearfull {req.nearfull} playlimit {req.playlimit}"
+            )
 
-        msg = (
-            f"Billing checkin from {request_ip}: game {req.gameid} ver {req.gamever} keychip {req.keychipid} playcount "
-            f"{req.playcnt} billing_type {req.billingtype.name} nearfull {req.nearfull} playlimit {req.playlimit}"
-        )
-        self.logger.info(msg)
-        await self.data.base.log_event("billing", "BILLING_CHECKIN_OK", logging.INFO, msg)
         if req.traceleft > 0:
             self.logger.warn(f"{req.traceleft} unsent tracelogs")
         kc_playlimit = req.playlimit
@@ -696,7 +780,7 @@ class BillingInfo:
             self.boardid = str(data.get("boardid", None))
             self.tenpoip = str(data.get("tenpoip", None))
             self.libalibver = float(data.get("libalibver", None))
-            self.data.max = int(data.get("datamax", None))
+            self.datamax = int(data.get("datamax", None))
             self.billingtype = BillingType(int(data.get("billingtype", None)))
             self.protocolver = float(data.get("protocolver", None))
             self.operatingfix = bool(data.get("operatingfix", None))
@@ -876,7 +960,7 @@ class DLReport:
         
         return True
 
-cfg_dir = environ.get("DIANA_CFG_DIR", "config")
+cfg_dir = environ.get("ARTEMIS_CFG_DIR", "config")
 cfg: CoreConfig = CoreConfig()
 if path.exists(f"{cfg_dir}/core.yaml"):
     cfg.update(yaml.safe_load(open(f"{cfg_dir}/core.yaml")))

core/config.py

@@ -81,7 +81,7 @@ class ServerConfig:
         is_using_proxy is True and this value is non-zero
         """
         return CoreConfig.get_config_field(
-            self.__config, "core", "title", "proxy_port", default=0
+            self.__config, "core", "server", "proxy_port", default=0
         )
 
     @property
@@ -91,7 +91,7 @@ class ServerConfig:
         instead of 'port' if is_using_proxy is True and this value is non-zero
         """
         return CoreConfig.get_config_field(
-            self.__config, "core", "title", "proxy_port_ssl", default=0
+            self.__config, "core", "server", "proxy_port_ssl", default=0
         )
 
     @property
@@ -173,7 +173,7 @@ class DatabaseConfig:
     @property
     def protocol(self) -> str:
         return CoreConfig.get_config_field(
-            self.__config, "core", "database", "type", default="mysql"
+            self.__config, "core", "database", "protocol", default="mysql"
         )
 
     @property
@@ -320,6 +320,12 @@ class AimedbConfig:
             self.__config, "core", "aimedb", "enable", default=True
         )
 
+    @property
+    def listen_address(self) -> bool:
+        return CoreConfig.get_config_field(
+            self.__config, "core", "aimedb", "listen_address", default=""
+        )
+
     @property
     def loglevel(self) -> int:
         return CoreConfig.str_to_loglevel(

core/data/alembic/versions/2bf9f38d9444_add_event_log_info.py

@@ -0,0 +1,48 @@
+"""add_event_log_info
+
+Revision ID: 2bf9f38d9444
+Revises: 81e44dd6047a
+Create Date: 2024-05-21 23:00:17.468407
+
+"""
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import mysql
+
+# revision identifiers, used by Alembic.
+revision = '2bf9f38d9444'
+down_revision = '81e44dd6047a'
+branch_labels = None
+depends_on = None
+
+
+def upgrade():
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.add_column('event_log', sa.Column('user', sa.INTEGER(), nullable=True))
+    op.add_column('event_log', sa.Column('arcade', sa.INTEGER(), nullable=True))
+    op.add_column('event_log', sa.Column('machine', sa.INTEGER(), nullable=True))
+    op.add_column('event_log', sa.Column('ip', sa.TEXT(length=39), nullable=True))
+    op.alter_column('event_log', 'when_logged',
+               existing_type=mysql.TIMESTAMP(),
+               server_default=sa.text('now()'),
+               existing_nullable=False)
+    op.create_foreign_key(None, 'event_log', 'machine', ['machine'], ['id'], onupdate='cascade', ondelete='cascade')
+    op.create_foreign_key(None, 'event_log', 'arcade', ['arcade'], ['id'], onupdate='cascade', ondelete='cascade')
+    op.create_foreign_key(None, 'event_log', 'aime_user', ['user'], ['id'], onupdate='cascade', ondelete='cascade')
+    # ### end Alembic commands ###
+
+
+def downgrade():
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.drop_constraint(None, 'event_log', type_='foreignkey')
+    op.drop_constraint(None, 'event_log', type_='foreignkey')
+    op.drop_constraint(None, 'event_log', type_='foreignkey')
+    op.alter_column('event_log', 'when_logged',
+               existing_type=mysql.TIMESTAMP(),
+               server_default=sa.text('current_timestamp()'),
+               existing_nullable=False)
+    op.drop_column('event_log', 'ip')
+    op.drop_column('event_log', 'machine')
+    op.drop_column('event_log', 'arcade')
+    op.drop_column('event_log', 'user')
+    # ### end Alembic commands ###

core/data/alembic/versions/2d024cf145a1_add_event_log_game_version.py

@@ -0,0 +1,46 @@
+"""add_event_log_game_version
+
+Revision ID: 2d024cf145a1
+Revises: 2bf9f38d9444
+Create Date: 2024-05-21 23:41:31.445331
+
+"""
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import mysql
+
+# revision identifiers, used by Alembic.
+revision = '2d024cf145a1'
+down_revision = '2bf9f38d9444'
+branch_labels = None
+depends_on = None
+
+
+def upgrade():
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.add_column('event_log', sa.Column('game', sa.TEXT(length=4), nullable=True))
+    op.add_column('event_log', sa.Column('version', sa.TEXT(length=24), nullable=True))
+    op.alter_column('event_log', 'ip',
+               existing_type=mysql.TINYTEXT(),
+               type_=sa.TEXT(length=39),
+               existing_nullable=True)
+    op.alter_column('event_log', 'when_logged',
+               existing_type=mysql.TIMESTAMP(),
+               server_default=sa.text('now()'),
+               existing_nullable=False)
+    # ### end Alembic commands ###
+
+
+def downgrade():
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.alter_column('event_log', 'when_logged',
+               existing_type=mysql.TIMESTAMP(),
+               server_default=sa.text('current_timestamp()'),
+               existing_nullable=False)
+    op.alter_column('event_log', 'ip',
+               existing_type=sa.TEXT(length=39),
+               type_=mysql.TINYTEXT(),
+               existing_nullable=True)
+    op.drop_column('event_log', 'version')
+    op.drop_column('event_log', 'game')
+    # ### end Alembic commands ###

core/data/alembic/versions/4a02e623e5e6_mai2_add_favs_rivals.py

@@ -0,0 +1,48 @@
+"""mai2_add_favs_rivals
+
+Revision ID: 4a02e623e5e6
+Revises: 8ad40a6e7be2
+Create Date: 2024-06-08 19:02:43.856395
+
+"""
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import mysql
+
+# revision identifiers, used by Alembic.
+revision = '4a02e623e5e6'
+down_revision = '8ad40a6e7be2'
+branch_labels = None
+depends_on = None
+
+
+def upgrade():
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.create_table('mai2_item_favorite_music',
+    sa.Column('id', sa.Integer(), nullable=False),
+    sa.Column('user', sa.Integer(), nullable=False),
+    sa.Column('musicId', sa.Integer(), nullable=False),
+    sa.ForeignKeyConstraint(['user'], ['aime_user.id'], onupdate='cascade', ondelete='cascade'),
+    sa.PrimaryKeyConstraint('id'),
+    sa.UniqueConstraint('user', 'musicId', name='mai2_item_favorite_music_uk'),
+    mysql_charset='utf8mb4'
+    )
+    op.create_table('mai2_user_rival',
+    sa.Column('id', sa.Integer(), nullable=False),
+    sa.Column('user', sa.Integer(), nullable=False),
+    sa.Column('rival', sa.Integer(), nullable=False),
+    sa.Column('show', sa.Boolean(), server_default='0', nullable=False),
+    sa.ForeignKeyConstraint(['rival'], ['aime_user.id'], onupdate='cascade', ondelete='cascade'),
+    sa.ForeignKeyConstraint(['user'], ['aime_user.id'], onupdate='cascade', ondelete='cascade'),
+    sa.PrimaryKeyConstraint('id'),
+    sa.UniqueConstraint('user', 'rival', name='mai2_user_rival_uk'),
+    mysql_charset='utf8mb4'
+    )
+    # ### end Alembic commands ###
+
+
+def downgrade():
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.drop_table('mai2_user_rival')
+    op.drop_table('mai2_item_favorite_music')
+    # ### end Alembic commands ###

core/data/alembic/versions/6a7e8277763b_gekichu_rating_tables.py

@@ -0,0 +1,56 @@
+"""GekiChu rating tables
+
+Revision ID: 6a7e8277763b
+Revises: d8950c7ce2fc
+Create Date: 2024-03-13 12:18:53.210018
+
+"""
+from alembic import op
+from sqlalchemy import Column, Integer, String
+
+
+# revision identifiers, used by Alembic.
+revision = '6a7e8277763b'
+down_revision = 'd8950c7ce2fc'
+branch_labels = None
+depends_on = None
+
+GEKICHU_RATING_TABLE_NAMES = [
+    "chuni_profile_rating",
+    "ongeki_profile_rating",
+]
+
+def upgrade():
+    for table_name in GEKICHU_RATING_TABLE_NAMES:
+        op.create_table(
+            table_name,
+            Column("id", Integer, primary_key=True, nullable=False),
+            Column("user", Integer, nullable=False),
+            Column("version", Integer, nullable=False),
+            Column("type", String(255), nullable=False),
+            Column("index", Integer, nullable=False),
+            Column("musicId", Integer),
+            Column("difficultId", Integer),
+            Column("romVersionCode", Integer),
+            Column("score", Integer),
+            mysql_charset="utf8mb4",
+        )
+        op.create_foreign_key(
+            None,
+            table_name,
+            "aime_user",
+            ["user"],
+            ["id"],
+            ondelete="cascade",
+            onupdate="cascade",
+        )
+        op.create_unique_constraint(
+            f"{table_name}_uk",
+            table_name,
+            ["user", "version", "type", "index"],
+        )
+
+
+def downgrade():
+    for table_name in GEKICHU_RATING_TABLE_NAMES:
+        op.drop_table(table_name)

core/data/alembic/versions/7dc13e364e53_cxb_add_playlog_grade.py

@@ -0,0 +1,28 @@
+"""cxb_add_playlog_grade
+
+Revision ID: 7dc13e364e53
+Revises: 2d024cf145a1
+Create Date: 2024-05-28 22:31:22.264926
+
+"""
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import mysql
+
+# revision identifiers, used by Alembic.
+revision = '7dc13e364e53'
+down_revision = '2d024cf145a1'
+branch_labels = None
+depends_on = None
+
+
+def upgrade():
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.add_column('cxb_playlog', sa.Column('grade', sa.Integer(), nullable=True))
+    # ### end Alembic commands ###
+
+
+def downgrade():
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.drop_column('cxb_playlog', 'grade')
+    # ### end Alembic commands ###

core/data/alembic/versions/81e44dd6047a_mai2_buddies_support.py

@@ -0,0 +1,68 @@
+"""mai2_buddies_support
+
+Revision ID: 81e44dd6047a
+Revises: 6a7e8277763b
+Create Date: 2024-03-12 19:10:37.063907
+
+"""
+
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import mysql
+
+# revision identifiers, used by Alembic.
+revision = "81e44dd6047a"
+down_revision = "6a7e8277763b"
+branch_labels = None
+depends_on = None
+
+
+def upgrade():
+    op.create_table(
+        "mai2_playlog_2p",
+        sa.Column("id", sa.Integer(), nullable=False),
+        sa.Column("user", sa.Integer(), nullable=False),
+        sa.Column("userId1", sa.Integer(), nullable=True),
+        sa.Column("userId2", sa.Integer(), nullable=True),
+        sa.Column("userName1", sa.String(length=25), nullable=True),
+        sa.Column("userName2", sa.String(length=25), nullable=True),
+        sa.Column("regionId", sa.Integer(), nullable=True),
+        sa.Column("placeId", sa.Integer(), nullable=True),
+        sa.Column("user2pPlaylogDetailList", sa.JSON(), nullable=True),
+        sa.ForeignKeyConstraint(
+            ["user"], ["aime_user.id"], onupdate="cascade", ondelete="cascade"
+        ),
+        sa.PrimaryKeyConstraint("id"),
+        mysql_charset="utf8mb4",
+    )
+
+    op.add_column(
+        "mai2_playlog",
+        sa.Column(
+            "extBool1", sa.Boolean(), nullable=True, server_default=sa.text("NULL")
+        ),
+    )
+
+    op.add_column(
+        "mai2_profile_detail",
+        sa.Column(
+            "renameCredit", sa.Integer(), nullable=True, server_default=sa.text("NULL")
+        ),
+    )
+    op.add_column(
+        "mai2_profile_detail",
+        sa.Column(
+            "currentPlayCount",
+            sa.Integer(),
+            nullable=True,
+            server_default=sa.text("NULL"),
+        ),
+    )
+
+
+def downgrade():
+    op.drop_table("mai2_playlog_2p")
+
+    op.drop_column("mai2_playlog", "extBool1")
+    op.drop_column("mai2_profile_detail", "renameCredit")
+    op.drop_column("mai2_profile_detail", "currentPlayCount")

core/data/alembic/versions/8ad40a6e7be2_ongeki_fix_clearstatus.py

@@ -0,0 +1,30 @@
+"""ongeki: fix clearStatus
+
+Revision ID: 8ad40a6e7be2
+Revises: 7dc13e364e53
+Create Date: 2024-05-29 19:03:30.062157
+
+"""
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import mysql
+
+# revision identifiers, used by Alembic.
+revision = '8ad40a6e7be2'
+down_revision = '7dc13e364e53'
+branch_labels = None
+depends_on = None
+
+
+def upgrade():
+    op.alter_column('ongeki_score_best', 'clearStatus',
+               existing_type=mysql.TINYINT(display_width=1),
+               type_=sa.Integer(),
+               existing_nullable=False)
+
+
+def downgrade():
+    op.alter_column('ongeki_score_best', 'clearStatus',
+               existing_type=sa.Integer(),
+               type_=mysql.TINYINT(display_width=1),
+               existing_nullable=False)

core/data/database.py

@@ -118,7 +118,13 @@ class Data:
     def schema_upgrade(self, ver: str = None):
         self.__alembic_cmd(
             "upgrade",
-            "head",
+            "head" if not ver else ver,
+        )
+
+    def schema_downgrade(self, ver: str):
+        self.__alembic_cmd(
+            "downgrade",
+            ver,
         )
 
     async def create_owner(self, email: Optional[str] = None, code: Optional[str] = "00000000000000000000") -> None:
@@ -127,7 +133,7 @@ class Data:
         )
         hash = bcrypt.hashpw(pw.encode(), bcrypt.gensalt())
 
-        user_id = await self.user.create_user("sysowner", email, hash.decode(), 255)
+        user_id = await self.user.create_user(username="sysowner", email=email, password=hash.decode(), permission=255)
         if user_id is None:
             self.logger.error(f"Failed to create owner with email {email}")
             return
@@ -229,3 +235,19 @@ class Data:
             "-m",
             message,
         )
+
+    async def create_revision_auto(self, message: str) -> None:
+        if not message:
+            self.logger.info("Message is required for create-revision")
+            return
+        
+        for _, mod in Utils.get_all_titles().items():
+            if hasattr(mod, "database"):
+                mod.database(self.config)
+        
+        self.__alembic_cmd(
+            "revision",
+            "--autogenerate",
+            "-m",
+            message,
+        )

core/data/schema/arcade.py

@@ -94,7 +94,7 @@ class ArcadeData(BaseData):
             return None
         return result.fetchone()
 
-    async def put_machine(
+    async def create_machine(
         self,
         arcade_id: int,
         serial: str = "",
@@ -102,12 +102,12 @@ class ArcadeData(BaseData):
         game: str = None,
         is_cab: bool = False,
     ) -> Optional[int]:
-        if arcade_id:
+        if not arcade_id:
             self.logger.error(f"{__name__ }: Need arcade id!")
             return None
 
         sql = machine.insert().values(
-            arcade=arcade_id, keychip=serial, board=board, game=game, is_cab=is_cab
+            arcade=arcade_id, serial=serial, board=board, game=game, is_cab=is_cab
         )
 
         result = await self.execute(sql)
@@ -148,15 +148,15 @@ class ArcadeData(BaseData):
             return None
         return result.fetchall()
 
-    async def put_arcade(
+    async def create_arcade(
         self,
-        name: str,
+        name: str = None,
         nickname: str = None,
         country: str = "JPN",
         country_id: int = 1,
         state: str = "",
         city: str = "",
-        regional_id: int = 1,
+        region_id: int = 1,
     ) -> Optional[int]:
         if nickname is None:
             nickname = name
@@ -168,7 +168,7 @@ class ArcadeData(BaseData):
             country_id=country_id,
             state=state,
             city=city,
-            regional_id=regional_id,
+            region_id=region_id,
         )
 
         result = await self.execute(sql)
@@ -206,8 +206,8 @@ class ArcadeData(BaseData):
             return None
         return result.lastrowid
 
-    async def format_serial(
-        self, platform_code: str, platform_rev: int, serial_num: int, append: int = 4152
+    def format_serial( # TODO: Actual serial stuff
+        self, platform_code: str, platform_rev: int, serial_num: int, append: int = 8888
     ) -> str:
         return f"{platform_code}{platform_rev:02d}A{serial_num:04d}{append:04d}"  # 0x41 = A, 0x52 = R
 

core/data/schema/base.py

@@ -8,7 +8,8 @@ from sqlalchemy.engine.base import Connection
 from sqlalchemy.sql import text, func, select
 from sqlalchemy.exc import SQLAlchemyError
 from sqlalchemy import MetaData, Table, Column
-from sqlalchemy.types import Integer, String, TIMESTAMP, JSON
+from sqlalchemy.types import Integer, String, TIMESTAMP, JSON, INTEGER, TEXT
+from sqlalchemy.schema import ForeignKey
 from sqlalchemy.dialects.mysql import insert
 
 from core.config import CoreConfig
@@ -22,6 +23,12 @@ event_log = Table(
     Column("system", String(255), nullable=False),
     Column("type", String(255), nullable=False),
     Column("severity", Integer, nullable=False),
+    Column("user", INTEGER, ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("arcade", INTEGER, ForeignKey("arcade.id", ondelete="cascade", onupdate="cascade")),
+    Column("machine", INTEGER, ForeignKey("machine.id", ondelete="cascade", onupdate="cascade")),
+    Column("ip", TEXT(39)),
+    Column("game", TEXT(4)),
+    Column("version", TEXT(24)),
     Column("message", String(1000), nullable=False),
     Column("details", JSON, nullable=False),
     Column("when_logged", TIMESTAMP, nullable=False, server_default=func.now()),
@@ -75,12 +82,19 @@ class BaseData:
         return randrange(10000, 9999999)
 
     async def log_event(
-        self, system: str, type: str, severity: int, message: str, details: Dict = {}
+        self, system: str, type: str, severity: int, message: str, details: Dict = {}, user: int = None, 
+        arcade: int = None, machine: int = None, ip: str = None, game: str = None, version: str = None
     ) -> Optional[int]:
         sql = event_log.insert().values(
             system=system,
             type=type,
             severity=severity,
+            user=user,
+            arcade=arcade,
+            machine=machine,
+            ip=ip,
+            game=game,
+            version=version,
             message=message,
             details=json.dumps(details),
         )
@@ -94,8 +108,8 @@ class BaseData:
 
         return result.lastrowid
 
-    async def get_event_log(self, entries: int = 100) -> Optional[List[Dict]]:
-        sql = event_log.select().limit(entries).all()
+    async def get_event_log(self, entries: int = 100) -> Optional[List[Row]]:
+        sql = event_log.select().limit(entries)
         result = await self.execute(sql)
 
         if result is None:

core/data/schema/card.py

@@ -74,6 +74,7 @@ class CardData(BaseData):
         if card["is_banned"]:
             return True
         return False
+    
     async def get_card_locked(self, access_code: str) -> Optional[bool]:
         """
         Given a 20 digit access code as a string, check if the card is locked
@@ -112,6 +113,15 @@ class CardData(BaseData):
             return None
         return result.lastrowid
 
+    async def update_card_last_login(self, access_code: str) -> None:
+        sql = aime_card.update(aime_card.c.access_code == access_code).values(
+            last_login_date=func.now()
+        )
+        
+        result = await self.execute(sql)
+        if result is None:
+            self.logger.warn(f"Failed to update last login time for {access_code}")
+
     def to_access_code(self, luid: str) -> str:
         """
         Given a felica cards internal 16 hex character luid, convert it to a 0-padded 20 digit access code as a string

core/data/schema/user.py

@@ -108,3 +108,19 @@ class UserData(BaseData):
         if result is None:
             return False
         return result.fetchall()
+
+    async def change_password(self, user_id: int, new_passwd: str) -> bool:
+        sql = aime_user.update(aime_user.c.id == user_id).values(password = new_passwd)
+
+        result = await self.execute(sql)
+        return result is not None
+
+    async def change_username(self, user_id: int, new_name: str) -> bool:
+        sql = aime_user.update(aime_user.c.id == user_id).values(username = new_name)
+
+        result = await self.execute(sql)
+        return result is not None
+
+    async def get_user_by_username(self, username: str) -> Optional[Row]:
+        result = await self.execute(aime_user.select(aime_user.c.username == username))
+        if result: return result.fetchone()

core/frontend.py

@@ -10,6 +10,9 @@ import bcrypt
 import re
 import jwt
 import yaml
+import secrets
+import string
+import random
 from base64 import b64decode
 from enum import Enum
 from datetime import datetime, timezone
@@ -41,11 +44,13 @@ class ShopOwner():
         self.permissions = perms
 
 class UserSession():
-    def __init__(self, usr_id: int = 0, ip: str = "", perms: int = 0, ongeki_ver: int = 7):
+    def __init__(self, usr_id: int = 0, ip: str = "", perms: int = 0, ongeki_ver: int = 7, chunithm_ver: int = -1, maimai_version: int = -1):
         self.user_id = usr_id
         self.current_ip = ip
         self.permissions = perms
         self.ongeki_version = ongeki_ver
+        self.chunithm_version = chunithm_ver
+        self.maimai_version = maimai_version
 
 class FrontendServlet():
     def __init__(self, cfg: CoreConfig, config_dir: str) -> None:
@@ -129,8 +134,13 @@ class FrontendServlet():
             ]),
             Mount("/sys", routes=[
                 Route("/", self.system.render_GET, methods=['GET']),
+                Route("/logs", self.system.render_logs, methods=['GET']),
                 Route("/lookup.user", self.system.lookup_user, methods=['GET']),
                 Route("/lookup.shop", self.system.lookup_shop, methods=['GET']),
+                Route("/add.user", self.system.add_user, methods=['POST']),
+                Route("/add.card", self.system.add_card, methods=['POST']),
+                Route("/add.shop", self.system.add_shop, methods=['POST']),
+                Route("/add.cab", self.system.add_cab, methods=['POST']),
             ]),
             Mount("/shop", routes=[
                 Route("/", self.arcade.render_GET, methods=['GET']),
@@ -180,10 +190,10 @@ class FE_Base():
             title=self.core_config.server.name,
             game_list=self.environment.globals["game_list"],
             sesh=vars(sesh) if sesh is not None else vars(UserSession()),
-        ))
+        ), media_type="text/html; charset=utf-8")
         
         if sesh is None:
-            resp.delete_cookie("DIANA_SESH")
+            resp.delete_cookie("ARTEMIS_SESH")
         return resp
     
     def get_routes(self) -> List[Route]:
@@ -206,7 +216,10 @@ class FE_Base():
             sesh.user_id = tk['user_id']
             sesh.current_ip = tk['current_ip']
             sesh.permissions = tk['permissions']
-            
+            sesh.chunithm_version = tk['chunithm_version']
+            sesh.maimai_version = tk['maimai_version']
+            sesh.ongeki_version = tk['ongeki_version']
+
             if sesh.user_id <= 0:
                 self.logger.error("User session failed to validate due to an invalid ID!")
                 return UserSession()
@@ -231,7 +244,7 @@ class FE_Base():
             return UserSession()
     
     def validate_session(self, request: Request) -> Optional[UserSession]:
-        sesh = request.cookies.get('DIANA_SESH', "")
+        sesh = request.cookies.get('ARTEMIS_SESH', "")
         if not sesh:
             return None
         
@@ -245,12 +258,22 @@ class FE_Base():
         if usr_sesh.permissions <= 0 or usr_sesh.permissions > 255:
             self.logger.error(f"User session failed to validate due to an invalid permission value! {usr_sesh.permissions}")
             return None
-        
+
         return usr_sesh
     
     def encode_session(self, sesh: UserSession, exp_seconds: int = 86400) -> str:
         try:
-            return jwt.encode({ "user_id": sesh.user_id, "current_ip": sesh.current_ip, "permissions": sesh.permissions, "ongeki_version": sesh.ongeki_version, "exp": int(datetime.now(tz=timezone.utc).timestamp()) + exp_seconds }, b64decode(self.core_config.frontend.secret), algorithm="HS256")
+            return jwt.encode({
+                "user_id": sesh.user_id,
+                "current_ip": sesh.current_ip,
+                "permissions": sesh.permissions,
+                "ongeki_version": sesh.ongeki_version,
+                "chunithm_version": sesh.chunithm_version,
+                "maimai_version": sesh.maimai_version,
+                "exp": int(datetime.now(tz=timezone.utc).timestamp()) + exp_seconds }, 
+                b64decode(self.core_config.frontend.secret), 
+                algorithm="HS256"
+            )
         except jwt.InvalidKeyError:
             self.logger.error("Failed to encode User session because the secret is invalid!")
             return ""
@@ -281,8 +304,8 @@ class FE_Gate(FE_Base):
             title=f"{self.core_config.server.name} | Login Gate",
             error=err,
             sesh=vars(UserSession()),
-        ))
-        resp.delete_cookie("DIANA_SESH")
+        ), media_type="text/html; charset=utf-8")
+        resp.delete_cookie("ARTEMIS_SESH")
         return resp
     
     async def render_login(self, request: Request):
@@ -298,8 +321,12 @@ class FE_Gate(FE_Base):
 
         uid = await self.data.card.get_user_id_from_card(access_code)
         if uid is None:
-            self.logger.debug(f"Failed to find user for card {access_code}")
-            return RedirectResponse("/gate/?e=1", 303)
+            user = await self.data.user.get_user_by_username(access_code) # Lookup as username
+            if not user:
+                self.logger.debug(f"Failed to find user for card/username {access_code}")
+                return RedirectResponse("/gate/?e=1", 303)
+            
+            uid = user['id']
 
         user = await self.data.user.get_user(uid)
         if user is None:
@@ -310,7 +337,7 @@ class FE_Gate(FE_Base):
             sesh = await self.data.user.check_password(uid)
 
             if sesh is not None:
-                return RedirectResponse(f"/gate/create?ac={access_code}")
+                return RedirectResponse(f"/gate/create?ac={access_code}", 303)
             
             return RedirectResponse("/gate/?e=1", 303)
 
@@ -328,16 +355,17 @@ class FE_Gate(FE_Base):
         usr_sesh = self.encode_session(sesh)
         self.logger.debug(f"Created session with JWT {usr_sesh}")
         resp = RedirectResponse("/user/", 303)
-        resp.set_cookie("DIANA_SESH", usr_sesh)
+        resp.set_cookie("ARTEMIS_SESH", usr_sesh)
 
         return resp
 
     async def render_create(self, request: Request):
         ip = Utils.get_ip_addr(request)
-        access_code: str = request.query_params.get("access_code", "")
-        username: str = request.query_params.get("username", "")
-        email: str = request.query_params.get("email", "")
-        passwd: bytes = request.query_params.get("passwd", "").encode()
+        frm = await request.form()
+        access_code: str = frm.get("access_code", "")
+        username: str = frm.get("username", "")
+        email: str = frm.get("email", "")
+        passwd: bytes = frm.get("passwd", "").encode()
 
         if not access_code or not username or not email or not passwd:
             return RedirectResponse("/gate/?e=1", 303)
@@ -365,13 +393,13 @@ class FE_Gate(FE_Base):
         
         usr_sesh = self.encode_session(sesh)
         self.logger.debug(f"Created session with JWT {usr_sesh}")
-        resp = RedirectResponse("/user", 303)
-        resp.set_cookie("DIANA_SESH", usr_sesh)
+        resp = RedirectResponse("/user/", 303)
+        resp.set_cookie("ARTEMIS_SESH", usr_sesh)
 
         return resp
 
     async def render_create_get(self, request: Request):
-        ac = request.query_params.get(b"ac", [b""])[0].decode()
+        ac = request.query_params.get("ac", "")
         if len(ac) != 20:
             return RedirectResponse("/gate/?e=2", 303)
 
@@ -392,7 +420,7 @@ class FE_Gate(FE_Base):
             title=f"{self.core_config.server.name} | Create User",
             code=ac,
             sesh={"user_id": 0, "permissions": 0},
-        ))
+        ), media_type="text/html; charset=utf-8")
 
 class FE_User(FE_Base):
     async def render_GET(self, request: Request):
@@ -439,10 +467,7 @@ class FE_User(FE_Base):
             elif ac.startswith("3"):
                 c_type = "Banapass"
             elif ac.startswith("010"):                
-                c_type = "Unknown Aime"
-                desc, _ = self.data.card.get_aime_ac_key_desc(ac)
-                if desc is not None:
-                    c_type = desc
+                c_type = "Aime" # TODO: Aime verification
             elif ac.startswith("0008"):
                 c_type = "Generated AIC"
             else:
@@ -483,11 +508,11 @@ class FE_User(FE_Base):
             success=succ,
             username=user['username'],
             arcades=arcade_data
-        ))
+        ), media_type="text/html; charset=utf-8")
     
     async def render_logout(self, request: Request):
         resp = RedirectResponse("/gate/", 303)
-        resp.delete_cookie("DIANA_SESH")
+        resp.delete_cookie("ARTEMIS_SESH")
         return resp
     
     async def edit_card(self, request: Request) -> RedirectResponse:
@@ -553,11 +578,17 @@ class FE_System(FE_Base):
         if not usr_sesh or not self.test_perm_minimum(usr_sesh.permissions, PermissionOffset.USERMOD):
             return RedirectResponse("/gate/", 303)
         
+        if request.query_params.get("e", None):
+            err = int(request.query_params.get("e"))
+        else:
+            err = 0
+        
         return Response(template.render(
             title=f"{self.core_config.server.name} | System", 
             sesh=vars(usr_sesh), 
             usrlist=[],
-        ))
+            error = err
+        ), media_type="text/html; charset=utf-8")
         
     async def lookup_user(self, request: Request):
         template = self.environment.get_template("core/templates/sys/index.jinja")
@@ -590,7 +621,7 @@ class FE_System(FE_Base):
             sesh=vars(usr_sesh), 
             usrlist=usrlist,
             shoplist=[],
-        ))
+        ), media_type="text/html; charset=utf-8")
 
     async def lookup_shop(self, request: Request):
         shoplist = []
@@ -624,7 +655,7 @@ class FE_System(FE_Base):
                     usrlist=[],
                     shoplist=shoplist,
                     error=4
-                ))
+                ), media_type="text/html; charset=utf-8")
         
         if sn_search:
             sn_search = sn_search.upper().replace("-", "").strip()
@@ -653,7 +684,7 @@ class FE_System(FE_Base):
                     usrlist=[],
                     shoplist=shoplist,
                     error=10
-                ))
+                ), media_type="text/html; charset=utf-8")
             
         
         return Response(template.render(
@@ -661,8 +692,144 @@ class FE_System(FE_Base):
             sesh=vars(usr_sesh), 
             usrlist=[],
             shoplist=shoplist,
-        ))
+        ), media_type="text/html; charset=utf-8")
 
+    async def add_user(self, request: Request):
+        template = self.environment.get_template("core/templates/sys/index.jinja")
+        
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh or not self.test_perm(usr_sesh.permissions, PermissionOffset.ACMOD):
+            return RedirectResponse("/gate/", 303)
+        
+        frm = await request.form()
+        username = frm.get("userName", None)
+        email = frm.get("userEmail", None)
+        perm = frm.get("usrPerm", "1")
+        passwd = "".join(
+            secrets.choice(string.ascii_letters + string.digits) for i in range(20)
+        )
+        hash = bcrypt.hashpw(passwd.encode(), bcrypt.gensalt())
+
+        if not email:
+            return RedirectResponse("/sys/?e=4", 303)
+
+        uid = await self.data.user.create_user(username=username if username else None, email=email, password=hash.decode(), permission=int(perm))
+        return Response(template.render(
+            title=f"{self.core_config.server.name} | System", 
+            sesh=vars(usr_sesh), 
+            usradd={"id": uid, "username": username, "password": passwd},
+        ), media_type="text/html; charset=utf-8")
+
+    async def add_card(self, request: Request):
+        template = self.environment.get_template("core/templates/sys/index.jinja")
+        
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh or not self.test_perm(usr_sesh.permissions, PermissionOffset.ACMOD):
+            return RedirectResponse("/gate/", 303)
+        
+        frm = await request.form()
+        userid = frm.get("cardUsr", None)
+        access_code = frm.get("cardAc", None)
+        idm = frm.get("cardIdm", None)
+
+        if userid is None or access_code is None or not userid.isdigit() or not len(access_code) == 20 or not access_code.isdigit:
+            return RedirectResponse("/sys/?e=4", 303)
+        
+        cardid = await self.data.card.create_card(int(userid), access_code)
+        if not cardid:
+            return RedirectResponse("/sys/?e=99", 303)
+
+        if idm is not None:
+            # TODO: save IDM
+            pass
+        
+        return Response(template.render(
+            title=f"{self.core_config.server.name} | System", 
+            sesh=vars(usr_sesh), 
+            cardadd={"id": cardid, "user": userid, "access_code": access_code},
+        ), media_type="text/html; charset=utf-8")
+
+    async def add_shop(self, request: Request):
+        template = self.environment.get_template("core/templates/sys/index.jinja")
+        
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh or not self.test_perm(usr_sesh.permissions, PermissionOffset.ACMOD):
+            return RedirectResponse("/gate/", 303)
+        
+        frm = await request.form()
+        name = frm.get("shopName", None)
+        country = frm.get("shopCountry", "JPN")
+        ip = frm.get("shopIp", None)
+
+        acid = await self.data.arcade.create_arcade(name if name else None, name if name else None, country)
+        if not acid:
+            return RedirectResponse("/sys/?e=99", 303)
+        
+        if ip:
+            # TODO: set IP
+            pass
+        
+        return Response(template.render(
+            title=f"{self.core_config.server.name} | System", 
+            sesh=vars(usr_sesh), 
+            shopadd={"id": acid},
+        ), media_type="text/html; charset=utf-8")
+
+    async def add_cab(self, request: Request):
+        template = self.environment.get_template("core/templates/sys/index.jinja")
+        
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh or not self.test_perm(usr_sesh.permissions, PermissionOffset.ACMOD):
+            return RedirectResponse("/gate/", 303)
+
+        frm = await request.form()
+        shopid = frm.get("cabShop", None)
+        serial = frm.get("cabSerial", None)
+        game_code = frm.get("cabGame", None)
+
+        if not shopid or not shopid.isdigit():
+            return RedirectResponse("/sys/?e=4", 303)
+        
+        if not serial:
+            serial = self.data.arcade.format_serial("A69E", 1, random.randint(1, 9999))
+        
+        cab_id = await self.data.arcade.create_machine(int(shopid), serial, None, game_code if game_code else None)
+        
+        return Response(template.render(
+            title=f"{self.core_config.server.name} | System", 
+            sesh=vars(usr_sesh), 
+            cabadd={"id": cab_id, "serial": serial},
+        ), media_type="text/html; charset=utf-8")
+
+    async def render_logs(self, request: Request):
+        template = self.environment.get_template("core/templates/sys/logs.jinja")
+        events = []
+        
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh or not self.test_perm(usr_sesh.permissions, PermissionOffset.SYSADMIN):
+            return RedirectResponse("/sys/?e=11", 303)
+        
+        logs = await self.data.base.get_event_log()
+        if not logs:
+            logs = []
+        
+        for log in logs:
+            evt = log._asdict()
+            if not evt['user']: evt["user"] = "NONE"
+            if not evt['arcade']: evt["arcade"] = "NONE"
+            if not evt['machine']: evt["machine"] = "NONE"
+            if not evt['ip']: evt["ip"] = "NONE"
+            if not evt['game']: evt["game"] = "NONE"
+            if not evt['version']: evt["version"] = "NONE"
+            evt['when_logged'] = evt['when_logged'].strftime("%x %X")
+            events.append(evt)
+        
+        return Response(template.render(
+            title=f"{self.core_config.server.name} | Event Logs", 
+            sesh=vars(usr_sesh), 
+            events=events
+        ), media_type="text/html; charset=utf-8")
+        
 class FE_Arcade(FE_Base):
     async def render_GET(self, request: Request):
         template = self.environment.get_template("core/templates/arcade/index.jinja")
@@ -677,14 +844,14 @@ class FE_Arcade(FE_Base):
             return Response(template.render(
                 title=f"{self.core_config.server.name} | Arcade", 
                 sesh=vars(usr_sesh),
-            ))
+            ), media_type="text/html; charset=utf-8")
         
         sinfo = await self.data.arcade.get_arcade(shop_id)
         if not sinfo:
             return Response(template.render(
                 title=f"{self.core_config.server.name} | Arcade", 
                 sesh=vars(usr_sesh),
-            ))
+            ), media_type="text/html; charset=utf-8")
         
         cabs = await self.data.arcade.get_arcade_machines(shop_id)
         cablst = []
@@ -705,7 +872,7 @@ class FE_Arcade(FE_Base):
                 "cabs": cablst
             }
             
-        ))
+        ), media_type="text/html; charset=utf-8")
 
 class FE_Machine(FE_Base):
     async def render_GET(self, request: Request):
@@ -721,15 +888,15 @@ class FE_Machine(FE_Base):
             return Response(template.render(
                 title=f"{self.core_config.server.name} | Machine", 
                 sesh=vars(usr_sesh),
-            ))
+            ), media_type="text/html; charset=utf-8")
         
         return Response(template.render(
             title=f"{self.core_config.server.name} | Machine", 
             sesh=vars(usr_sesh),
             arcade={}
-        ))
+        ), media_type="text/html; charset=utf-8")
 
-cfg_dir = environ.get("DIANA_CFG_DIR", "config")
+cfg_dir = environ.get("ARTEMIS_CFG_DIR", "config")
 cfg: CoreConfig = CoreConfig()
 if path.exists(f"{cfg_dir}/core.yaml"):
     cfg.update(yaml.safe_load(open(f"{cfg_dir}/core.yaml")))

core/mucha.py

@@ -2,6 +2,7 @@ from typing import Dict, Any, Optional
 import logging, coloredlogs
 from logging.handlers import TimedRotatingFileHandler
 from starlette.requests import Request
+from starlette.responses import PlainTextResponse
 from datetime import datetime
 from Crypto.Cipher import Blowfish
 import pytz
@@ -9,9 +10,11 @@ import pytz
 from .config import CoreConfig
 from .utils import Utils
 from .title import TitleServlet
+from .data import Data
+from .const import *
 
 class MuchaServlet:
-    mucha_registry: Dict[str, str] = {}
+    mucha_registry: Dict[str, Dict[str, str]] = {}
     def __init__(self, cfg: CoreConfig, cfg_dir: str) -> None:
         self.config = cfg
         self.config_dir = cfg_dir
@@ -35,91 +38,148 @@ class MuchaServlet:
 
         self.logger.setLevel(cfg.mucha.loglevel)
         coloredlogs.install(level=cfg.mucha.loglevel, logger=self.logger, fmt=log_fmt_str)
+        
+        self.data = Data(cfg)
 
         for _, mod in TitleServlet.title_registry.items():
-            if hasattr(mod, "get_mucha_info"):
-                enabled, game_cds, netid_prefixes = mod.get_mucha_info(
-                    self.config, self.config_dir
-                )
-                if enabled:
-                    for x in range(len(game_cds)):
-                        self.mucha_registry[game_cds[x]] = netid_prefixes[x]
+            enabled, game_cds, netids = mod.get_mucha_info(self.config, self.config_dir)
+            if enabled:
+                for x in range(len(game_cds)):
+                    self.mucha_registry[game_cds[x]] = { "netid_prefix": netids[x] }
 
         self.logger.info(f"Serving {len(self.mucha_registry)} games")
 
-    def handle_boardauth(self, request: Request, _: Dict) -> bytes:
-        req_dict = self.mucha_preprocess(request.content.getvalue())
+    async def handle_boardauth(self, request: Request) -> bytes:
+        bod = await request.body()
+        req_dict = self.mucha_preprocess(bod)
         client_ip = Utils.get_ip_addr(request)
 
         if req_dict is None:
             self.logger.error(
-                f"Error processing mucha request {request.content.getvalue()}"
+                f"Error processing mucha request {bod}"
             )
-            return b"RESULTS=000"
+            return PlainTextResponse("RESULTS=000")
 
         req = MuchaAuthRequest(req_dict)
-        self.logger.info(f"Boardauth request from {client_ip} for {req.gameVer}")        
         self.logger.debug(f"Mucha request {vars(req)}")
+        
+        if not req.gameCd or not req.gameVer or not req.sendDate or not req.countryCd or not req.serialNum:
+            self.logger.warn(f"Missing required fields - {vars(req)}")
+            return PlainTextResponse("RESULTS=000")
 
-        if req.gameCd not in self.mucha_registry:
-            self.logger.warning(f"Unknown gameCd {req.gameCd}")
-            return b"RESULTS=000"
+        minfo = self.mucha_registry.get(req.gameCd, {})
+
+        if not minfo:
+            self.logger.warning(f"Unknown gameCd {req.gameCd} from {client_ip}")
+            return PlainTextResponse("RESULTS=000")
 
-        # TODO: Decrypt S/N
         b_key = b""
         for x in range(8):
             b_key += req.sendDate[(x - 1) & 7].encode()
+        
+        b_iv = b_key # what the fuck namco
 
-        cipher = Blowfish.new(b_key, Blowfish.MODE_ECB)
-        sn_decrypt = cipher.decrypt(bytes.fromhex(req.serialNum))
-        self.logger.debug(f"Decrypt SN to {sn_decrypt.hex()}")
+        cipher = Blowfish.new(b_key, Blowfish.MODE_CBC, b_iv)
+        try:
+            sn_decrypt = cipher.decrypt(bytes.fromhex(req.serialNum))[:12].decode()
+        except Exception as e:
+            self.logger.error(f"Decrypt SN {req.serialNum} failed! - {e}")
+            return PlainTextResponse("RESULTS=000")
+
+        self.logger.info(f"Boardauth request from {sn_decrypt} ({client_ip}) for {req.gameVer}")
 
         resp = MuchaAuthResponse(
-            f"{self.config.server.hostname}{':' + str(self.config.server.port) if self.config.server.is_develop else ''}"
+            f"{self.config.server.hostname}{':' + str(self.config.server.port) if not self.config.server.is_using_proxy else ''}"
         )
 
+        netid = minfo.get('netid_prefix', "ABxN") + sn_decrypt[5:]
+
+        cab = await self.data.arcade.get_machine(netid)
+        if cab:
+            arcade = await self.data.arcade.get_arcade(cab['id'])
+            if not arcade:
+                self.logger.error(f"Failed to get arcade with id {cab['id']}")
+                return PlainTextResponse("RESULTS=000")
+
+            resp.AREA_0 = arcade["region_id"] or AllnetJapanRegionId.AICHI.name
+            resp.AREA_0_EN = arcade["region_id"] or AllnetJapanRegionId.AICHI.name
+            resp.AREA_FULL_0 = arcade["region_id"] or AllnetJapanRegionId.AICHI.name
+            resp.AREA_FULL_0_EN = arcade["region_id"] or AllnetJapanRegionId.AICHI.name
+            
+            resp.AREA_1 = arcade["country"] or cab['country'] or AllnetCountryCode.JAPAN.value
+            resp.AREA_1_EN = arcade["country"] or cab['country'] or AllnetCountryCode.JAPAN.value
+            resp.AREA_FULL_1 = arcade["country"] or cab['country'] or AllnetCountryCode.JAPAN.value
+            resp.AREA_FULL_1_EN = arcade["country"] or cab['country'] or AllnetCountryCode.JAPAN.value
+
+            resp.AREA_2 = arcade["city"] if arcade["city"] else ""
+            resp.AREA_2_EN = arcade["city"] if arcade["city"] else ""
+            resp.AREA_FULL_2 = arcade["city"] if arcade["city"] else ""
+            resp.AREA_FULL_2_EN = arcade["city"] if arcade["city"] else ""
+
+            resp.AREA_3 = ""
+            resp.AREA_3_EN = ""
+            resp.AREA_FULL_3 = ""
+            resp.AREA_FULL_3_EN = ""
+
+            resp.PREFECTURE_ID = arcade['region_id']
+            resp.COUNTRY_CD = arcade['country'] or cab['country'] or AllnetCountryCode.JAPAN.value
+            resp.PLACE_ID = req.placeId if req.placeId else f"{arcade['country'] or cab['country'] or AllnetCountryCode.JAPAN.value}{arcade['id']:04X}"
+            resp.SHOP_NAME = arcade['name']
+            resp.SHOP_NAME_EN = arcade['name']
+            resp.SHOP_NICKNAME = arcade['nickname']
+            resp.SHOP_NICKNAME_EN = arcade['nickname']
+
+        elif self.config.server.allow_unregistered_serials:
+            self.logger.info(f"Allow unknown serial {netid} ({sn_decrypt}) to auth")
+        
+        else:
+            self.logger.warn(f'Auth failed for NetID {netid}')
+            return PlainTextResponse("RESULTS=000")
+
         self.logger.debug(f"Mucha response {vars(resp)}")
 
-        return self.mucha_postprocess(vars(resp))
+        return PlainTextResponse(self.mucha_postprocess(vars(resp)))
 
-    def handle_updatecheck(self, request: Request, _: Dict) -> bytes:
-        req_dict = self.mucha_preprocess(request.content.getvalue())
+    async def handle_updatecheck(self, request: Request) -> bytes:
+        bod = await request.body()
+        req_dict = self.mucha_preprocess(bod)
         client_ip = Utils.get_ip_addr(request)
 
         if req_dict is None:
             self.logger.error(
-                f"Error processing mucha request {request.content.getvalue()}"
+                f"Error processing mucha request {bod}"
             )
-            return b"RESULTS=000"
+            return PlainTextResponse("RESULTS=000")
 
         req = MuchaUpdateRequest(req_dict)
-        self.logger.info(f"Updatecheck request from {client_ip} for {req.gameVer}")        
+        self.logger.info(f"Updatecheck request from {req.serialNum} ({client_ip}) for {req.gameVer}")        
         self.logger.debug(f"Mucha request {vars(req)}")
 
         if req.gameCd not in self.mucha_registry:
             self.logger.warning(f"Unknown gameCd {req.gameCd}")
-            return b"RESULTS=000"
+            return PlainTextResponse("RESULTS=000")
 
-        resp = MuchaUpdateResponse(req.gameVer, f"{self.config.server.hostname}{':' + str(self.config.server.port) if self.config.server.is_develop else ''}")
+        resp = MuchaUpdateResponse(req.gameVer, f"{self.config.server.hostname}{':' + str(self.config.server.port) if not self.config.server.is_using_proxy else ''}")
 
         self.logger.debug(f"Mucha response {vars(resp)}")
 
-        return self.mucha_postprocess(vars(resp))
+        return PlainTextResponse(self.mucha_postprocess(vars(resp)))
 
-    def handle_dlstate(self, request: Request, _: Dict) -> bytes:
-        req_dict = self.mucha_preprocess(request.content.getvalue())
+    async def handle_dlstate(self, request: Request) -> bytes:
+        bod = await request.body()
+        req_dict = self.mucha_preprocess(bod)
         client_ip = Utils.get_ip_addr(request)
 
         if req_dict is None:
             self.logger.error(
-                f"Error processing mucha request {request.content.getvalue()}"
+                f"Error processing mucha request {bod}"
             )
-            return b""
+            return PlainTextResponse("RESULTS=000")
         
         req = MuchaDownloadStateRequest(req_dict)
-        self.logger.info(f"DownloadState request from {client_ip} for {req.gameCd} -> {req.updateVer}")        
+        self.logger.info(f"DownloadState request from {req.serialNum} ({client_ip}) for {req.gameCd} -> {req.updateVer}")        
         self.logger.debug(f"request {vars(req)}")
-        return b"RESULTS=001"
+        return PlainTextResponse("RESULTS=001")
 
     def mucha_preprocess(self, data: bytes) -> Optional[Dict]:
         try:
@@ -169,7 +229,7 @@ class MuchaAuthResponse:
         self.RESULTS = "001"
         self.AUTH_INTERVAL = "86400"
         self.SERVER_TIME = datetime.strftime(datetime.now(), "%Y%m%d%H%M")
-        self.UTC_SERVER_TIME = datetime.strftime(datetime.now(pytz.UTC), "%Y%m%d%H%M")
+        self.SERVER_TIME_UTC = datetime.strftime(datetime.now(pytz.UTC), "%Y%m%d%H%M")
 
         self.CHARGE_URL = f"https://{mucha_url}/charge/"
         self.FILE_URL = f"https://{mucha_url}/file/"

core/templates/arcade/index.jinja

@@ -10,7 +10,7 @@ Cab added successfully
 {% endif %}
 <ul style="font-size: 20px;">
 {% for c in arcade.cabs %}
-<li><a href="/cab/{{ c.id }}">{{ c.serial }} ({{ c.game }})</a>&nbsp;<button class="btn btn-secondary" onclick="prep_edit_form()">Edit</button>&nbsp;<button class="btn-danger btn">Delete</button></li>
+<li><a href="/cab/{{ c.id }}">{{ c.serial }}</a> ({{ c.game if c.game else "Any" }})&nbsp;<button class="btn btn-secondary" onclick="prep_edit_form()">Edit</button>&nbsp;<button class="btn-danger btn">Delete</button></li>
 {% endfor %}
 </ul>
 {% else %}

core/templates/gate/gate.jinja

@@ -15,18 +15,18 @@
         -moz-appearance: textfield;
     }
 </style>
-<form id="login" style="max-width: 240px; min-width: 10%;" action="/gate/gate.login" method="post">
+<form id="login" style="max-width: 240px; min-width: 15%;" action="/gate/gate.login" method="post">
     <div class="form-group row">
-        <label for="access_code">Card Access Code</label><br>
-        <input form="login" class="form-control" name="access_code" id="access_code" type="number" placeholder="00000000000000000000" maxlength="20" required>
+        <label for="access_code">Access Code or Username</label><br>
+        <input form="login" class="form-control" name="access_code" id="access_code" placeholder="00000000000000000000" maxlength="20" required aria-describedby="access_code_help">
+        <div id="access_code_help" class="form-text">20 Digit access code from a card registered to your account, or your account username. (NOT your username from a game!)</div>
     </div>
     <div class="form-group row">
         <label for="passwd">Password</label><br>
-        <input id="passwd" class="form-control" name="passwd" type="password" placeholder="password">
+        <input id="passwd" class="form-control" name="passwd" type="password" placeholder="password" aria-describedby="passwd_help">
+        <div id="passwd_help" class="form-text">Leave blank if registering for the webui. Your card must have been used on a game connected to this server to register.</div>
     </div>
     <p></p>
     <input id="submit" class="btn btn-primary" style="display: block; margin: 0 auto;" form="login" type="submit" value="Login">
 </form>
-<h6>*To register for the webui, type in the access code of your card, as shown in a game, and leave the password field blank.</h6>
-<h6>*If you have not registered a card with this server, you cannot create a webui account.</h6>
 {% endblock content %}

core/templates/sys/index.jinja

@@ -4,6 +4,7 @@
 {% if error is defined %}
 {% include "core/templates/widgets/err_banner.jinja" %}
 {% endif %}
+<h2>Search</h2>
 <div class="row" id="rowForm">
     {% if "{:08b}".format(sesh.permissions)[6] == "1" %}
     <div class="col-sm-6" style="max-width: 25%;">
@@ -21,7 +22,12 @@
             OR
             <div class="form-group">
                 <label for="usrEmail">Email address</label>
-                <input type="email" class="form-control" id="usrEmail" name="usrEmail" aria-describedby="emailHelp">
+                <input type="email" class="form-control" id="usrEmail" name="usrEmail">
+            </div>
+            OR
+            <div class="form-group">
+                <label for="usrAc">Access Code</label>
+                <input type="text" class="form-control" id="usrAc" name="usrAc" maxlength="20" placeholder="00000000000000000000">
             </div>
             <br />
             <button type="submit" class="btn btn-primary">Search</button>
@@ -45,6 +51,9 @@
             <button type="submit" class="btn btn-primary">Search</button>
         </form>
     </div>
+    <div class="col-sm-6" style="max-width: 25%;">
+        <a href="/sys/logs"><button class="btn btn-primary">Event Logs</button></a>
+    </div>
     {% endif %}
 </div>
 <div class="row" id="rowResult" style="margin: 10px;">
@@ -63,7 +72,121 @@
     </div>
     {% endif %}
 </div>
+<h2>Add</h2>
 <div class="row" id="rowAdd">
-
+    {% if "{:08b}".format(sesh.permissions)[6] == "1" %}
+    <div class="col-sm-6" style="max-width: 25%;">
+        <form id="usrAdd" name="usrAdd" action="/sys/add.user" class="form-inline" method="POST">
+            <h3>Add User</h3>
+            <div class="form-group">
+                <label for="usrName">Username</label>
+                <input type="text" class="form-control" id="usrName" name="usrName">
+            </div>
+            <br>
+            <div class="form-group">
+                <label for="usrEmail">Email address</label>
+                <input type="email" class="form-control" id="usrEmail" name="usrEmail" required>
+            </div>
+            <br>
+            <div class="form-group">
+                <label for="usrPerm">Permission Level</label>
+                <input type="number" class="form-control" id="usrPerm" name="usrPerm" value="1">
+            </div>
+            <br />
+            <button type="submit" class="btn btn-primary">Add</button>
+        </form>
+    </div>
+    <div class="col-sm-6" style="max-width: 25%;">
+        <form id="cardAdd" name="cardAdd" action="/sys/add.card" class="form-inline" method="POST">
+            <h3>Add Card</h3>
+            <div class="form-group">
+                <label for="cardUsr">User ID</label>
+                <input type="number" class="form-control" id="cardUsr" name="cardUsr" required>
+            </div>
+            <br>
+            <div class="form-group">
+                <label for="cardAc">Access Code</label>
+                <input type="text" class="form-control" id="cardAc" name="cardAc" maxlength="20" placeholder="00000000000000000000" required>
+            </div>
+            <br>
+            <div class="form-group">
+                <label for="cardIdm">IDm/Chip ID</label>
+                <input type="text" class="form-control" id="cardIdm" name="cardIdm" disabled>
+            </div>
+            <br />
+            <button type="submit" class="btn btn-primary">Add</button>
+        </form>
+    </div>
+    {% endif %}
+    {% if "{:08b}".format(sesh.permissions)[5] == "1" %}
+    <div class="col-sm-6" style="max-width: 25%;">
+        <form id="shopAdd" name="shopAdd" action="/sys/add.shop" class="form-inline" method="POST">
+            <h3>Add Shop</h3>
+            <div class="form-group">
+                <label for="shopName">Name</label>
+                <input type="text" class="form-control" id="shopName" name="shopName">
+            </div>
+            <br>
+            <div class="form-group">
+                <label for="shopCountry">Country Code</label>
+                <input type="text" class="form-control" id="shopCountry" name="shopCountry" maxlength="3" placeholder="JPN">
+            </div>
+            <br />
+            <div class="form-group">
+                <label for="shopIp">VPN IP</label>
+                <input type="text" class="form-control" id="shopIp" name="shopIp">
+            </div>
+            <br />
+            <button type="submit" class="btn btn-primary">Add</button>
+        </form>
+    </div>
+    <div class="col-sm-6" style="max-width: 25%;">
+        <form id="cabAdd" name="cabAdd" action="/sys/add.cab" class="form-inline" method="POST">
+            <h3>Add Machine</h3>
+            <div class="form-group">
+                <label for="cabShop">Shop ID</label>
+                <input type="number" class="form-control" id="cabShop" name="cabShop" required>
+            </div>
+            <br>
+            <div class="form-group">
+                <label for="cabSerial">Serial</label>
+                <input type="text" class="form-control" id="cabSerial" name="cabSerial">
+            </div>
+            <br />
+            <div class="form-group">
+                <label for="cabGame">Game Code</label>
+                <input type="text" class="form-control" id="cabGame" name="cabGame" maxlength="4" placeholder="SXXX">
+            </div>
+            <br />
+            <button type="submit" class="btn btn-primary">Add</button>
+        </form>
+    </div>
+    {% endif %}
+</div>
+<div class="row" id="rowAddResult" style="margin: 10px;">
+    {% if "{:08b}".format(sesh.permissions)[6] == "1" %}
+    <div id="userAddResult" class="col-sm-6" style="max-width: 25%;">
+        {% if usradd is defined %}
+            <pre>Added user {{ usradd.username if usradd.username is not none else "with no name"}} with id {{usradd.id}} and password {{ usradd.password }}</pre>
+        {% endif %}
+    </div>
+    <div id="cardAddResult" class="col-sm-6" style="max-width: 25%;">
+        {% if cardadd is defined %}
+            <pre>Added {{ cardadd.access_code }} with id {{cardadd.id}} to user {{ cardadd.user }}</pre>
+        {% endif %}
+    </div>
+    {% endif %}
+    {% if "{:08b}".format(sesh.permissions)[5] == "1" %}
+    <div id="shopAddResult" class="col-sm-6" style="max-width: 25%;">
+        {% if shopadd is defined %}
+            <pre>Added Shop {{ shopadd.id }}</pre></a>
+        {% endif %}
+    </div>
+    <div id="cabAddResult" class="col-sm-6" style="max-width: 25%;">
+        {% if cabadd is defined %}
+            <pre>Added Machine {{ cabadd.id }} with serial {{ cabadd.serial }}</pre></a>
+        {% endif %}
+    </div>
+    {% endif %}
 </div>
 {% endblock content %}

core/templates/sys/logs.jinja

@@ -0,0 +1,182 @@
+{% extends "core/templates/index.jinja" %}
+{% block content %}
+<h1>Event Logs</h1>
+<table class="table table-dark table-striped-columns" id="tbl_events">
+    <caption>Viewing last 100 logs</caption>
+    <thead>
+        <tr>
+            <th>Severity</th>
+            <th>System</th>
+            <th>Name</th>
+            <th>User</th>
+            <th>Arcade</th>
+            <th>Machine</th>
+            <th>Game</th>
+            <th>Version</th>
+            <th>Message</th>
+            <th>Params</th>
+        </tr>
+    </thead>
+    {% if events is not defined or events|length == 0 %}
+    <tr>
+        <td colspan="10" style="text-align:center"><i>No Events</i></td>
+    </tr>
+    {% endif %}
+</table>
+<div id="div_tbl_ctrl">
+<select id="sel_per_page" onchange="update_tbl()">
+<option value="10" selected>10</option>
+<option value="25">25</option>
+<option value="50">50</option>
+<option value="100">100</option>
+</select>
+&nbsp;
+<button class="btn btn-primary" id="btn_prev" disabled onclick="chg_page(-1)"><<</button>
+<button class="btn btn-primary" id="btn_next" onclick="chg_page(1)">>></button>
+</div>
+<script type="text/javascript">
+{% if events is defined %}
+const TBL_DATA = {{events}};
+{% else %}
+const TBL_DATA = [];
+{% endif %}
+
+var per_page = 0;
+var page = 0;
+
+function update_tbl() {
+    if (TBL_DATA.length == 0) { return; }
+    var tbl = document.getElementById("tbl_events");
+    
+    for (var i = 0; i < per_page; i++) {
+        try{
+            tbl.deleteRow(1);
+        } catch {
+            break;
+        }
+    }
+
+    per_page = document.getElementById("sel_per_page").value;
+
+    if (per_page >= TBL_DATA.length) {
+        page = 0;
+        document.getElementById("btn_next").disabled = true;
+        document.getElementById("btn_prev").disabled = true;
+    }
+    
+    for (var i = 0; i < per_page; i++) {
+        let off = (page * per_page) + i;
+        if (off >= TBL_DATA.length ) {
+            break;
+        }
+        
+        var data = TBL_DATA[off];
+        var row = tbl.insertRow(i + 1);
+        var cell_severity = row.insertCell(0);
+        switch (data.severity) {
+            case 10: 
+                cell_severity.innerHTML = "DEBUG";
+                row.classList.add("table-success");
+                break;
+            case 20: 
+                cell_severity.innerHTML = "INFO";
+                row.classList.add("table-info");
+                break;
+            case 30: 
+                cell_severity.innerHTML = "WARN";
+                row.classList.add("table-warning");
+                break;
+            case 40:
+                cell_severity.innerHTML = "ERROR";
+                row.classList.add("table-danger");
+                break;
+            case 50:
+                cell_severity.innerHTML = "CRITICAL";
+                row.classList.add("table-danger");
+                break;
+            default: 
+                cell_severity.innerHTML = "---";
+                row.classList.add("table-primary");
+                break;
+        }
+        
+        var cell_mod = row.insertCell(1);
+        cell_mod.innerHTML = data.system;
+        
+        var cell_name = row.insertCell(2);
+        cell_name.innerHTML = data.type;
+        
+        var cell_usr = row.insertCell(3);
+        if (data.user == 'NONE') {
+            cell_usr.innerHTML = "---";
+        } else { 
+            cell_usr.innerHTML = "<a href=\"/user/" + data.user + "\">" + data.user + "</a>";
+        }
+
+        var cell_arcade = row.insertCell(4);
+        if (data.arcade == 'NONE') {
+            cell_arcade.innerHTML = "---";
+        } else { 
+            cell_arcade.innerHTML = "<a href=\"/shop/" + data.arcade + "\">" + data.arcade + "</a>";
+        }
+
+        var cell_machine = row.insertCell(5);
+        if (data.arcade == 'NONE') {
+            cell_machine.innerHTML = "---";
+        } else { 
+            cell_machine.innerHTML = "<a href=\"/cab/" + data.machine + "\">" + data.machine + "</a>";
+        }
+
+        var cell_game = row.insertCell(6);
+        if (data.game == 'NONE') {
+            cell_game.innerHTML = "---";
+        } else { 
+            cell_game.innerHTML = data.game;
+        }
+
+        var cell_version = row.insertCell(7);
+        if (data.version == 'NONE') {
+            cell_version.innerHTML = "---";
+        } else { 
+            cell_version.innerHTML = data.version;
+        }
+
+        var cell_msg = row.insertCell(8);
+        if (data.message == '') {
+            cell_msg.innerHTML = "---";
+        } else { 
+            cell_msg.innerHTML = data.message;
+        }
+
+        var cell_deets = row.insertCell(9);
+        if (data.details == '{}') {
+            cell_deets.innerHTML = "---";
+        } else { 
+            cell_deets.innerHTML = data.details;
+        }
+
+    }
+}
+
+function chg_page(num) {
+    var max_page = TBL_DATA.length / per_page;
+    page = page + num;
+
+    if (page > max_page) {
+        page = max_page;
+        document.getElementById("btn_next").disabled = true;
+        document.getElementById("btn_prev").disabled = false;
+        return;
+    } else if (page < 0) {
+        page = 0;
+        document.getElementById("btn_next").disabled = false;
+        document.getElementById("btn_prev").disabled = true;
+        return;
+    }
+
+    update_tbl();
+}
+
+update_tbl();
+</script>
+{% endblock content %}

core/templates/widgets/err_banner.jinja

@@ -19,9 +19,10 @@ New password not acceptable
 New Nickname too long
 {% elif error == 9 %}
 You must be logged in to preform this action
-New Nickname too long
 {% elif error == 10 %}
 Invalid serial number
+{% elif error == 11 %}
+Access Denied
 {% else %}
 An unknown error occoured
 {% endif %}

core/title.py

@@ -80,7 +80,8 @@ class BaseServlet:
             cfg_dir (str): Config directory
 
         Returns:
-            Tuple[bool, str]: Tuple where offset 0 is true if the game is enabled, false otherwise, and offset 1 is the game CD
+            Tuple[bool, List[str], List[str]]: Tuple where offset 0 is true if the game is enabled, false otherwise, and offset 1 is the game CDs handled
+            by this servlette, and offset 2 is mucha netID prefixes that should be used for each game CD. 
         """
         return (False, [], [])
 

core/utils.py

@@ -34,7 +34,8 @@ class Utils:
 
     @classmethod
     def get_ip_addr(cls, req: Request) -> str:
-        return req.headers.get("x-forwarded-for", req.client.host)
+        ip = req.headers.get("x-forwarded-for", req.client.host)
+        return ip.split(", ")[0]
     
     @classmethod
     def get_title_port(cls, cfg: CoreConfig):

dbutils.py

@@ -22,7 +22,7 @@ if __name__ == "__main__":
     parser.add_argument("--email", "-e", type=str, help="Email for the new user")
     parser.add_argument("--access_code", "-a", type=str, help="Access code for new/transfer user", default="00000000000000000000")
     parser.add_argument("--message", "-m", type=str, help="Revision message")
-    parser.add_argument("action", type=str, help="create, upgrade, create-owner, migrate, create-revision")
+    parser.add_argument("action", type=str, help="create, upgrade, downgrade, create-owner, migrate, create-revision, create-autorevision")
     args = parser.parse_args()
 
     cfg = CoreConfig()
@@ -48,10 +48,15 @@ if __name__ == "__main__":
     elif args.action == "upgrade":
         data.schema_upgrade(args.version)
 
+    elif args.action == "downgrade":
+        if not args.version:
+            logging.getLogger("database").error(f"Version argument required for downgrade")
+            exit(1)
+        data.schema_downgrade(args.version)
+
     elif args.action == "create-owner":
         loop = asyncio.get_event_loop()
         loop.run_until_complete(data.create_owner(args.email, args.access_code))
-        data.schema_upgrade(args.version)
 
     elif args.action == "migrate":
         loop = asyncio.get_event_loop()
@@ -61,5 +66,9 @@ if __name__ == "__main__":
         loop = asyncio.get_event_loop()
         loop.run_until_complete(data.create_revision(args.message))
 
+    elif args.action == "create-autorevision":
+        loop = asyncio.get_event_loop()
+        loop.run_until_complete(data.create_revision_auto(args.message))
+
     else:
         logging.getLogger("database").info(f"Unknown action {args.action}")

docs/INSTALL_LINUX.md

@@ -0,0 +1,107 @@
+# Installing ARTEMiS on Linux
+This guide assumes a fresh install of Debian 12 or Rasperry Pi OS. If you're using a different distrubution, your package manager commands and package names may be different then what's listed below. Please check with your repository's package manager for package names.
+
+## Install prerequisits
+### Python
+Some installs may come with python already installed. You can verify this by trying the following commands:
+- `python --version`
+- `python3 --version`
+- `python3.<minor version> --version` where `<minor version>` is a python 3 release (eg 11, 10)
+
+If your python version is at least 3.7, you can move to the next step
+
+### Libraries and other software
+ARTEMiS depends on mysql and memcached. As stated above, package names may vary by distrubution, but this is generally what you should expect to install.
+#### Rasperry Pi OS
+`sudo apt install git mariadb-server python3-pip memcached libmemcached-dev `
+
+#### Debian 12
+`sudo apt install git mariadb-server python3-pip memcached libmemcached-dev default-libmysqlclient-dev pkg-config`
+
+### Optional: Install proxy
+If you intend to use a proxy (recomended for public-facing production setups), we recomend nginx
+`sudo apt install nginx`
+
+## Database setup
+### mysql_secure_installation
+If you already have your database installed and configured, and are able to log in, skip down to the [Creating the database](#creating-the-database) section below. Otherwise, setup your newly installed database.
+
+`sudo mysql_secure_installation`
+
+Leave the root password blank, do not switch to unix socket, do reset the root password to something secure, and answer yes to the rest of the prompts. You can then log into your database with `sudo mysql`
+
+### Creating the database
+Once you're logged in, run the following commands, as root, to set up our database. Make sure you note down whatever you decide to make the password for the aime account, as you will need it to configure artemis.
+
+```sql
+CREATE USER 'aime'@'localhost' IDENTIFIED BY '<password>';
+CREATE DATABASE aime;
+GRANT Alter,Create,Delete,Drop,Index,Insert,References,Select,Update ON aime.* TO 'aime'@'localhost';
+quit
+```
+We have now set up our new user, `aime`, created a database called `aime` and given our user all the permissions it needs on every table of that database.
+
+### Configure memcached
+Under the file /etc/memcached.conf, please make sure the following parameters are set:
+
+```
+# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
+# Note that the daemon will grow to this size, but does not start out holding this much
+# memory
+
+-I 128m
+-m 1024
+```
+
+** This is mandatory to avoid memcached overload caused by Crossbeats or by massive profiles
+
+Restart memcached using:  sudo systemctl restart memcached
+
+## Getting ARTEMiS
+### Clone from gitea
+use `git clone https://gitea.tendokyu.moe/Hay1tsme/artemis.git` to pull down ARTEMiS into a folder called `artemis` created at wherever your current working directory is. `cd` into `artemis`.
+
+### Optional: Create a venv
+Python venvs are a way to install and manage packages on a per-project basis and are recomended on systems that will have multiple python scripts running on them to avoid dependancy issues. If this server will be running ARTEMiS and ONLY ARTEMiS, then it is possible to get away without creating one. If you do want to create one, you will have to install an additional package:
+
+`sudo apt install python3-venv` (like above, package name may vary depending on distro and python version)
+
+Now, simply run `python -m venv .venv` (may have to use python3 or python 3.11 instead of python) to create your virtual environment in the folder `.venv`. In order to install packages and run scripts in this environment, you have to 'activate' it by running `source .venv/bin/activate`. Your terminal should now have (venv) appended to it.
+
+### Optional: Use the develop branch
+By default, pulling down ARTEMiS from gitea will pull the `master` branch. This branch is updated less frequently, but is considered stable and ready for production use. If you'd rather have more updates, but a possibility for instability or bugs, you can switch to the develop branch by running `git checkout develop`. You can run `git checkout master` to switch back to stable.
+
+## Install python libraries
+Run `pip install -r requirements.txt` to install all of ARTEMiS' dependencies. If any installs fail, you may have missed a step in the [Install prerequisits](#install-prerequisits) section above. If you're absolutly sure you didn't, submit an issue on gitea.
+
+## Configuration
+### Copy example configs
+From the `artemis` directory, run `cp -r example_config config` to copy the example configuration files to a new folder called `config`. All of the config changes you make will be done in the `config` folder.
+
+### Optional: Generate AimeDB and Frontend JWT Secrets
+AimeDB and the frontend utalize JSON Web Tokens (JWT) for card authentication and session cookies respectivly. While generating a secret for AimeDB is optional, if you intend to run the frontend, a secret is required. You can generate a secret easily by running:
+
+`openssl rand --base64 64`
+
+With 64 being the number of bytes. You shouldn't need to go higher then 64, but you can if desired. **NOTE: When pasting secrets into the config file, make sure you remove any newlines!**
+
+### Edit `core.yaml`
+Before editing `core.yaml`, you should familiarize yourself with the name and function of each of the config options. You can find a full list in [config.md](config.md)
+
+Open `core.yaml` in the `config` folder in your prefered text editor. The only configuration option that it is absolutly mandatory to change is `aimedb`->`key`. This key must be set for the server to start, and the key must be correct, otherwise you will not be able to process aimedb requests. The correct key is floating around online, and finding it is left as an excersie to the reader. 
+
+Another option that should be changed is `database`->`password` to be the password you set when you created your database user. You did write it down somewhere, right?
+
+Since you are presumably not running the games on the same computer you're installing this server on, you're going to want to change `server`->`hostname` to be whatever hostname or IP address other PCs can reach this server by. Note that some games reject IPs and require hostnames, so setting a hostname is always recomended over an IP.
+
+### Edit game configs
+Every game has their own yaml file with settings that you may want to tweek. `InitialD Zero` and `Pokken` both have `hostname` fields in their config file that you should edit, and some games support encryption, if supplied with proper keys.
+
+### A note about IDZ
+InitialD Zero is currently the only game where it is required to specify encryption information (the AES key and at least one RSA key) for the game to start. These keys are, like the aimedb key, floating around online and will not be provided. If you don't have the keys, and don't plan on anybody connecting to your server playing InitialD Zero, it's best to set `enabled` to `False` in idz.yaml to disable the game.
+
+## Create database tables
+ARTEMiS uses alembic to manage datbase versioning. `dbutils.py` acts as a wrapper for alembic, and can execute some necessassary database functions. To create the database tables, run `python dbutils.py create`. Confirm that there are no errors, and you're good to go. If you intend to use the frontend, you may also want to run `python dbutils.py create-owner -a <your 20 digit access code here>` to create a superuser account to log in with.
+
+## Run ARTEMiS
+Once you have everything configured properly, simply run `python index.py` to start ARTEMiS. Verify that clients can connect to all services (allnet, billing, aimedb, and game servers) and setup is complete.

docs/INSTALL_WINDOWS.md

@@ -1,102 +1,77 @@
-# ARTEMiS - Windows 10/11 Guide
-This step-by-step guide assumes that you are using a fresh install of Windows 10/11 without MySQL installed, some of the steps can be skipped if you already have an installation with MySQL 8.0 or even some of the modules already present on your environment
+# Installing ARTEMiS on Windows
+This guide assumes a fresh install of Windows 10. Please be aware that due to the lack of memcached and the general woes of running a server on Windows, this is only recommended for local setups or small hosting-for-the-homies type servers.
 
-# Setup
-## Install Python Python 3.9 (recommended) or 3.10
-1. Download Python 3.9 : [Link](https://www.python.org/ftp/python/3.9.13/python-3.9.13-amd64.exe)
-2. Install python-3.9.13-amd64.exe
-    1. Select Customize installation
-    2. Make sure that pip, tcl/tk, and the for all users are checked and hit Next
-    3. Make sure that you enable "Create shortcuts for installed applications" and "Add Python to environment variables" and hit Install
+## Install prerequisites
+### Python
+- Python versions from 3.8 to 3.11 work with ARTEMiS. We recommend 3.11.
+    - https://www.python.org/ftp/python/3.11.7/python-3.11.7-amd64.exe
+- Install using whichever options best suit your environment, making sure that the Python executable is on path, such that you can open CMD, type `python --version` and see the version of Python you have installed.
+- If you already have a working version of Python installed, skip this step.
 
-## Install MySQL 8.0
-1. Download MySQL 8.0 Server : [Link](https://dev.mysql.com/get/Downloads/MySQLInstaller/mysql-installer-community-8.0.34.0.msi)
-2. Install mysql-installer-web-community-8.0.34.0.msi
-    1. Click on "Add ..." on the side
-    2. Click on the "+" next to MySQL Servers
-    3. Make sure MySQL Server 8.0.34 - X64 is under the products to be installed.
-    4. Hit Next and Next once installed
-    5. Select the configuration type "Development Computer"
-    6. Hit Next
-    7. Select "Use Legacy Authentication Method (Retain MySQL 5.x compatibility)" and hit Next
-    8. Enter a root password and then hit Next >
-    9. Leave everything under Windows Service as default and hit Next >
-    10. Click on Execute and for it to finish and hit Next> and then Finish
-3. Open MySQL 8.0 Command Line Client and login as your root user
-4. Change `<Enter Password Here>` to a new password for the user aime, type those commands to create your user and the database
+### MariaDB
+- It is always recommended to use MariaDB over MySQL because Oracle is a terrible company.
+- While the latest release of v10 is recommended, as it is an LTS release, v11 should work fine.
+    - https://ftp.osuosl.org/pub/mariadb//mariadb-10.11.6/winx64-packages/mariadb-10.11.6-winx64.msi
+- REMEMBER YOUR ROOT PASSWORD SO YOU CAN LOG IN IN FUTURE STEPS.
 
-```sql
-CREATE USER 'aime'@'localhost' IDENTIFIED BY '<Enter Password Here>';
+### Git
+- While technically optional, it is strongly recommended to obtain ARTEMiS via git clone instead of just downloading it.
+    - https://git-scm.com/download/win
+- It is recommended to use Notepad++ as the default editor (if you have it installed), other than that, the default settings should be fine.
+
+### Optional: GUI database viewer
+- Having a GUI database editor is recommended but not required.
+- MariaDB will try to install HeidiSQL, but we recommend DBeaver.
+    - https://dbeaver.io/download/
+
+## Obtain ARTEMiS
+### Via git (recommended)
+- `git clone https://gitea.tendokyu.moe/Hay1tsme/artemis.git` via cmd in whatever folder you want to install ARTEMiS.
+    - You can switch to the develop branch for latest changes via `git checkout develop`.
+
+### Via http download
+- Download [here](https://gitea.tendokyu.moe/Hay1tsme/artemis/archive/master.zip).
+    - Develop branch can be found [here](https://gitea.tendokyu.moe/Hay1tsme/artemis/archive/develop.zip).
+- Extract the zip file somewhere.
+
+## Database setup
+- Log into your server as root, either via GUI (recommended) or CMD
+- Create the `aime` user, replace `<password>` with a password you choose. Remember it!
+```
+CREATE USER 'aime'@'localhost' IDENTIFIED BY '<password>';
 CREATE DATABASE aime;
 GRANT Alter,Create,Delete,Drop,Index,Insert,References,Select,Update ON aime.* TO 'aime'@'localhost';
-FLUSH PRIVILEGES;
-exit;
 ```
+- If you create the database via a GUI, make sure you grant all the above permissions.
 
-## Install Python modules
-1. Change your work path to the artemis-master folder using 'cd' and install the requirements:
+## Create a venv
+- Python virtual environments are a good way to manage packages and make dealing with python and pip easier.
+- `python -m pip venv venv`
+- `venv\Scripts\activate.bat` to activate the venv whenever you need to interact with ARTEMiS.
+- All the rest of the steps assume your venv is activated.
 
-```shell
-pip install -r requirements.txt
-```	
+## Install pip modules
+- `pip install -r requirements.txt`
 
-## Copy/Rename the folder `example_config` to `config`
+## Setup configuration
+- Create a new `config` folder and copy the files in `example_config` over.
+- edit `core.yaml`
+    - Put the password you created for the aime user into the `database` section.
+    - Put in the aimedb key (YOU DO NOT GENERATE THIS KEY, FIND IT SOMEWHERE).
+    - Set your hostname to be whatever hostname or IP address games can reach your server at (many games reject localhost and 127.0.0.1).
+    - Optional: generate base64-encoded secrets for aimedb and frontend.
+    - See [config.md](docs/config.md) for a full list of options.
+- edit `idz.yaml`
+    - If you don't plan on anyone using your server to play Initial D Zero, it is best to disable it to cut down on console spam on boot.
+- Edit other game yamls
+    - Add keys, set hostnames, ports, etc. Specific settings will depend on the game. See [game_specific_info](docs/game_specific_info.md).
 
-## Adjust `config/core.yaml`
+## Create Database Tables
+- `python dbutils.py create`
 
-1. Make sure to change the server `hostname` to be set to your local machine IP (ex.: 192.168.xxx.xxx) 
-    - In case you want to run this only locally, set the following values:
+## Firewall
+- If you're planning on serving games not on your PC, open at least ports 80, 8443, and 22345 in windows firewall
+    - Also set `listen_address` to either your local IP to serve on your LAN, or `0.0.0.0` for all interfaces, to accept connections from other places.
 
-```yaml
-server:
-    listen_address: 0.0.0.0
-title: 
-    hostname: 192.168.xxx.xxx
-```
-
-1. Adjust the proper MySQL information you created earlier
-```yaml
-database:
-  host: "localhost"
-  username: "aime"
-  password: "<Enter Password Here>"
-  name: "aime"
-```
-3. Add the AimeDB key at the bottom of the file
-4. If the webui is needed, change the flag from False to True
-
-## Create the database tables for ARTEMiS
-
-```shell
-python dbutils.py create
-```
-
-## Firewall Adjustements 
-Make sure the following ports are open both on your router and local Windows firewall in case you want to use this for public use (NOT recommended):
-> Port 80 (TCP), 443 (TCP), 8443 (TCP), 22345 (TCP), 8080 (TCP), 8090 (TCP) **webui, 8444 (TCP) **mucha
-
-## Running the ARTEMiS instance
-```shell
-python index.py
-```
-
-# Troubleshooting
-
-## Game does not connect to ARTEMiS Allnet server
-1. Double-check your core.yaml, the listen_address is most likely either not binded to the proper IP or the port is not opened
-
-## Game does not connect to Title Server
-1. Verify that your core.yaml is setup properly for both the server listen_address and title hostname
-2. Boot your game and verify that an AllNet response does show and if it does, attempt to open the URI that is shown under a browser such as Edge, Chrome & Firefox.
-3. If a page is shown, the server is working properly and if it doesn't, double check your port forwarding and also that you have entered the proper local IP under the Title hostname in core.yaml.
-
-## Unhandled command under AimeDB
-1. Double check your AimeDB key under core.yaml, it is incorrect.
-
-## AttributeError: module 'collections' has no attribute 'Hashable'
-1. This means the pyYAML module is obsolete, simply rerun pip with the -U (force update) flag, as shown below.
-    - Change your work path to the artemis-master (or artemis-develop) folder using 'cd' and run the following commands:
-
-```shell
-pip install -r requirements.txt -U
-```
+## Start ARTEMiS
+- `python index.py`

docs/INSTALL_WINDOWS_old.md

@@ -0,0 +1,102 @@
+# ARTEMiS - Windows 10/11 Guide
+This step-by-step guide assumes that you are using a fresh install of Windows 10/11 without MySQL installed, some of the steps can be skipped if you already have an installation with MySQL 8.0 or even some of the modules already present on your environment
+
+# Setup
+## Install Python Python 3.9 (recommended) or 3.10
+1. Download Python 3.9 : [Link](https://www.python.org/ftp/python/3.9.13/python-3.9.13-amd64.exe)
+2. Install python-3.9.13-amd64.exe
+    1. Select Customize installation
+    2. Make sure that pip, tcl/tk, and the for all users are checked and hit Next
+    3. Make sure that you enable "Create shortcuts for installed applications" and "Add Python to environment variables" and hit Install
+
+## Install MySQL 8.0
+1. Download MySQL 8.0 Server : [Link](https://dev.mysql.com/get/Downloads/MySQLInstaller/mysql-installer-community-8.0.34.0.msi)
+2. Install mysql-installer-web-community-8.0.34.0.msi
+    1. Click on "Add ..." on the side
+    2. Click on the "+" next to MySQL Servers
+    3. Make sure MySQL Server 8.0.34 - X64 is under the products to be installed.
+    4. Hit Next and Next once installed
+    5. Select the configuration type "Development Computer"
+    6. Hit Next
+    7. Select "Use Legacy Authentication Method (Retain MySQL 5.x compatibility)" and hit Next
+    8. Enter a root password and then hit Next >
+    9. Leave everything under Windows Service as default and hit Next >
+    10. Click on Execute and for it to finish and hit Next> and then Finish
+3. Open MySQL 8.0 Command Line Client and login as your root user
+4. Change `<Enter Password Here>` to a new password for the user aime, type those commands to create your user and the database
+
+```sql
+CREATE USER 'aime'@'localhost' IDENTIFIED BY '<Enter Password Here>';
+CREATE DATABASE aime;
+GRANT Alter,Create,Delete,Drop,Index,Insert,References,Select,Update ON aime.* TO 'aime'@'localhost';
+FLUSH PRIVILEGES;
+exit;
+```
+
+## Install Python modules
+1. Change your work path to the artemis-master folder using 'cd' and install the requirements:
+
+```shell
+pip install -r requirements.txt
+```	
+
+## Copy/Rename the folder `example_config` to `config`
+
+## Adjust `config/core.yaml`
+
+1. Make sure to change the server `hostname` to be set to your local machine IP (ex.: 192.168.xxx.xxx) 
+    - In case you want to run this only locally, set the following values:
+
+```yaml
+server:
+    listen_address: 0.0.0.0
+title: 
+    hostname: 192.168.xxx.xxx
+```
+
+1. Adjust the proper MySQL information you created earlier
+```yaml
+database:
+  host: "localhost"
+  username: "aime"
+  password: "<Enter Password Here>"
+  name: "aime"
+```
+3. Add the AimeDB key at the bottom of the file
+4. If the webui is needed, change the flag from False to True
+
+## Create the database tables for ARTEMiS
+
+```shell
+python dbutils.py create
+```
+
+## Firewall Adjustements 
+Make sure the following ports are open both on your router and local Windows firewall in case you want to use this for public use (NOT recommended):
+> Port 80 (TCP), 443 (TCP), 8443 (TCP), 22345 (TCP), 8080 (TCP), 8090 (TCP) **webui, 8444 (TCP) **mucha
+
+## Running the ARTEMiS instance
+```shell
+python index.py
+```
+
+# Troubleshooting
+
+## Game does not connect to ARTEMiS Allnet server
+1. Double-check your core.yaml, the listen_address is most likely either not binded to the proper IP or the port is not opened
+
+## Game does not connect to Title Server
+1. Verify that your core.yaml is setup properly for both the server listen_address and title hostname
+2. Boot your game and verify that an AllNet response does show and if it does, attempt to open the URI that is shown under a browser such as Edge, Chrome & Firefox.
+3. If a page is shown, the server is working properly and if it doesn't, double check your port forwarding and also that you have entered the proper local IP under the Title hostname in core.yaml.
+
+## Unhandled command under AimeDB
+1. Double check your AimeDB key under core.yaml, it is incorrect.
+
+## AttributeError: module 'collections' has no attribute 'Hashable'
+1. This means the pyYAML module is obsolete, simply rerun pip with the -U (force update) flag, as shown below.
+    - Change your work path to the artemis-master (or artemis-develop) folder using 'cd' and run the following commands:
+
+```shell
+pip install -r requirements.txt -U
+```

docs/config.md

@@ -8,7 +8,7 @@
 - `allow_user_registration`: Allows users to register in-game via the AimeDB `register` function. Disable to be able to control who can use cards on your server. Default `True`
 - `allow_unregistered_serials`: Allows games that do not have registered keychips to connect and authenticate. Disable to restrict who can connect to your server. Recomended to disable for production setups. Default `True`
 - `name`: Name for the server, used by some games in their default MOTDs. Default `ARTEMiS`
-- `is_develop`: Flags that the server is a development instance without a proxy standing in front of it. Setting to `False` tells the server not to listen for SSL, because the proxy should be handling all SSL-related things, among other things. Default `True`
+- `is_develop`: Flags that the server is a development instance, and enables some useful development features. Disable for production setups. Default `True`.
 - `is_using_proxy`: Flags that you'll be using some other software, such as nginx, to proxy requests, and to send `proxy_port` or `proxy_port_ssl` to games instead of `port`. Default `False`
 - `proxy_port`: Which port your front-facing proxy will be listening on. Ignored if `is_using_proxy` is `False` or if set to `0`. Default `0`
 - `proxy_port`: Which port your front-facing proxy will be listening for ssl connections on. Ignored if `is_using_proxy` is `False` or if set to `0`. Default `0`
@@ -26,29 +26,30 @@
 - `name`: Name of the database the server should expect. Default `aime`
 - `port`: Port the database server is listening on. Default `3306`
 - `protocol`: Protocol used in the connection string, e.i `mysql` would result in `mysql://...`. Default `mysql`
-- `sha2_password`: Weather or not the password in the connection string should be hashed via SHA2. Default `False`
+- `sha2_password`: Whether or not the password in the connection string should be hashed via SHA2. Default `False`
 - `loglevel`: Logging level for the database. Default `info`
 - `memcached_host`: Host of the memcached server. Default `localhost`
 ## Frontend
-- `enable`: Weather or not the frontend should run. Default `False`
+- `enable`: Whether or not the frontend servlet should run. Frontend can still be run via `python -m uvicorn core.frontend:app` even if this is set to `False`. Default `False`
 - `port`: Port the frontend should listen on. Default `8080`
 - `loglevel`: Logging level for the frontend server. Default `info`
 - `secret`: Base64-encoded JWT secret for session cookies, generated by you. Default `""`
 ## Allnet
-- `standalone`: Weather allnet should be launched as a standalone service on it's own port.
-- `port`: Port the billing server should listen for connections on. Games are hardcoded to ask for port `80` so only change if you have a proxy redirecting properly. Default `80`
+- `standalone`: Whether allnet should launch it's own servlet on it's own port, or be part of the main servlet on the default port. Disable if you either have something proxying `naominet.jp` requests to port 80, or have port 80 set in `server` -> `port`
+- `port`: Port the allnet server should listen for connections on if it's running standalone. Games are hardcoded to ask for port `80` so only change if you have a proxy redirecting properly. Ignored if `standalone` is `False`. Default `80`
 - `loglevel`: Logging level for the allnet server. Default `info`
 - `allow_online_updates`: Allow allnet to distribute online updates via DownloadOrders. This system is currently non-functional, so leave it disabled. Default `False`
 - `update_cfg_folder`: Folder where delivery INI files will be checked for. Ignored if `allow_online_updates` is `False`. Default `""`
 ## Billing
-- `standalone`: Weather or not the billing server should be launched as a standalone service on it's own port. Setting this to `True` requires that you have `ssl_key` and `ssl_cert` set. Default `False`
+- `standalone`: Whether the billing server should launch it's own servlet on it's own port, or be part of the main servlet on the default port. Setting this to `True` requires that you have `ssl_key` and `ssl_cert` set. Default `False`
 - `loglevel`: Logging level for the billing server. Default `info`
 - `port`: Port the billing server should listen for connections on. Games are hardcoded to ask for port `8443` so only change if you have a proxy redirecting properly. Ignored if `standalone` is `False`. Default `8443`
 - `ssl_key`: Location of the ssl server key for the billing server. Ignored if `standalone` is `False`. Default `cert/server.key`
 - `ssl_cert`: Location of the ssl server certificate for the billing server. Ignored if `standalone` is `False`.  Must match the CA distributed to users or the billing server will not connect. Default `cert/server.pem`
 - `signing_key`: Location of the RSA Private key used to sign billing requests. Must match the public key distributed to users or the billing server will not connect. Default `cert/billing.key`
 ## Aimedb
-- `enable`: Weather or not aimedb should run. Default `True`
+- `enable`: Whether or not aimedb should run. Default `True`
+- `listen_address`: IP Address or hostname that the aimedb server will listen for connections on. Leave this blank to use the listen address under `server`. Default `""`
 - `loglevel`: Logging level for the aimedb server. Default `info`
 - `port`: Port the aimedb server should listen for connections on. Games are hardcoded to ask for port `22345` so only change if you have a proxy redirecting properly. Default `22345`
 - `key`: Key to encrypt/decrypt aimedb requests and responses. MUST be set or the server will not start. If set incorrectly, your server will not properly handle aimedb requests. Default `""`

docs/game_specific_info.md

@@ -9,7 +9,15 @@ using the megaime database. Clean installations always create the latest databas
 To upgrade the core database and the database for every game, execute:
 
 ```shell
-python dbutils.py autoupgrade
+python dbutils.py upgrade
+```
+
+If you are using the old master branch that was not setup with alembic, make sure to do the following steps in order:
+- Pull down latest master/develop
+- Update core.yaml
+- Back up your existing database
+```shell
+python dbutils.py migrate
 ```
 
 # Table of content
@@ -22,7 +30,7 @@ python dbutils.py autoupgrade
     - [Card Maker](#card-maker)
     - [WACCA](#wacca)
     - [Sword Art Online Arcade](#sao)
-	- [Initial D THE ARCADE](#initial-d-the-arcade)
+    - [Initial D THE ARCADE](#initial-d-the-arcade)
 
 
 # Supported Games
@@ -93,7 +101,7 @@ crypto:
 Always make sure your database (tables) are up-to-date:
 
 ```shell
-python dbutils.py --game SDBT upgrade
+python dbutils.py upgrade
 ```
 
 ### Online Battle
@@ -192,6 +200,7 @@ Config file is located in `config/cxb.yaml`.
 | SDEZ      | 18         | maimai DX UNiVERSE PLUS |
 | SDEZ      | 19         | maimai DX FESTiVAL      |
 | SDEZ      | 20         | maimai DX FESTiVAL PLUS |
+| SDEZ      | 21         | maimai DX BUDDiES       |
 
 ### Importer
 
@@ -215,7 +224,7 @@ The importer for maimai Pre-DX will import Events and Music. Not all games will
 Always make sure your database (tables) are up-to-date:
 
 ```shell
-python dbutils.py --game SDEZ upgrade
+python dbutils.py upgrade
 ```
 
 Pre-Dx uses the same database as DX, so only upgrade using the SDEZ game code!
@@ -259,7 +268,7 @@ In order to use custom PV Lists, simply drop in your .dat files inside of /title
 Always make sure your database (tables) are up-to-date:
 
 ```shell
-python dbutils.py --game SBZV upgrade
+python dbutils.py upgrade
 ```
 
 ## O.N.G.E.K.I.
@@ -315,7 +324,7 @@ crypto:
 Always make sure your database (tables) are up-to-date:
 
 ```shell
-python dbutils.py --game SDDT upgrade
+python dbutils.py upgrade
 ```
 
 ### Controlling Events (Ranking Event, Technical Challenge Event, Mission Event)
@@ -406,6 +415,7 @@ After that, on next login the present should be received (or whenever it suppose
   * UNiVERSE PLUS: Yes
   * FESTiVAL: Yes (added in A031)
   * FESTiVAL PLUS: Yes (added in A035)
+  * BUDDiES: Yes (added in A039)
 * O.N.G.E.K.I. bright MEMORY: Yes
 
 
@@ -542,7 +552,7 @@ Config file is located in `config/wacca.yaml`.
 Always make sure your database (tables) are up-to-date:
 
 ```shell
-python dbutils.py --game SDFE upgrade
+python dbutils.py upgrade
 ```
 
 ### VIP Rewards
@@ -594,7 +604,7 @@ Below is a list of VIP rewards. Currently, VIP is not implemented, and thus thes
 In order to use the importer locate your game installation folder and execute:
 
 ```shell
-python read.py --game SDEW --version <version ID> --binfolder /path/to/game/extractedassets
+python read.py --game SDEW --version 0 --binfolder /titles/sao/data/
 ```
 
 The importer for SAO will import all items, heroes, support skills and titles data.
@@ -615,21 +625,23 @@ Config file is located in `config/sao.yaml`.
 Always make sure your database (tables) are up-to-date:
 
 ```shell
-python dbutils.py --game SDEW upgrade
+python dbutils.py upgrade
 ```
 
 ### Notes
 - Defrag Match will crash at loading
 - Co-Op Online is not supported
-- Shop is not functionnal
+- Shop is displayed but cannot purchase heroes or items
 - Player title is currently static and cannot be changed in-game
 - QR Card Scanning currently only load a static hero
-
-**Network hashing in GssSite.dll must be disabled**
+- Ex-quests progression not supported yet
+- Daily Missions not implemented
+- EX TOWER 1,2 & 3 are not yet supported
+- Daily Yui coin not yet fixed
 
 ### Credits for SAO support:
 
-- Midorica - Limited Network Support
+- Midorica - Network Support
 - Dniel97 - Helping with network base
 - tungnotpunk - Source
 
@@ -678,7 +690,7 @@ Config file is located in `config/idac.yaml`.
 Always make sure your database (tables) are up-to-date:
 
 ```shell
-python dbutils.py --game SDGT upgrade
+python dbutils.py upgrade
 ```
 
 ### Notes

docs/prod.md

@@ -13,9 +13,13 @@ Step 1 is to edit your artemis configuration. Some recomended changes:
 - `allow_unregistered_serials` -> `False`
 ### `billing`
 - `standalone` -> `False`
+### `allnet`
+- `standalone` -> `False`
 ### `frontend`
 - `enable` -> `True` if you want the frontend
-- `port` -> `8080` if you're using the default nginx config
+- `port` -> `8090` if you're using the default nginx config, otherwise whatever port your proxy will be sending requests to
+### `aimedb`
+- `listen_address` -> `0.0.0.0` unless you're proxying aimedb requests (not recomended at this time), in which case, leave this option unchanged
 
 If you plan to serve artemis behind a VPN, these additional settings are also recomended
 - `check_arcade_ip` -> `True`

example_config/core.yaml

@@ -55,6 +55,7 @@ billing:
 
 aimedb:
   enable: True
+  listen_address: ""
   loglevel: "info"
   port: 22345
   key: ""

example_config/mai2.yaml

@@ -12,3 +12,6 @@ uploads:
   photos_dir: ""
   movies: False
   movies_dir: ""
+
+crypto:
+  encrypted_only: False

example_config/nginx_example.conf

@@ -6,7 +6,7 @@ server {
 	location / {
 		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 		proxy_pass_request_headers on;
-		proxy_pass http://localhost:8080/;
+		proxy_pass http://127.0.0.1:8080/;
 	}
 }
 
@@ -18,7 +18,7 @@ server {
 	location / {
 		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 		proxy_pass_request_headers on;
-		proxy_pass http://localhost:8080/;
+		proxy_pass http://127.0.0.1:8080/;
 	}
 }
 
@@ -38,7 +38,9 @@ server {
 	ssl_prefer_server_ciphers off;
 
 	location / {
-		proxy_pass http://localhost:8080/;
+		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+		proxy_pass_request_headers on;
+		proxy_pass http://127.0.0.1:8080/;
 	}
 }
 
@@ -58,7 +60,9 @@ server {
 	ssl_prefer_server_ciphers off;
 
 	location / {
-		proxy_pass http://localhost:8080/;
+		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+		proxy_pass_request_headers on;
+		proxy_pass http://127.0.0.1:8080/;
 	}
 }
 
@@ -70,7 +74,7 @@ server {
     location / {
         return 301 https://$host$request_uri;
 		# If you don't want https redirection, or are using something like cloudflare to manage HTTPS, comment out the line above and uncomment the line below
-		# proxy_pass http://localhost:8080/;
+		# proxy_pass http://127.0.0.1:8090/;
     }
 }
 
@@ -96,6 +100,6 @@ server {
 	location / {
 		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 		proxy_pass_request_headers on;
-		proxy_pass http://localhost:8080/;
+		proxy_pass http://127.0.0.1:8090/;
 	}
 }

index.py

@@ -40,7 +40,8 @@ async def launch_billing(cfg: CoreConfig) -> None:
         log_level="info" if cfg.server.is_develop else "critical",
         ssl_version=3,
         ssl_certfile=cfg.billing.ssl_cert,
-        ssl_keyfile=cfg.billing.ssl_key
+        ssl_keyfile=cfg.billing.ssl_key,
+        ssl_ciphers="DEFAULT:!aNULL:!eNULL:!MD5:!3DES:!DES:!RC4:!IDEA:!SEED:!aDSS:!SRP:!PSK",
     )
     server = uvicorn.Server(server_cfg)
     await server.serve()

readme.md

@@ -4,37 +4,69 @@ A network service emulator for games running SEGA'S ALL.NET service, and similar
 # Supported games
 Games listed below have been tested and confirmed working. Only game versions older then the version currently active in arcades, or games versions that have not recieved a major update in over one year, are supported.
 
-+ CHUNITHM
-    + All versions up to SUN PLUS
-
-+ crossbeats REV.
-    + All versions + omnimix
-
-+ maimai DX
-    + All versions up to FESTiVAL PLUS
-
-+ Hatsune Miku: Project DIVA Arcade
-    + All versions
-
 + Card Maker
     + 1.30
     + 1.35
 
-+ O.N.G.E.K.I.
-    + All versions up to bright MEMORY
++ CHUNITHM INTL
+    + SUPERSTAR
+    + SUPERSTAR PLUS
+    + NEW
+    + NEW PLUS
+    + SUN
+    + SUN PLUS
 
-+ WACCA
-    + Lily R
-    + Reverse
++ CHUNITHM JP
+    + AIR
+    + AIR PLUS
+    + AMAZON
+    + AMAZON PLUS
+    + CRYSTAL
+    + CRYSTAL PLUS
+    + PARADISE
+    + PARADISE LOST
+    + NEW
+    + NEW PLUS
+    + SUN
+    + SUN PLUS
+
++ crossbeats REV.
+    + Crossbeats REV.
+    + Crossbeats REV. SUNRiSE S1
+    + Crossbeats REV. SUNRiSE S2 + omnimix
+
++ Hatsune Miku: Project DIVA Arcade
+    + Future Tone Arcade - All versions
+
++ Initial D THE ARCADE
+    + Season 2
+
++ maimai DX
+    + Splash
+    + Splash Plus
+    + UNiVERSE
+    + UNiVERSE PLUS
+    + FESTiVAL
+    + FESTiVAL PLUS
+    + BUDDiES
+
++ O.N.G.E.K.I.
+    + SUMMER
+    + SUMMER PLUS
+    + R.E.D.
+    + R.E.D. PLUS
+    + bright
+    + bright MEMORY
 
 + POKKÉN TOURNAMENT
     + Final Online
 
-+ Sword Art Online Arcade (partial support)
-    + Final
++ Sword Art Online Arcade
+    + Final (Single player only)
 
-+ Initial D THE ARCADE
-    + Season 2
++ WACCA
+    + Lily R
+    + Reverse
 
 ## Requirements
 - python 3 (tested working with 3.9 and 3.10, other versions YMMV)
@@ -43,7 +75,7 @@ Games listed below have been tested and confirmed working. Only game versions ol
 - mysql/mariadb server
 
 ## Setup guides
-Follow the platform-specific guides for [windows](docs/INSTALL_WINDOWS.md), [ubuntu](docs/INSTALL_UBUNTU.md) or [docker](docs/INSTALL_DOCKER.md) to setup and run the server.
+Follow the platform-specific guides for [windows](docs/INSTALL_WINDOWS.md), [linux (Debian 12 or Rasperry Pi OS recomended, but anything works)](docs/INSTALL_LINUX.md) or [docker](docs/INSTALL_DOCKER.md) to setup and run the server.
 
 ## Game specific information
 Read [Games specific info](docs/game_specific_info.md) for all supported games, importer settings, configuration option and database upgrades.

requirements.txt

@@ -1,22 +1,24 @@
-mypy
-wheel
-pytz
-pyyaml
-sqlalchemy==1.4.46
-mysqlclient
-pyopenssl
-service_identity
-PyCryptodome
-inflection
-coloredlogs
-pylibmc; platform_system != "Windows"
-wacky
-bcrypt
-jinja2
-protobuf
-pillow
-pyjwt==2.8.0
-websockets
-starlette
-asyncio
-uvicorn
+mypy
+wheel
+pytz
+pyyaml
+sqlalchemy==1.4.46
+mysqlclient
+pyopenssl
+service_identity
+PyCryptodome
+inflection
+coloredlogs
+pylibmc; platform_system != "Windows"
+wacky
+bcrypt
+jinja2
+protobuf
+pillow
+pyjwt==2.8.0
+websockets
+starlette
+asyncio
+uvicorn
+alembic
+python-multipart

titles/chuni/__init__.py

@@ -1,9 +1,11 @@
-from titles.chuni.index import ChuniServlet
-from titles.chuni.const import ChuniConstants
-from titles.chuni.database import ChuniData
-from titles.chuni.read import ChuniReader
+from .index import ChuniServlet
+from .const import ChuniConstants
+from .database import ChuniData
+from .read import ChuniReader
+from .frontend import ChuniFrontend
 
 index = ChuniServlet
 database = ChuniData
 reader = ChuniReader
+frontend = ChuniFrontend
 game_codes = [ChuniConstants.GAME_CODE, ChuniConstants.GAME_CODE_NEW, ChuniConstants.GAME_CODE_INT]

titles/chuni/base.py

@@ -811,6 +811,12 @@ class ChuniBase:
         upsert = data["upsertUserAll"]
         user_id = data["userId"]
 
+        if int(user_id) & 0x1000000000001 == 0x1000000000001:
+            place_id = int(user_id) & 0xFFFC00000000
+            
+            self.logger.info("Guest play from place ID %d, ignoring.", place_id)
+            return {"returnCode": "1"}
+
         if "userData" in upsert:
             try:
                 upsert["userData"][0]["userName"] = self.read_wtf8(
@@ -925,6 +931,17 @@ class ChuniBase:
             for rp in upsert["userRecentPlayerList"]:
                 pass
 
+        for rating_type in {"userRatingBaseList", "userRatingBaseHotList", "userRatingBaseNextList"}:
+            if rating_type not in upsert:
+                continue
+            
+            await self.data.profile.put_profile_rating(
+                user_id,
+                self.version,
+                rating_type,
+                upsert[rating_type],
+            )
+
         return {"returnCode": "1"}
 
     async def handle_upsert_user_chargelog_api_request(self, data: Dict) -> Dict:

titles/chuni/frontend.py

@@ -0,0 +1,246 @@
+from typing import List
+from starlette.routing import Route, Mount
+from starlette.requests import Request
+from starlette.responses import Response, RedirectResponse
+from os import path
+import yaml
+import jinja2
+
+from core.frontend import FE_Base, UserSession
+from core.config import CoreConfig
+from .database import ChuniData
+from .config import ChuniConfig
+from .const import ChuniConstants
+
+
+class ChuniFrontend(FE_Base):
+    def __init__(
+        self, cfg: CoreConfig, environment: jinja2.Environment, cfg_dir: str
+    ) -> None:
+        super().__init__(cfg, environment)
+        self.data = ChuniData(cfg)
+        self.game_cfg = ChuniConfig()
+        if path.exists(f"{cfg_dir}/{ChuniConstants.CONFIG_NAME}"):
+            self.game_cfg.update(
+                yaml.safe_load(open(f"{cfg_dir}/{ChuniConstants.CONFIG_NAME}"))
+            )
+        self.nav_name = "Chunithm"
+
+    def get_routes(self) -> List[Route]:
+        return [
+            Route("/", self.render_GET, methods=['GET']),
+            Route("/rating", self.render_GET_rating, methods=['GET']),
+            Mount("/playlog", routes=[
+                Route("/", self.render_GET_playlog, methods=['GET']),
+                Route("/{index}", self.render_GET_playlog, methods=['GET']),
+            ]),
+            Route("/update.name", self.update_name, methods=['POST']),
+            Route("/version.change", self.version_change, methods=['POST']),
+        ]
+
+    async def render_GET(self, request: Request) -> bytes:
+        template = self.environment.get_template(
+            "titles/chuni/templates/chuni_index.jinja"
+        )
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh:
+            usr_sesh = UserSession()
+
+        if usr_sesh.user_id > 0:
+            versions = await self.data.profile.get_all_profile_versions(usr_sesh.user_id)
+            profile = []
+            if versions:
+                # chunithm_version is -1 means it is not initialized yet, select a default version from existing.
+                if usr_sesh.chunithm_version < 0:
+                    usr_sesh.chunithm_version = versions[0]
+                profile = await self.data.profile.get_profile_data(usr_sesh.user_id, usr_sesh.chunithm_version)
+
+            resp = Response(template.render(
+                title=f"{self.core_config.server.name} | {self.nav_name}",
+                game_list=self.environment.globals["game_list"],
+                sesh=vars(usr_sesh),
+                user_id=usr_sesh.user_id,
+                profile=profile,
+                version_list=ChuniConstants.VERSION_NAMES,
+                versions=versions,
+                cur_version=usr_sesh.chunithm_version
+            ), media_type="text/html; charset=utf-8")
+
+            if usr_sesh.chunithm_version >= 0:
+                encoded_sesh = self.encode_session(usr_sesh)
+                resp.set_cookie("ARTEMIS_SESH", encoded_sesh)
+            return resp
+
+        else:
+            return RedirectResponse("/gate/", 303)
+
+    async def render_GET_rating(self, request: Request) -> bytes:
+        template = self.environment.get_template(
+            "titles/chuni/templates/chuni_rating.jinja"
+        )
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh:
+            usr_sesh = UserSession()
+
+        if usr_sesh.user_id > 0:
+            if usr_sesh.chunithm_version < 0:
+                return RedirectResponse("/game/chuni/", 303)
+            profile = await self.data.profile.get_profile_data(usr_sesh.user_id, usr_sesh.chunithm_version)
+            rating = await self.data.profile.get_profile_rating(usr_sesh.user_id, usr_sesh.chunithm_version)
+            hot_list=[]
+            base_list=[]
+            if profile and rating:
+                song_records = []
+                for song in rating:
+                    music_chart = await self.data.static.get_music_chart(usr_sesh.chunithm_version, song.musicId, song.difficultId)
+                    if music_chart:
+                        if (song.score < 800000):
+                            song_rating = 0
+                        elif (song.score >= 800000 and song.score < 900000):
+                            song_rating = music_chart.level / 2 - 5
+                        elif (song.score >= 900000 and song.score < 925000):
+                            song_rating = music_chart.level - 5
+                        elif (song.score >= 925000 and song.score < 975000):
+                            song_rating = music_chart.level - 3
+                        elif (song.score >= 975000 and song.score < 1000000):
+                            song_rating = (song.score - 975000) / 2500 * 0.1 + music_chart.level
+                        elif (song.score >= 1000000 and song.score < 1005000):
+                            song_rating = (song.score - 1000000) / 1000 * 0.1 + 1 + music_chart.level
+                        elif (song.score >= 1005000 and song.score < 1007500):
+                            song_rating = (song.score - 1005000) / 500 * 0.1 + 1.5 + music_chart.level
+                        elif (song.score >= 1007500 and song.score < 1009000):
+                            song_rating = (song.score - 1007500) / 100 * 0.01 + 2 + music_chart.level
+                        elif (song.score >= 1009000):
+                            song_rating = 2.15 + music_chart.level
+                        song_rating = int(song_rating * 10 ** 2) / 10 ** 2
+                        song_records.append({
+                            "difficultId": song.difficultId,
+                            "musicId": song.musicId,
+                            "title": music_chart.title,
+                            "level": music_chart.level,
+                            "score": song.score,
+                            "type": song.type,
+                            "song_rating": song_rating,
+                        })
+                hot_list = [obj for obj in song_records if obj["type"] == "userRatingBaseHotList"]
+                base_list = [obj for obj in song_records if obj["type"] == "userRatingBaseList"]
+            return Response(template.render(
+                title=f"{self.core_config.server.name} | {self.nav_name}",
+                game_list=self.environment.globals["game_list"],
+                sesh=vars(usr_sesh),
+                profile=profile,
+                hot_list=hot_list,
+                base_list=base_list,
+            ), media_type="text/html; charset=utf-8")
+        else:
+            return RedirectResponse("/gate/", 303)
+
+    async def render_GET_playlog(self, request: Request) -> bytes:
+        template = self.environment.get_template(
+            "titles/chuni/templates/chuni_playlog.jinja"
+        )
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh:
+            usr_sesh = UserSession()
+
+        if usr_sesh.user_id > 0:
+            if usr_sesh.chunithm_version < 0:
+                return RedirectResponse("/game/chuni/", 303)
+            path_index = request.path_params.get('index')
+            if not path_index or int(path_index) < 1:
+                index = 0
+            else:
+                index = int(path_index) - 1 # 0 and 1 are 1st page
+            user_id = usr_sesh.user_id
+            playlog_count = await self.data.score.get_user_playlogs_count(user_id)
+            if playlog_count < index * 20 :
+                return Response(template.render(
+                    title=f"{self.core_config.server.name} | {self.nav_name}",
+                    game_list=self.environment.globals["game_list"],
+                    sesh=vars(usr_sesh),
+                    playlog_count=0
+                ), media_type="text/html; charset=utf-8")
+            playlog = await self.data.score.get_playlogs_limited(user_id, index, 20)
+            playlog_with_title = []
+            for record in playlog:
+                music_chart = await self.data.static.get_music_chart(usr_sesh.chunithm_version, record.musicId, record.level)
+                if music_chart:
+                    difficultyNum=music_chart.level
+                    artist=music_chart.artist
+                    title=music_chart.title
+                else:
+                    difficultyNum=0
+                    artist="unknown"
+                    title="musicid: " + str(record.musicId)
+                playlog_with_title.append({
+                    "raw": record,
+                    "title": title,
+                    "difficultyNum": difficultyNum,
+                    "artist": artist,
+                })
+            return Response(template.render(
+                title=f"{self.core_config.server.name} | {self.nav_name}",
+                game_list=self.environment.globals["game_list"],
+                sesh=vars(usr_sesh),
+                user_id=usr_sesh.user_id,
+                playlog=playlog_with_title,
+                playlog_count=playlog_count
+            ), media_type="text/html; charset=utf-8")
+        else:
+            return RedirectResponse("/gate/", 303)
+
+    async def update_name(self, request: Request) -> bytes:
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh:
+            return RedirectResponse("/gate/", 303)
+
+        form_data = await request.form()
+        new_name: str  = form_data.get("new_name")
+        new_name_full = ""
+
+        if not new_name:
+            return RedirectResponse("/gate/?e=4", 303)
+
+        if len(new_name) > 8:
+            return RedirectResponse("/gate/?e=8", 303)
+
+        for x in new_name: # FIXME: This will let some invalid characters through atm
+            o = ord(x)
+            try:
+                if o == 0x20:
+                    new_name_full += chr(0x3000)
+                elif o < 0x7F and o > 0x20:
+                    new_name_full += chr(o + 0xFEE0)
+                elif o <= 0x7F:
+                    self.logger.warn(f"Invalid ascii character {o:02X}")
+                    return RedirectResponse("/gate/?e=4", 303)
+                else:
+                    new_name_full += x
+
+            except Exception as e:
+                self.logger.error(f"Something went wrong parsing character {o:04X} - {e}")
+                return RedirectResponse("/gate/?e=4", 303)
+
+        if not await self.data.profile.update_name(usr_sesh.user_id, new_name_full):
+            return RedirectResponse("/gate/?e=999", 303)
+
+        return RedirectResponse("/game/chuni/?s=1", 303)
+
+    async def version_change(self, request: Request):
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh:
+            usr_sesh = UserSession()
+
+        if usr_sesh.user_id > 0:
+            form_data = await request.form()
+            chunithm_version = form_data.get("version")
+            self.logger.info(f"version change to: {chunithm_version}")
+            if(chunithm_version.isdigit()):
+                usr_sesh.chunithm_version=int(chunithm_version)
+                encoded_sesh = self.encode_session(usr_sesh)
+                self.logger.info(f"Created session with JWT {encoded_sesh}")
+                resp = RedirectResponse("/game/chuni/", 303)
+                resp.set_cookie("ARTEMIS_SESH", encoded_sesh)
+            return resp
+        else:
+            return RedirectResponse("/gate/", 303)

titles/chuni/index.py

@@ -195,11 +195,11 @@ class ChuniServlet(BaseServlet):
               internal_ver = ChuniConstants.VER_CHUNITHM_NEW_PLUS
           elif version >= 210 and version < 215:  # SUN
               internal_ver = ChuniConstants.VER_CHUNITHM_SUN
-          elif version >= 215: # SUN
+          elif version >= 215: # SUN PLUS
               internal_ver = ChuniConstants.VER_CHUNITHM_SUN_PLUS
         elif game_code == "SDGS": # Int
-          if version < 110: # SUPERSTAR
-              internal_ver = ChuniConstants.VER_CHUNITHM_PARADISE # FIXME: Not sure what was intended to go here? was just "PARADISE"
+          if version < 110: # SUPERSTAR / SUPERSTAR PLUS
+              internal_ver = ChuniConstants.VER_CHUNITHM_PARADISE # SUPERSTAR / SUPERSTAR PLUS worked fine with it
           elif version >= 110 and version < 115: # NEW
               internal_ver = ChuniConstants.VER_CHUNITHM_NEW
           elif version >= 115 and version < 120: # NEW PLUS!!
@@ -272,7 +272,13 @@ class ChuniServlet(BaseServlet):
         self.logger.info(f"v{version} {endpoint} request from {client_ip}")
         self.logger.debug(req_data)
 
-        endpoint = endpoint.replace("C3Exp", "") if game_code == "SDGS" else endpoint
+        if game_code == "SDGS" and version >= 110:
+            endpoint = endpoint.replace("C3Exp", "")
+        elif game_code == "SDGS" and version < 110:
+            endpoint = endpoint.replace("Exp", "")
+        else:
+            endpoint = endpoint
+
         func_to_find = "handle_" + inflection.underscore(endpoint) + "_request"
         handler_cls = self.versions[internal_ver](self.core_cfg, self.game_cfg)
 

titles/chuni/new.py

@@ -339,10 +339,10 @@ class ChuniNew(ChuniBase):
         }
 
     async def handle_cm_get_user_character_api_request(self, data: Dict) -> Dict:
-        return super().handle_get_user_character_api_request(data)
+        return await super().handle_get_user_character_api_request(data)
 
     async def handle_cm_get_user_item_api_request(self, data: Dict) -> Dict:
-        return super().handle_get_user_item_api_request(data)
+        return await super().handle_get_user_item_api_request(data)
 
     async def handle_roll_gacha_api_request(self, data: Dict) -> Dict:
         """

titles/chuni/schema/profile.py

@@ -1,10 +1,9 @@
 from typing import Dict, List, Optional
-from sqlalchemy import Table, Column, UniqueConstraint, PrimaryKeyConstraint, and_
-from sqlalchemy.types import Integer, String, TIMESTAMP, Boolean, JSON, BigInteger
-from sqlalchemy.engine.base import Connection
+from sqlalchemy import Table, Column, UniqueConstraint, and_
+from sqlalchemy.types import Integer, String, Boolean, JSON, BigInteger
 from sqlalchemy.schema import ForeignKey
 from sqlalchemy.engine import Row
-from sqlalchemy.sql import func, select
+from sqlalchemy.sql import select, delete
 from sqlalchemy.dialects.mysql import insert
 
 from core.data.schema import BaseData, metadata
@@ -393,8 +392,39 @@ team = Table(
     mysql_charset="utf8mb4",
 )
 
+rating = Table(
+    "chuni_profile_rating",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column("version", Integer, nullable=False),
+    Column("type", String(255), nullable=False),
+    Column("index", Integer, nullable=False),
+    Column("musicId", Integer),
+    Column("difficultId", Integer),
+    Column("romVersionCode", Integer),
+    Column("score", Integer),
+    UniqueConstraint("user", "version", "type", "index", name="chuni_profile_rating_best_uk"),
+    mysql_charset="utf8mb4",
+)
+
 
 class ChuniProfileData(BaseData):
+    async def update_name(self, user_id: int, new_name: str) -> bool:
+        sql = profile.update(profile.c.user == user_id).values(
+            userName=new_name
+        )
+        result = await self.execute(sql)
+
+        if result is None:
+            self.logger.warning(f"Failed to set user {user_id} name to {new_name}")
+            return False
+        return True
+
     async def put_profile_data(
         self, aime_id: int, version: int, profile_data: Dict
     ) -> Optional[int]:
@@ -703,3 +733,50 @@ class ChuniProfileData(BaseData):
         return {
             "total_play_count": total_play_count
         }
+    
+    async def put_profile_rating(
+        self,
+        aime_id: int,
+        version: int,
+        rating_type: str,
+        rating_data: List[Dict],
+    ):
+        inserted_values = [
+            {"user": aime_id, "version": version, "type": rating_type, "index": i, **x}
+            for (i, x) in enumerate(rating_data)
+        ]
+        sql = insert(rating).values(inserted_values)
+        update_dict = {x.name: x for x in sql.inserted if x.name != "id"}
+        sql = sql.on_duplicate_key_update(**update_dict)
+        result = await self.execute(sql)
+
+        if result is None:
+            self.logger.warn(
+                f"put_profile_rating: Could not insert {rating_type}, aime_id: {aime_id}",
+            )
+            return
+        
+        return result.lastrowid
+
+    async def get_profile_rating(self, aime_id: int, version: int) -> Optional[List[Row]]:
+        sql = select(rating).where(and_(
+                rating.c.user == aime_id,
+                rating.c.version <= version,
+            ))
+
+        result = await self.execute(sql)
+        if result is None:
+            self.logger.warning(f"Rating of user {aime_id}, version {version} was None")
+            return None
+        return result.fetchall()
+
+    async def get_all_profile_versions(self, aime_id: int) -> Optional[List[Row]]:
+        sql = select([profile.c.version]).where(profile.c.user == aime_id)
+        result = await self.execute(sql)
+        if result is None:
+            self.logger.warning(f"user {aime_id}, has no profile")
+            return None
+        else:
+            versions_raw = result.fetchall()
+            versions = [row[0] for row in versions_raw]
+        return sorted(versions, reverse=True)

titles/chuni/schema/score.py

@@ -190,6 +190,23 @@ class ChuniScoreData(BaseData):
             return None
         return result.fetchall()
 
+    async def get_playlogs_limited(self, aime_id: int, index: int, count: int) -> Optional[Row]:
+        sql = select(playlog).where(playlog.c.user == aime_id).order_by(playlog.c.id.desc()).limit(count).offset(index * count)
+
+        result = await self.execute(sql)
+        if result is None:
+            self.logger.warning(f" aime_id {aime_id} has no playlog ")
+            return None
+        return result.fetchall()
+
+    async def get_user_playlogs_count(self, aime_id: int) -> Optional[Row]:
+        sql = select(func.count()).where(playlog.c.user == aime_id)
+        result = await self.execute(sql)
+        if result is None:
+            self.logger.warning(f" aime_id {aime_id} has no playlog ")
+            return None
+        return result.scalar()
+
     async def put_playlog(self, aime_id: int, playlog_data: Dict, version: int) -> Optional[int]:
         # Calculate the ROM version that should be inserted into the DB, based on the version of the ggame being inserted
         # We only need from Version 10 (Plost) and back, as newer versions include romVersion in their upsert

titles/chuni/templates/chuni_header.jinja

@@ -0,0 +1,24 @@
+<div class="chuni-header">
+  <h1>Chunithm</h1>
+  <ul class="chuni-navi">
+    <li><a class="nav-link" href="/game/chuni">PROFILE</a></li>
+    <li><a class="nav-link" href="/game/chuni/rating">RATING</a></li>
+    <li><a class="nav-link" href="/game/chuni/playlog">RECORD</a></li>
+    <li><a class="nav-link" href="/game/chuni/musics">MUSICS</a></li>
+    <li><a class="nav-link" href="/game/chuni/userbox">USER BOX</a></li>
+  </ul>
+</div>
+<script>
+  $(document).ready(function () {
+    var currentPath = window.location.pathname;
+    if (currentPath === '/game/chuni/') {
+      $('.nav-link[href="/game/chuni"]').addClass('active');
+    } else if (currentPath.startsWith('/game/chuni/playlog')) {
+      $('.nav-link[href="/game/chuni/playlog"]').addClass('active');
+    } else if (currentPath.startsWith('/game/chuni/rating')) {
+      $('.nav-link[href="/game/chuni/rating"]').addClass('active');
+    } else if (currentPath.startsWith('/game/chuni/musics')) {
+      $('.nav-link[href="/game/chuni/musics"]').addClass('active');
+    }
+  });
+</script>

titles/chuni/templates/chuni_index.jinja

@@ -0,0 +1,150 @@
+{% extends "core/templates/index.jinja" %}
+{% block content %}
+<style>
+  {% include 'titles/chuni/templates/css/chuni_style.css' %}
+</style>
+<div class="container">
+  {% include 'titles/chuni/templates/chuni_header.jinja' %}
+  {% if profile is defined and profile is not none and profile|length > 0 %}
+  <div class="row">
+    <div class="col-lg-8 m-auto mt-3">
+      <div class="card bg-card rounded">
+        <table class="table-large table-rowdistinct">
+          <caption align="top">OVERVIEW</caption>
+          <tr>
+            <th>{{ profile.userName }}</th>
+            <th>
+              <button type="button" class="btn btn-primary btn-sm" data-bs-toggle="modal" data-bs-target="#name_change">Edit</button>
+            </th>
+          </tr>
+          <tr>
+            <td>version:</td>
+            <td>
+              <select name="version" id="version" onChange="changeVersion(this)">
+                {% for ver in versions %}
+                {% if ver == cur_version %}
+                <option value="{{ ver }}" selected>{{ version_list[ver] }}</option>
+                {% else %}
+                <option value="{{ ver }}">{{ version_list[ver] }}</option>
+                {% endif %}
+                {% endfor %}
+              </select>
+              {% if versions | length > 1 %}
+              <p style="margin-block-end: 0;">You have {{ versions | length }} versions.</p>
+              {% endif %}
+            </td>
+          </tr>
+          <tr>
+            <td>Level:</td>
+            <td>{{ profile.level }}</td>
+          </tr>
+          <tr>
+            <td>Rating:</td>
+            <td>
+              <span class="{% if profile.playerRating >= 1600 %}rainbow{% elif profile.playerRating < 1600 and profile.playerRating >= 1525 %}platinum{% elif profile.playerRating < 1525 and profile.playerRating >=1500 %}platinum{% endif %}">
+                {{ profile.playerRating|float/100 }}
+              </span>
+              <span>
+                (highest: {{ profile.highestRating|float/100 }})
+              </span>
+            </td>
+          </tr>
+          <tr>
+            <td>Over Power:</td>
+            <td>{{ profile.overPowerPoint|float/100 }}({{ profile.overPowerRate|float/100 }})</td>
+          </tr>
+          <tr>
+            <td>Current Point:</td>
+            <td>{{ profile.point }}</td>
+          </tr>
+          <tr>
+            <td>Total Point:</td>
+            <td>{{ profile.totalPoint }}</td>
+          </tr>
+          <tr>
+            <td>Play Counts:</td>
+            <td>{{ profile.playCount }}</td>
+          </tr>
+          <tr>
+            <td>Last Play Date:</td>
+            <td>{{ profile.lastPlayDate }}</td>
+          </tr>
+        </table>
+      </div>
+    </div>
+    <div class="col-lg-8 m-auto mt-3"> 
+      <div class="card bg-card rounded">
+          <table class="table-large table-rowdistinct">
+            <caption align="top">SCORE</caption>
+            <tr>
+              <td>Total High Score:</td>
+              <td>{{ profile.totalHiScore }}</td>
+            </tr>
+            <tr>
+              <td>Total Basic High Score:</td>
+              <td>{{ profile.totalBasicHighScore }}</td>
+            </tr>
+            <tr>
+              <td>Total Advanced High Score:</td>
+              <td>{{ profile.totalAdvancedHighScore }}</td>
+            </tr>
+            <tr>
+              <td>Total Expert High Score:</td>
+              <td>{{ profile.totalExpertHighScore }}</td>
+            </tr>
+            <tr>
+              <td>Total Master High Score:</td>
+              <td>{{ profile.totalMasterHighScore }}</td>
+            </tr>
+            <tr>
+              <td>Total Ultima High Score :</td>
+              <td>{{ profile.totalUltimaHighScore }}</td>
+            </tr>
+          </table>
+      </div>
+    </div>
+  </div>
+  {% if error is defined %}
+  {% include "core/templates/widgets/err_banner.jinja" %}
+  {% endif %}
+  {% elif sesh is defined and sesh is not none and sesh.user_id > 0 %}
+  No profile information found for this account.
+  {% else %}
+  Login to view profile information.
+  {% endif %}
+</div>
+<div class="modal fade" id="name_change" tabindex="-1" aria-labelledby="name_change_label" data-bs-theme="dark"
+  aria-hidden="true">
+  <div class="modal-dialog modal-dialog-centered">
+    <div class="modal-content">
+      <div class="modal-header">
+        <h5 class="modal-title">Name change</h5>
+      </div>
+      <div class="modal-body">
+        <form id="new_name_form" action="/game/chuni/update.name" method="post" style="outline: 0;">
+          <label class="form-label" for="new_name">new name:</label>
+          <input class="form-control" aria-describedby="newNameHelp" form="new_name_form" id="new_name"
+            name="new_name" maxlength="14" type="text" required>
+          <div id="newNameHelp" class="form-text">name must be full-width character string.
+          </div>
+        </form>
+      </div>
+      <div class="modal-footer">
+        <input type=submit class="btn btn-primary" type="button" form="new_name_form">
+        <button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Close</button>
+      </div>
+    </div>
+  </div>
+</div>
+<script>
+  function changeVersion(sel) {
+    $.post("/game/chuni/version.change", { version: sel.value })
+      .done(function (data) {
+        location.reload();
+      })
+      .fail(function () {
+        alert("Failed to update version.");
+      });
+  }
+</script>
+{% endblock content %}

titles/chuni/templates/chuni_playlog.jinja

@@ -0,0 +1,184 @@
+{% extends "core/templates/index.jinja" %}
+{% block content %}
+<style>
+  {% include 'titles/chuni/templates/css/chuni_style.css' %}
+</style>
+<div class="container">
+  {% include 'titles/chuni/templates/chuni_header.jinja' %}
+  {% if playlog is defined and playlog is not none %}
+  <div class="row">
+    <h4 style="text-align: center;">Playlog counts: {{ playlog_count }}</h4>
+    {% set rankName = ['D', 'C', 'B', 'BB', 'BBB', 'A', 'AA', 'AAA', 'S', 'S+', 'SS', 'SS+', 'SSS', 'SSS+'] %}
+    {% set difficultyName = ['normal', 'hard', 'expert', 'master', 'ultimate'] %}
+    {% for record in playlog %}
+    <div class="col-lg-6 mt-3">
+      <div class="card bg-card rounded card-hover">
+        <div class="card-header row">
+          <div class="col-8 scrolling-text">
+            <h5 class="card-text"> {{ record.title }} </h5>
+            <br>
+            <h6 class="card-text"> {{ record.artist }} </h6>
+          </div>
+          <div class="col-4">
+            <h6 class="card-text">{{ record.raw.userPlayDate }}</h6>
+            <h6 class="card-text">TRACK {{ record.raw.track }}</h6>
+          </div>
+        </div>
+        <div class="card-body row">
+          <div class="col-3" style="text-align: center;">
+            <h4 class="card-text">{{ record.raw.score }}</h4>
+            <h2>{{ rankName[record.raw.rank] }}</h2>
+            <h6
+              class="{% if record.raw.level == 0 %}normal{% elif record.raw.level == 1 %}advanced{% elif record.raw.level == 2 %}expert{% elif record.raw.level == 3 %}master{% elif record.raw.level == 4 %}ultima{% endif %}">
+              {{ difficultyName[record.raw.level] }}&nbsp&nbsp{{ record.difficultyNum }}
+            </h6>
+          </div>
+          <div class="col-6" style="text-align: center;">
+            <table class="table-small table-rowdistinc">
+              <tr>
+                <td>JUSTICE CRITIAL</td>
+                <td>
+                  {{ record.raw.judgeCritical + record.raw.judgeHeaven }}
+                </td>
+              </tr>
+              <tr>
+                <td>JUSTICE</td>
+                <td>
+                  {{ record.raw.judgeJustice }}
+                </td>
+              </tr>
+              <tr>
+                <td>ATTACK</td>
+                <td>
+                  {{ record.raw.judgeAttack }}
+                </td>
+              </tr>
+              <tr>
+                <td>MISS</td>
+                <td>
+                  {{ record.raw.judgeGuilty }}
+                </td>
+              </tr>
+            </table>
+          </div>
+          <div class="col-3" style="text-align: center;">
+            {%if record.raw.isFullCombo == 1 %}
+            <h6>FULL COMBO</h6>
+            {% endif %}
+            {%if record.raw.isAllJustice == 1 %}
+            <h6>ALL JUSTICE</h6>
+            {% endif %}
+            {%if record.raw.isNewRecord == 1 %}
+            <h6>NEW RECORD</h6>
+            {% endif %}
+            {%if record.raw.fullChainKind > 0 %}
+            <h6>FULL CHAIN</h6>
+            {% endif %}
+          </div>
+        </div>
+      </div>
+    </div>
+    {% endfor %}
+  </div>
+  {% set playlog_pages = playlog_count // 20 + 1 %}
+  {% elif sesh is defined and sesh is not none and sesh.user_id > 0 %}
+  No Playlog information found for this account.
+  {% else %}
+  Login to view profile information.
+  {% endif %}
+</div>
+<footer class="navbar-fixed-bottom">
+  <nav aria-label="Playlog page navigation">
+    <ul class="pagination justify-content-center mt-3">
+      <li class="page-item"><a id="prev_page" class="page-link" href="#">Previous</a></li>
+      <li class="page-item"><a id="first_page" class="page-link" href="/game/chuni/playlog/">1</a></li>
+      <li class="page-item"><a id="prev_3_page" class="page-link" href="">...</a></li>
+      <li class="page-item"><a id="front_page" class="page-link" href="">2</a></li>
+      <li class="page-item"><a id="cur_page" class="page-link active" href="">3</a></li>
+      <li class="page-item"><a id="back_page" class="page-link" href="">4</a></li>
+      <li class="page-item"><a id="next_3_page" class="page-link" href="">...</a></li>
+      <li class="page-item"><a id="last_page" class="page-link" href="/game/chuni/playlog/{{ playlog_pages }}">{{
+          playlog_pages }}</a></li>
+      <li class="page-item"><a id="next_page" class="page-link" href="#">Next</a></li>
+      &nbsp
+    </ul>
+  </nav>
+  <div class="row">
+    <div class="col-5"></div>
+    <div class="col-2">
+      <div class="input-group rounded">
+        <input id="page_input" type="text" class="form-control" placeholder="go to page">
+        <span class="input-group-btn">
+          <button id="go_button" class="btn btn-light" type="button">
+            Go!
+          </button>
+        </span>
+      </div>
+    </div>
+    <div class="col-5"></div>
+  </div>
+</footer>
+
+<script>
+  $(document).ready(function () {
+    $('.scrolling-text p, .scrolling-text h1, .scrolling-text h2, .scrolling-text h3, .scrolling-text h4, .scrolling-text h5, .scrolling-text h6').each(function () {
+      var parentWidth = $(this).parent().width();
+      var elementWidth = $(this).outerWidth();
+      var elementWidthWithPadding = $(this).outerWidth(true);
+
+      if (elementWidthWithPadding > parentWidth) {
+        $(this).addClass('scrolling');
+      }
+    });
+
+    var currentUrl = window.location.pathname;
+    var currentPage = parseInt(currentUrl.split('/').pop());
+    var rootUrl = '/game/chuni/playlog/';
+    var playlogPages = {{ playlog_pages }};
+  if (Number.isNaN(currentPage)) {
+    currentPage = 1;
+  }
+  $('#cur_page').text(currentPage);
+  $('#prev_page').attr('href', rootUrl + (currentPage - 1))
+  $('#next_page').attr('href', rootUrl + (currentPage + 1))
+  $('#front_page').attr('href', rootUrl + (currentPage - 1))
+  $('#front_page').text(currentPage - 1);
+  $('#back_page').attr('href', rootUrl + (currentPage + 1))
+  $('#back_page').text(currentPage + 1);
+  $('#prev_3_page').attr('href', rootUrl + (currentPage - 3))
+  $('#next_3_page').attr('href', rootUrl + (currentPage + 3))
+  if ((currentPage - 1) < 3) {
+    $('#prev_3_page').hide();
+    if ((currentPage - 1) < 2) {
+      $('#front_page').hide();
+      if (currentPage === 1) {
+        $('#first_page').hide();
+        $('#prev_page').addClass('disabled');
+      }
+    }
+  }
+  if ((playlogPages - currentPage) < 3) {
+    $('#next_3_page').hide();
+    if ((playlogPages - currentPage) < 2) {
+      $('#back_page').hide();
+      if (currentPage === playlogPages) {
+        $('#last_page').hide();
+        $('#next_page').addClass('disabled');
+      }
+    }
+  }
+
+  $('#go_button').click(function () {
+    var pageNumber = parseInt($('#page_input').val());
+
+    if (!Number.isNaN(pageNumber) && pageNumber <= playlogPages && pageNumber >= 0) {
+      var url = '/game/chuni/playlog/' + pageNumber;
+      window.location.href = url;
+    } else {
+      $('#page_input').val('');
+      $('#page_input').attr('placeholder', 'invalid input!');
+    }
+  });
+  });
+</script>
+{% endblock content %}

titles/chuni/templates/chuni_rating.jinja

@@ -0,0 +1,79 @@
+{% extends "core/templates/index.jinja" %}
+{% block content %}
+<style>
+  {% include 'titles/chuni/templates/css/chuni_style.css' %}
+</style>
+<div class="container">
+  {% include 'titles/chuni/templates/chuni_header.jinja' %}
+  {% if profile is defined and profile is not none and profile.id > 0 %}
+  <h4 style="text-align: center;">Rating: {{ profile.playerRating|float/100 }}&nbsp&nbsp&nbsp&nbspPlayer Counts: {{
+    profile.playCount }}</h4>
+  <div class="row">
+    {% if hot_list %}
+    <div class="col-lg-6 mt-3">
+      <div class="card  bg-card rounded">
+        <table class="table-large table-rowdistinct">
+          <caption align="top">Recent 10</caption>
+          <tr>
+            <th>Music</th>
+            <th>Difficulty</th>
+            <th>Score</th>
+            <th>Rating</th>
+          </tr>
+          {% for row in hot_list %}
+          <tr>
+            <td>{{ row.title }}</td>
+            <td
+              class="{% if row.difficultId == 0 %}basic{% elif row.difficultId == 1 %}{% elif row.difficultId == 2 %}expert{% elif row.difficultId == 3 %}master{% else %}{% endif %}">
+              {{ row.level }}
+            </td>
+            <td>{{ row.score }}</td>
+            <td class="{% if row.song_rating >= 16 %}rainbow{% endif %}">
+              {{ row.song_rating }}
+            </td>
+          </tr>
+          {% endfor %}
+        </table>
+      </div>
+    </div>
+    {% else %}
+    <p>No r10 found</p>
+    {% endif %}
+    {% if base_list %}
+    <div class="col-lg-6 mt-3">
+      <div class="card bg-card rounded">
+        <table class="table-large table-rowdistinct">
+          <caption align="top">Best 30</caption>
+          <tr>
+            <th>Music</th>
+            <th>Difficulty</th>
+            <th>Score</th>
+            <th>Rating</th>
+          </tr>
+          {% for row in base_list %}
+          <tr>
+            <td>{{ row.title }}</td>
+            <td
+              class="{% if row.difficultId == 0 %}normal{% elif row.difficultId == 1 %}hard{% elif row.difficultId == 2 %}expert{% elif row.difficultId == 3 %}master{% else %}{% endif %}">
+              {{ row.level }}
+            </td>
+            <td>{{ row.score }}</td>
+            <td class="{% if row.song_rating >= 16 %}rainbow{% endif %}">
+              {{ row.song_rating }}
+            </td>
+          </tr>
+          {% endfor %}
+        </table>
+      </div>
+    </div>
+    {% else %}
+    <p>No b30 found</p>
+    {% endif %}
+  </div>
+  {% elif sesh is defined and sesh is not none and sesh.user_id > 0 %}
+  No profile information found for this account.
+  {% else %}
+  Login to view profile information.
+  {% endif %}
+</div>
+{% endblock content %}

titles/chuni/templates/css/chuni_style.css

@@ -0,0 +1,195 @@
+.chuni-header {
+    text-align: center;
+}
+
+ul.chuni-navi {
+    list-style-type: none;
+    padding: 0;
+    overflow: hidden;
+    background-color: #333;
+    text-align: center;
+    display: inline-block;
+}
+
+ul.chuni-navi li {
+    display: inline-block;
+}
+
+ul.chuni-navi li a {
+    display: block;
+    color: white;
+    text-align: center;
+    padding: 14px 16px;
+    text-decoration: none;
+}
+
+ul.chuni-navi li a:hover:not(.active) {
+    background-color: #111;
+}
+
+ul.chuni-navi li a.active {
+    background-color: #4CAF50;
+}
+
+ul.chuni-navi li.right {
+    float: right;
+}
+
+@media screen and (max-width: 600px) {
+
+    ul.chuni-navi li.right,
+    ul.chuni-navi li {
+        float: none;
+        display: block;
+        text-align: center;
+    }
+}
+
+table {
+    border-collapse: collapse;
+    border-spacing: 0;
+    border-collapse: separate;
+    overflow: hidden;
+    background-color: #555555;
+
+}
+
+th, td {
+    text-align: left;
+    border: none;
+
+}
+
+th {
+    color: white;
+}
+
+.table-rowdistinct tr:nth-child(even) {
+    background-color: #303030;
+}
+
+.table-rowdistinct tr:nth-child(odd) {
+    background-color: #555555;
+}
+
+caption {
+    text-align: center;
+    color: white;
+    font-size: 18px;
+    font-weight: bold;
+}
+
+.table-large {
+    margin: 16px;
+}
+
+.table-large th,
+.table-large td {
+    padding: 8px;
+}
+
+.table-small {
+    width: 100%;
+    margin: 4px;
+}
+
+.table-small th,
+.table-small td {
+    padding: 2px;
+}
+
+.bg-card {
+    background-color: #555555;
+}
+
+.card-hover {
+    transition: all 0.2s ease-in-out;
+}
+
+.card-hover:hover {
+    transform: scale(1.02);
+}
+
+.normal {
+    color: #28a745;
+    font-weight: bold;
+}
+
+.hard {
+    color: #ffc107;
+
+    font-weight: bold;
+}
+
+.expert {
+    color: #dc3545;
+    font-weight: bold;
+}
+
+.master {
+    color: #dd09e8;
+    font-weight: bold;
+}
+
+.ultimate {
+    color: #000000;
+    font-weight: bold;
+}
+
+.score {
+    color: #ffffff;
+    font-weight: bold;
+}
+
+.rainbow {
+    background: linear-gradient(to right, red, yellow, lime, aqua, blue, fuchsia) 0 / 5em;
+    background-clip: text;
+    -webkit-background-clip: text;
+    -webkit-text-fill-color: transparent;
+    font-weight: bold;
+}
+
+.platinum {
+    color: #FFFF00;
+    font-weight: bold;
+}
+
+.gold {
+    color: #FFFF00;
+    font-weight: bold;
+}
+
+.scrolling-text {
+    overflow: hidden;
+}
+
+.scrolling-text p {
+    white-space: nowrap;
+    display: inline-block;
+
+}
+
+.scrolling-text h6 {
+    white-space: nowrap;
+    display: inline-block;
+
+}
+
+.scrolling-text h5 {
+    white-space: nowrap;
+    display: inline-block;
+
+}
+
+.scrolling {
+    animation: scroll 10s linear infinite;
+}
+
+@keyframes scroll {
+    0% {
+        transform: translateX(100%);
+    }
+    100% {
+        transform: translateX(-100%);
+    }
+}

titles/cm/base.py

@@ -51,7 +51,7 @@ class CardMakerBase:
                 {
                     "modelKind": 1,
                     "type": 1,
-                    "titleUri": f"{uri}/{self._parse_int_ver(games_ver['maimai'])}/Maimai2Servlet/",
+                    "titleUri": f"{uri}/SDEZ/{self._parse_int_ver(games_ver['maimai'])}/Maimai2Servlet/",
                 },
                 # ONGEKI
                 {

titles/cm/index.py

@@ -74,7 +74,7 @@ class CardMakerServlet(BaseServlet):
     
     def get_routes(self) -> List[Route]:
         return [
-            Route("/SDED/{version:int}/{endpoint:str}", self.render_POST)
+            Route("/SDED/{version:int}/{endpoint:str}", self.render_POST, methods=['POST'])
         ]
     
     async def render_POST(self, request: Request) -> bytes:
@@ -117,7 +117,7 @@ class CardMakerServlet(BaseServlet):
 
         try:
             handler = getattr(self.versions[internal_ver], func_to_find)
-            resp = handler(req_data)
+            resp = await handler(req_data)
 
         except Exception as e:
             self.logger.error(f"Error handling v{version} method {endpoint} - {e}")

titles/cm/read.py

@@ -59,10 +59,10 @@ class CardMakerReader(BaseReader):
         if self.bin_dir is not None:
             data_dir = self._get_card_maker_directory(self.bin_dir)
 
-            self.read_chuni_card(f"{data_dir}/CHU/Data/A000/card")
-            self.read_chuni_gacha(f"{data_dir}/CHU/Data/A000/gacha")
+            await self.read_chuni_card(f"{data_dir}/CHU/Data/A000/card")
+            await self.read_chuni_gacha(f"{data_dir}/CHU/Data/A000/gacha")
 
-            self.read_mai2_card(f"{data_dir}/MAI/Data/A000/card")
+            await self.read_mai2_card(f"{data_dir}/MAI/Data/A000/card")
             for file, func in static_datas.items():
                 if os.path.exists(f"{self.bin_dir}/MU3/{file}"):
                     read_csv = getattr(CardMakerReader, func)
@@ -206,6 +206,7 @@ class CardMakerReader(BaseReader):
             "1.25": Mai2Constants.VER_MAIMAI_DX_UNIVERSE_PLUS,
             "1.30": Mai2Constants.VER_MAIMAI_DX_FESTIVAL,
             "1.35": Mai2Constants.VER_MAIMAI_DX_FESTIVAL_PLUS,
+            "1.40": Mai2Constants.VER_MAIMAI_DX_BUDDIES,
         }
 
         for root, dirs, files in os.walk(base_dir):
@@ -225,12 +226,6 @@ class CardMakerReader(BaseReader):
                             True if troot.find("disable").text == "false" else False
                         )
 
-                        # check if a date is part of the name and disable the
-                        # card if it is
-                        enabled = (
-                            False if re.search(r"\d{2}/\d{2}/\d{2}", name) else enabled
-                        )
-
                         await self.mai2_data.static.put_card(
                             version, card_id, name, enabled=enabled
                         )

titles/cxb/data/rss1/Shop/ShopList_Sale.csv

@@ -1,3 +0,0 @@
-saleID.,<EFBFBD>J<EFBFBD>n<EFBFBD><EFBFBD>,<EFBFBD>I<EFBFBD><EFBFBD><EFBFBD><EFBFBD>,ShopID,Price,    
-0,1411696799,1443236400,0,7000,
-1,1411783199,1443322800,1,7000,

titles/cxb/data/rss1/Shop/ShopList_SkinBg.csv

@@ -1,4 +0,0 @@
-shopID.,<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>pNo.,Ver.,<EFBFBD>o<EFBFBD><EFBFBD><EFBFBD>t<EFBFBD><EFBFBD><EFBFBD>O,<EFBFBD>o<EFBFBD><EFBFBD><EFBFBD>t<EFBFBD><EFBFBD><EFBFBD>O<EFBFBD>Q<EFBFBD><EFBFBD>ID,<EFBFBD><EFBFBD><EFBFBD><EFBFBD>,<EFBFBD>o<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>,<EFBFBD><EFBFBD><EFBFBD>Ŏ<EFBFBD><EFBFBD><EFBFBD>,ItemCode,<EFBFBD><EFBFBD><EFBFBD>i,<EFBFBD>\<5C><><EFBFBD>^<5E>C<EFBFBD>v,Text,Type,Value(op),Value,<EFBFBD>Ώۋ<EFBFBD>,Difficulty(op),Difficulty,Level(op),Level,Grade(Op),Grade,GaugeType(op),GaugeType,HS(op)i,HS,APP,DAP,F-V,F-H,FullCombo,Combo(op),Combo,ClearRate(op),ClearRate,<EFBFBD>v<EFBFBD><EFBFBD><EFBFBD>C<EFBFBD><EFBFBD>,<EFBFBD>n<EFBFBD><EFBFBD>,
-3000,1,1.00.00,1,-1,-,1411697520.0288,1443233520.0288,skb0000,10,1,MASTER<EFBFBD>ȏ<EFBFBD><EFBFBD><EFBFBD>2<EFBFBD><EFBFBD>S+<2B>ȏ<EFBFBD><C88F>t<EFBFBD><74><EFBFBD>R<EFBFBD><52><EFBFBD>{<7B>N<EFBFBD><4E><EFBFBD>A<EFBFBD><41><EFBFBD><EFBFBD><EFBFBD>B,1,1,1,-,1,2,-1,-1,1,1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-3001,2,1.00.00,1,-1,-,1411697520.0288,1443233520.0288,skb0001,10,1,Next Frontier (Master<65>j<EFBFBD><6A><EFBFBD>N<EFBFBD><4E><EFBFBD>A,0,-1,-1,bleeze,1,2,-1,-1,1,1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-3002,3,1.00.00,1,-1,-,1412103600.0288,1443639598.992,skb0002,10,2,Master<EFBFBD>ȏ<EFBFBD><EFBFBD><EFBFBD>1<EFBFBD>Ȃ<EFBFBD>S+<2B>ȏ<EFBFBD><C88F>ŃN<C583><4E><EFBFBD>A<EFBFBD><41><EFBFBD><EFBFBD><EFBFBD>B,1,1,1,-,1,2,-1,-1,1,1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,1,

titles/cxb/data/rss1/Shop/ShopList_SkinEffect.csv

@@ -1,11 +0,0 @@
-shopID.,<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>pNo.,Ver.,<EFBFBD>o<EFBFBD><EFBFBD><EFBFBD>t<EFBFBD><EFBFBD><EFBFBD>O,<EFBFBD>o<EFBFBD><EFBFBD><EFBFBD>t<EFBFBD><EFBFBD><EFBFBD>O<EFBFBD>Q<EFBFBD><EFBFBD>ID,<EFBFBD><EFBFBD><EFBFBD><EFBFBD>,<EFBFBD>o<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>,<EFBFBD><EFBFBD><EFBFBD>Ŏ<EFBFBD><EFBFBD><EFBFBD>,ItemCode,<EFBFBD><EFBFBD><EFBFBD>i,<EFBFBD>\<5C><><EFBFBD>^<5E>C<EFBFBD>v,Text,Type,Value(op),Value,<EFBFBD>Ώۋ<EFBFBD>,Difficulty(op),Difficulty,Level(op),Level,Grade(Op),Grade,GaugeType(op),GaugeType,HS(op)i,HS,APP,DAP,F-V,F-H,FullCombo,Combo(op),Combo,ClearRate(op),ClearRate,<EFBFBD>v<EFBFBD><EFBFBD><EFBFBD>C<EFBFBD><EFBFBD>,<EFBFBD>n<EFBFBD><EFBFBD>,
-5000,1,10000,1,-1,-,1411697520,1443233520,ske0000,10,1,MASTER<EFBFBD>ȏ<EFBFBD><EFBFBD>2<EFBFBD><EFBFBD>S+<2B>ȏ<EFBFBD>t<EFBFBD><74><EFBFBD>R<EFBFBD><52><EFBFBD>{<7B>N<EFBFBD><4E><EFBFBD>A<EFBFBD><41><EFBFBD><EFBFBD>B,1,1,1,-,1,2,-1,-1,1,1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-5001,2,10000,1,-1,-,1411697520,1443233520,ske0001,10,1,Next Frontier (Master<65>j<EFBFBD><6A><EFBFBD>N<EFBFBD><4E><EFBFBD>A,0,-1,-1,megaro,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-5002,3,10000,1,-1,-,1412103600,1443639598,ske0002,10,2,Master<EFBFBD>ȏ<EFBFBD><EFBFBD>1<EFBFBD>Ȃ<EFBFBD>S+<2B>ȏ<EFBFBD>ŃN<C583><4E><EFBFBD>A<EFBFBD><41><EFBFBD><EFBFBD>B,1,1,1,-,1,2,-1,-1,1,1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,1,
-5003,4,10000,1,-1,-,1412103600,1443639598,ske0003,10,0,Master<EFBFBD>ȏ<EFBFBD><EFBFBD>1<EFBFBD>Ȃ<EFBFBD>S+<2B>ȏ<EFBFBD>ŃN<C583><4E><EFBFBD>A<EFBFBD><41><EFBFBD><EFBFBD>B,1,1,1,-,1,2,-1,-1,1,1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-5004,5,10000,1,-1,-,1412103600,1443639598,ske0004,10,2,2<EFBFBD>ȃN<EFBFBD><EFBFBD><EFBFBD>A,1,1,2,-,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-5005,5,10000,1,-1,-,1412103600,1443639598,ske0005,10,2,3<EFBFBD>ȃN<EFBFBD><EFBFBD><EFBFBD>A,1,1,3,-,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-5006,5,10000,1,-1,-,1412103600,1443639598,ske0006,10,2,4<EFBFBD>ȃN<EFBFBD><EFBFBD><EFBFBD>A,1,1,4,-,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-5007,5,10000,1,-1,-,1412103600,1443639598,ske0007,10,2,5<EFBFBD>ȃN<EFBFBD><EFBFBD><EFBFBD>A,1,1,5,-,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-5008,5,10000,1,-1,-,1412103600,1443639598,ske0008,10,2,6<EFBFBD>ȃN<EFBFBD><EFBFBD><EFBFBD>A,1,1,6,-,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-5009,5,10000,1,-1,-,1412103600,1443639598,ske0009,10,2,7<EFBFBD>ȃN<EFBFBD><EFBFBD><EFBFBD>A,1,1,7,-,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,

titles/cxb/data/rss1/Shop/ShopList_SkinNotes.csv

@@ -1,6 +0,0 @@
-shopID.,<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>pNo.,Ver.,<EFBFBD>o<EFBFBD><EFBFBD><EFBFBD>t<EFBFBD><EFBFBD><EFBFBD>O,<EFBFBD>o<EFBFBD><EFBFBD><EFBFBD>t<EFBFBD><EFBFBD><EFBFBD>O<EFBFBD>Q<EFBFBD><EFBFBD>ID,<EFBFBD><EFBFBD><EFBFBD><EFBFBD>,<EFBFBD>o<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>,<EFBFBD><EFBFBD><EFBFBD>Ŏ<EFBFBD><EFBFBD><EFBFBD>,ItemCode,<EFBFBD><EFBFBD><EFBFBD>i,<EFBFBD>\<5C><><EFBFBD>^<5E>C<EFBFBD>v,Text,Type,Value(op),Value,<EFBFBD>Ώۋ<EFBFBD>,Difficulty(op),Difficulty,Level(op),Level,Grade(Op),Grade,GaugeType(op),GaugeType,HS(op)i,HS,APP,DAP,F-V,F-H,FullCombo,Combo(op),Combo,ClearRate(op),ClearRate,<EFBFBD>v<EFBFBD><EFBFBD><EFBFBD>C<EFBFBD><EFBFBD>,<EFBFBD>n<EFBFBD><EFBFBD>,
-4000,1,10000,1,-1,-,1411697520,4096483201,skt0000,10,1,MASTER<EFBFBD>ȏ<EFBFBD><EFBFBD>2<EFBFBD><EFBFBD>S+<2B>ȏ<EFBFBD>t<EFBFBD><74><EFBFBD>R<EFBFBD><52><EFBFBD>{<7B>N<EFBFBD><4E><EFBFBD>A<EFBFBD><41><EFBFBD><EFBFBD>B,1,1,1,-,1,2,-1,-1,1,1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-4001,2,10000,1,-1,-,1411697520,4096483201,skt0001,10,1,Next Frontier (Master<65>j<EFBFBD><6A><EFBFBD>N<EFBFBD><4E><EFBFBD>A,0,-1,-1,megaro,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-4002,3,10000,1,-1,-,1412103600,4096483201,skt0002,10,2,Master<EFBFBD>ȏ<EFBFBD><EFBFBD>1<EFBFBD>Ȃ<EFBFBD>S+<2B>ȏ<EFBFBD>ŃN<C583><4E><EFBFBD>A<EFBFBD><41><EFBFBD><EFBFBD>B,1,1,1,-,1,2,-1,-1,1,1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,1,
-4003,4,10000,1,-1,-,1412103600,4096483201,skt0003,10,0,Master<EFBFBD>ȏ<EFBFBD><EFBFBD>1<EFBFBD>Ȃ<EFBFBD>S+<2B>ȏ<EFBFBD>ŃN<C583><4E><EFBFBD>A<EFBFBD><41><EFBFBD><EFBFBD>B,1,1,1,-,1,2,-1,-1,1,1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,
-4004,5,10000,1,-1,-,1412103600,4096483201,skt0004,10,2,aaaaaaaaaaaaaaaaa,1,1,20,-,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,,,

titles/cxb/index.py

@@ -98,7 +98,7 @@ class CxbServlet(BaseServlet):
             t_port = f":{title_port_int}" if title_port_int != 80 else ""
 
         return (
-            f"{proto}://{self.core_cfg.title.hostname}{t_port}",
+            f"{proto}://{self.core_cfg.server.hostname}{t_port}",
             "",
         )
 

titles/cxb/rev.py

@@ -40,6 +40,7 @@ class CxbRev(CxbBase):
                 score_data["slow2"],
                 score_data["fail"],
                 score_data["combo"],
+                score_data["grade"],
             )
             return {"data": True}
         return {"data": True}
@@ -47,7 +48,7 @@ class CxbRev(CxbBase):
     @cached(lifetime=86400)
     async def handle_data_music_list_request(self, data: Dict) -> Dict:
         ret_str = ""
-        with open(r"titles/cxb/data/rss/MusicArchiveList.csv") as music:
+        with open(r"titles/cxb/data/rev/MusicArchiveList.csv") as music:
             lines = music.readlines()
             for line in lines:
                 line_split = line.split(",")
@@ -59,7 +60,7 @@ class CxbRev(CxbBase):
     async def handle_data_item_list_icon_request(self, data: Dict) -> Dict:
         ret_str = "\r\n#ItemListIcon\r\n"
         with open(
-            r"titles/cxb/data/rss/Item/ItemArchiveList_Icon.csv", encoding="utf-8"
+            r"titles/cxb/data/rev/Item/ItemArchiveList_Icon.csv", encoding="shift-jis"
         ) as item:
             lines = item.readlines()
             for line in lines:
@@ -70,7 +71,7 @@ class CxbRev(CxbBase):
     async def handle_data_item_list_skin_notes_request(self, data: Dict) -> Dict:
         ret_str = "\r\n#ItemListSkinNotes\r\n"
         with open(
-            r"titles/cxb/data/rss/Item/ItemArchiveList_SkinNotes.csv", encoding="utf-8"
+            r"titles/cxb/data/rev/Item/ItemArchiveList_SkinNotes.csv", encoding="utf-8"
         ) as item:
             lines = item.readlines()
             for line in lines:
@@ -81,7 +82,7 @@ class CxbRev(CxbBase):
     async def handle_data_item_list_skin_effect_request(self, data: Dict) -> Dict:
         ret_str = "\r\n#ItemListSkinEffect\r\n"
         with open(
-            r"titles/cxb/data/rss/Item/ItemArchiveList_SkinEffect.csv", encoding="utf-8"
+            r"titles/cxb/data/rev/Item/ItemArchiveList_SkinEffect.csv", encoding="utf-8"
         ) as item:
             lines = item.readlines()
             for line in lines:
@@ -92,7 +93,7 @@ class CxbRev(CxbBase):
     async def handle_data_item_list_skin_bg_request(self, data: Dict) -> Dict:
         ret_str = "\r\n#ItemListSkinBg\r\n"
         with open(
-            r"titles/cxb/data/rss/Item/ItemArchiveList_SkinBg.csv", encoding="utf-8"
+            r"titles/cxb/data/rev/Item/ItemArchiveList_SkinBg.csv", encoding="utf-8"
         ) as item:
             lines = item.readlines()
             for line in lines:
@@ -103,7 +104,7 @@ class CxbRev(CxbBase):
     async def handle_data_item_list_title_request(self, data: Dict) -> Dict:
         ret_str = "\r\n#ItemListTitle\r\n"
         with open(
-            r"titles/cxb/data/rss/Item/ItemList_Title.csv", encoding="shift-jis"
+            r"titles/cxb/data/rev/Item/ItemList_Title.csv", encoding="shift-jis"
         ) as item:
             lines = item.readlines()
             for line in lines:
@@ -114,7 +115,7 @@ class CxbRev(CxbBase):
     async def handle_data_shop_list_music_request(self, data: Dict) -> Dict:
         ret_str = "\r\n#ShopListMusic\r\n"
         with open(
-            r"titles/cxb/data/rss/Shop/ShopList_Music.csv", encoding="shift-jis"
+            r"titles/cxb/data/rev/Shop/ShopList_Music.csv", encoding="shift-jis"
         ) as shop:
             lines = shop.readlines()
             for line in lines:
@@ -125,7 +126,7 @@ class CxbRev(CxbBase):
     async def handle_data_shop_list_icon_request(self, data: Dict) -> Dict:
         ret_str = "\r\n#ShopListIcon\r\n"
         with open(
-            r"titles/cxb/data/rss/Shop/ShopList_Icon.csv", encoding="shift-jis"
+            r"titles/cxb/data/rev/Shop/ShopList_Icon.csv", encoding="shift-jis"
         ) as shop:
             lines = shop.readlines()
             for line in lines:
@@ -136,7 +137,7 @@ class CxbRev(CxbBase):
     async def handle_data_shop_list_title_request(self, data: Dict) -> Dict:
         ret_str = "\r\n#ShopListTitle\r\n"
         with open(
-            r"titles/cxb/data/rss/Shop/ShopList_Title.csv", encoding="shift-jis"
+            r"titles/cxb/data/rev/Shop/ShopList_Title.csv", encoding="shift-jis"
         ) as shop:
             lines = shop.readlines()
             for line in lines:
@@ -156,7 +157,7 @@ class CxbRev(CxbBase):
     async def handle_data_shop_list_sale_request(self, data: Dict) -> Dict:
         ret_str = "\r\n#ShopListSale\r\n"
         with open(
-            r"titles/cxb/data/rss/Shop/ShopList_Sale.csv", encoding="shift-jis"
+            r"titles/cxb/data/rev/Shop/ShopList_Sale.csv", encoding="shift-jis"
         ) as shop:
             lines = shop.readlines()
             for line in lines:
@@ -164,14 +165,19 @@ class CxbRev(CxbBase):
         return {"data": ret_str}
 
     async def handle_data_extra_stage_list_request(self, data: Dict) -> Dict:
-        return {"data": ""}
+        ret_str = ""
+        with open(r"titles/cxb/data/rev/ExtraStageList.csv", encoding="shift-jis") as stage:
+            lines = stage.readlines()
+            for line in lines:
+                ret_str += f"{line[:-1]}\r\n"
+        return {"data": ret_str}
 
     @cached(lifetime=86400)
     async def handle_data_exxxxx_request(self, data: Dict) -> Dict:
         extra_num = int(data["dldate"]["filetype"][-4:])
         ret_str = ""
         with open(
-            rf"titles/cxb/data/rss/Ex000{extra_num}.csv", encoding="shift-jis"
+            rf"titles/cxb/data/rev/Ex000{extra_num}.csv", encoding="shift-jis"
         ) as stage:
             lines = stage.readlines()
             for line in lines:
@@ -187,7 +193,7 @@ class CxbRev(CxbBase):
     @cached(lifetime=86400)
     async def handle_data_news_list_request(self, data: Dict) -> Dict:
         ret_str = ""
-        with open(r"titles/cxb/data/rss/NewsList.csv", encoding="UTF-8") as news:
+        with open(r"titles/cxb/data/rev/NewsList.csv", encoding="UTF-8") as news:
             lines = news.readlines()
             for line in lines:
                 ret_str += f"{line[:-1]}\r\n"
@@ -199,7 +205,7 @@ class CxbRev(CxbBase):
     @cached(lifetime=86400)
     async def handle_data_license_request(self, data: Dict) -> Dict:
         ret_str = ""
-        with open(r"titles/cxb/data/rss/License_Offline.csv", encoding="UTF-8") as lic:
+        with open(r"titles/cxb/data/rev/License_Offline.csv", encoding="UTF-8") as lic:
             lines = lic.readlines()
             for line in lines:
                 ret_str += f"{line[:-1]}\r\n"
@@ -209,7 +215,7 @@ class CxbRev(CxbBase):
     async def handle_data_course_list_request(self, data: Dict) -> Dict:
         ret_str = ""
         with open(
-            r"titles/cxb/data/rss/Course/CourseList.csv", encoding="UTF-8"
+            r"titles/cxb/data/rev/Course/CourseList.csv", encoding="UTF-8"
         ) as course:
             lines = course.readlines()
             for line in lines:
@@ -222,7 +228,7 @@ class CxbRev(CxbBase):
         extra_num = int(data["dldate"]["filetype"][-4:])
         ret_str = ""
         with open(
-            rf"titles/cxb/data/rss/Course/Cs000{extra_num}.csv", encoding="shift-jis"
+            rf"titles/cxb/data/rev/Course/Cs000{extra_num}.csv", encoding="shift-jis"
         ) as course:
             lines = course.readlines()
             for line in lines:
@@ -233,7 +239,7 @@ class CxbRev(CxbBase):
     async def handle_data_mission_list_request(self, data: Dict) -> Dict:
         ret_str = ""
         with open(
-            r"titles/cxb/data/rss/MissionList.csv", encoding="shift-jis"
+            r"titles/cxb/data/rev/MissionList.csv", encoding="shift-jis"
         ) as mission:
             lines = mission.readlines()
             for line in lines:
@@ -250,7 +256,7 @@ class CxbRev(CxbBase):
     async def handle_data_event_list_request(self, data: Dict) -> Dict:
         ret_str = ""
         with open(
-            r"titles/cxb/data/rss/Event/EventArchiveList.csv", encoding="shift-jis"
+            r"titles/cxb/data/rev/Event/EventArchiveList.csv", encoding="shift-jis"
         ) as mission:
             lines = mission.readlines()
             for line in lines:
@@ -292,7 +298,7 @@ class CxbRev(CxbBase):
     async def handle_data_event_stamp_list_request(self, data: Dict) -> Dict:
         ret_str = ""
         with open(
-            r"titles/cxb/data/rss/Event/EventStampList.csv", encoding="shift-jis"
+            r"titles/cxb/data/rev/Event/EventStampList.csv", encoding="shift-jis"
         ) as event:
             lines = event.readlines()
             for line in lines:

titles/cxb/rss1.py

@@ -73,42 +73,6 @@ class CxbRevSunriseS1(CxbBase):
             for line in lines:
                 ret_str += f"{line[:-1]}\r\n"
 
-        # ShopListSale load
-        ret_str += "\r\n#ShopListSale\r\n"
-        with open(
-            r"titles/cxb/data/rss1/Shop/ShopList_Sale.csv", encoding="shift-jis"
-        ) as shop:
-            lines = shop.readlines()
-            for line in lines:
-                ret_str += f"{line[:-1]}\r\n"
-
-        # ShopListSkinBg load
-        ret_str += "\r\n#ShopListSkinBg\r\n"
-        with open(
-            r"titles/cxb/data/rss1/Shop/ShopList_SkinBg.csv", encoding="shift-jis"
-        ) as shop:
-            lines = shop.readlines()
-            for line in lines:
-                ret_str += f"{line[:-1]}\r\n"
-
-        # ShopListSkinEffect load
-        ret_str += "\r\n#ShopListSkinEffect\r\n"
-        with open(
-            r"titles/cxb/data/rss1/Shop/ShopList_SkinEffect.csv", encoding="shift-jis"
-        ) as shop:
-            lines = shop.readlines()
-            for line in lines:
-                ret_str += f"{line[:-1]}\r\n"
-
-        # ShopListSkinNotes load
-        ret_str += "\r\n#ShopListSkinNotes\r\n"
-        with open(
-            r"titles/cxb/data/rss1/Shop/ShopList_SkinNotes.csv", encoding="shift-jis"
-        ) as shop:
-            lines = shop.readlines()
-            for line in lines:
-                ret_str += f"{line[:-1]}\r\n"
-
         # ShopListTitle load
         ret_str += "\r\n#ShopListTitle\r\n"
         with open(

titles/cxb/schema/score.py

@@ -39,6 +39,7 @@ playlog = Table(
     Column("slow2", Integer),
     Column("fail", Integer),
     Column("combo", Integer),
+    Column("grade", Integer),
     Column("date_scored", TIMESTAMP, server_default=func.now()),
     mysql_charset="utf8mb4",
 )
@@ -104,6 +105,7 @@ class CxbScoreData(BaseData):
         this_slow2: int,
         fail: int,
         combo: int,
+        grade: int,
     ) -> Optional[int]:
         """
         Add an entry to the user's play log
@@ -123,6 +125,7 @@ class CxbScoreData(BaseData):
             slow2=this_slow2,
             fail=fail,
             combo=combo,
+            grade=grade,
         )
 
         result = await self.execute(sql)

titles/diva/base.py

@@ -431,7 +431,7 @@ class DivaBase:
         profile = await self.data.profile.get_profile(data["pd_id"], self.version)
         profile_shop = await self.data.item.get_shop(data["pd_id"], self.version)
         if profile is None:
-            return
+            return {}
 
         mdl_have = "F" * 250
         # generate the mdl_have string if "unlock_all_modules" is disabled

titles/diva/index.py

@@ -79,7 +79,7 @@ class DivaServlet(BaseServlet):
 
         return True
 
-    async def render_POST(self, request: Request, game_code: str, matchers: Dict) -> bytes:
+    async def render_POST(self, request: Request) -> bytes:
         req_raw = await request.body()
         url_header = request.headers
 
@@ -98,8 +98,17 @@ class DivaServlet(BaseServlet):
             self.logger.info(f"Binary {bin_req_data['cmd']} Request")
             self.logger.debug(bin_req_data)
 
-            handler = getattr(self.base, f"handle_{bin_req_data['cmd']}_request")
-            resp = handler(bin_req_data)
+            try:
+                handler = getattr(self.base, f"handle_{bin_req_data['cmd']}_request")
+                resp = handler(bin_req_data)
+            
+            except AttributeError as e:
+                self.logger.warning(f"Unhandled {bin_req_data['cmd']} request {e}")
+                return PlainTextResponse(f"cmd={bin_req_data['cmd']}&req_id={bin_req_data['req_id']}&stat=ok")
+
+            except Exception as e:
+                self.logger.error(f"Error handling method {e}")
+                return PlainTextResponse(f"cmd={bin_req_data['cmd']}&req_id={bin_req_data['req_id']}&stat=ok")
 
             self.logger.debug(
                 f"Response cmd={bin_req_data['cmd']}&req_id={bin_req_data['req_id']}&stat=ok{resp}"

titles/diva/read.py

@@ -183,7 +183,11 @@ class DivaReader(BaseReader):
                     pv_list[pv_id] = self.add_branch(pv_list[pv_id], key_args, val)
 
         for pv_id, pv_data in pv_list.items():
-            song_id = int(pv_id.split("_")[1])
+            try:
+                song_id = int(pv_id.split("_")[1])
+            except ValueError:
+                self.logger.error(f"Invalid song ID format: {pv_id}")
+                continue
             if "songinfo" not in pv_data:
                 continue
             if "illustrator" not in pv_data["songinfo"]:

titles/diva/schema/customize.py

@@ -54,7 +54,7 @@ class DivaCustomizeItemData(BaseData):
         Given a game version and an aime id, return the cstmz_itm_have hex string
         required for diva directly
         """
-        items_list = self.get_customize_items(aime_id, version)
+        items_list = await self.get_customize_items(aime_id, version)
         if items_list is None:
             items_list = []
         item_have = 0

titles/diva/schema/module.py

@@ -50,7 +50,7 @@ class DivaModuleData(BaseData):
         Given a game version and an aime id, return the mdl_have hex string
         required for diva directly
         """
-        module_list = self.get_modules(aime_id, version)
+        module_list = await self.get_modules(aime_id, version)
         if module_list is None:
             module_list = []
         module_have = 0

titles/idac/frontend.py

@@ -141,4 +141,4 @@ class IDACFrontend(FE_Base):
             rank=rank,
             sesh=vars(usr_sesh),
             active_page="idac",
-        ))
+        ), media_type="text/html; charset=utf-8")

titles/idac/season2.py

@@ -450,7 +450,7 @@ class IDACSeason2(IDACBase):
         is_new_player = 0
 
         # check that the user_id from access_code matches the user_id
-        if user_id == self.data.card.get_user_id_from_card(access_code):
+        if user_id == await self.data.card.get_user_id_from_card(access_code):
             lock_result = 1
 
             # check if an IDAC profile already exists
@@ -1392,7 +1392,8 @@ class IDACSeason2(IDACBase):
         # get the use_count and story_use_count of the used car
         style_car_id = data.get("style_car_id")
         car_mileage = data.pop("car_mileage")
-        used_car = await self.data.item.get_car(user_id, self.version, style_car_id)._asdict()
+        used_car = await self.data.item.get_car(user_id, self.version, style_car_id)
+        used_car = used_car._asdict()
 
         # increase the use_count and story_use_count of the used car
         used_car["use_count"] += 1
@@ -1564,7 +1565,8 @@ class IDACSeason2(IDACBase):
         # get the use_count and story_use_count of the used car
         style_car_id = data.get("style_car_id")
         car_mileage = data.get("car_mileage")
-        used_car = await self.data.item.get_car(user_id, self.version, style_car_id)._asdict()
+        used_car = await self.data.item.get_car(user_id, self.version, style_car_id)
+        used_car = used_car._asdict()
 
         # increase the use_count and story_use_count of the used car
         used_car["use_count"] += 1
@@ -1669,7 +1671,7 @@ class IDACSeason2(IDACBase):
 
         return {
             "status_code": "0",
-            "special_mode_data": self._generate_special_data(user_id),
+            "special_mode_data": await self._generate_special_data(user_id),
             "car_use_count": [],
             "maker_use_count": [],
         }
@@ -1747,7 +1749,7 @@ class IDACSeason2(IDACBase):
 
         return {
             "status_code": "0",
-            "challenge_mode_data": self._generate_challenge_data(user_id),
+            "challenge_mode_data": await self._generate_challenge_data(user_id),
             "car_use_count": [],
             "maker_use_count": [],
         }
@@ -2230,7 +2232,8 @@ class IDACSeason2(IDACBase):
         # get the use_count and theory_use_count of the used car
         style_car_id = data.get("style_car_id")
         car_mileage = data.get("car_mileage")
-        used_car = await self.data.item.get_car(user_id, self.version, style_car_id)._asdict()
+        used_car = await self.data.item.get_car(user_id, self.version, style_car_id)
+        used_car = used_car._asdict()
 
         # increase the use_count and theory_use_count of the used car
         used_car["use_count"] += 1
@@ -2347,7 +2350,8 @@ class IDACSeason2(IDACBase):
         # get the use_count and net_vs_use_count of the used car
         style_car_id = data.get("style_car_id")
         car_mileage = data.pop("car_mileage")
-        used_car = await self.data.item.get_car(user_id, self.version, style_car_id)._asdict()
+        used_car = await self.data.item.get_car(user_id, self.version, style_car_id)
+        used_car = used_car._asdict()
 
         # increase the use_count and net_vs_use_count of the used car
         used_car["use_count"] += 1
@@ -2444,7 +2448,8 @@ class IDACSeason2(IDACBase):
         # get the use_count and net_vs_use_count of the used car
         style_car_id = data.get("style_car_id")
         car_mileage = data.pop("car_mileage")
-        used_car = await self.data.item.get_car(user_id, self.version, style_car_id)._asdict()
+        used_car = await self.data.item.get_car(user_id, self.version, style_car_id)
+        used_car = used_car._asdict()
 
         # increase the use_count and net_vs_use_count of the used car
         used_car["use_count"] += 1

titles/idz/index.py

@@ -137,7 +137,7 @@ class IDZServlet(BaseServlet):
                 continue
         
         loop = asyncio.get_running_loop()
-        IDZUserDB(self.core_cfg, self.game_cfg, self.rsa_keys, handler_map)
+        IDZUserDB(self.core_cfg, self.game_cfg, self.rsa_keys, handler_map).start()
         asyncio.create_task(
             loop.create_datagram_endpoint(
                 lambda: IDZEcho(),
@@ -157,7 +157,6 @@ class IDZServlet(BaseServlet):
             )
         )
 
-        self.logger.info(f"UserDB Listening on port {self.game_cfg.ports.userdb}")
 
     async def render_GET(self, request: Request) -> bytes:
         url_path = request.path_params.get('endpoint', '')

titles/idz/userdb.py

@@ -41,8 +41,8 @@ class IDZUserDB:
         self.skip_next = False
     
     def start(self) -> None:
-        self.logger.info(f"Start on port {self.config.aimedb.port}")
-        asyncio.create_task(asyncio.start_server(self.connection_cb, self.config.server.listen_address, self.config.aimedb.port))
+        self.logger.info(f"Start on port {self.game_config.ports.userdb}")
+        asyncio.create_task(asyncio.start_server(self.connection_cb, self.core_config.server.listen_address, self.game_config.ports.userdb))
 
     def append_padding(self, data: bytes):
         """Appends 0s to the end of the data until it's at the correct size"""
@@ -81,7 +81,7 @@ class IDZUserDB:
                     self.logger.debug("Connection closed")
                     return
                 
-                await self.data.Received(data, reader, writer)
+                await self.dataReceived(data, reader, writer)
                 await writer.drain()
             
             except ConnectionResetError as e:

titles/mai2/__init__.py

@@ -2,10 +2,12 @@ from titles.mai2.index import Mai2Servlet
 from titles.mai2.const import Mai2Constants
 from titles.mai2.database import Mai2Data
 from titles.mai2.read import Mai2Reader
+from .frontend import Mai2Frontend
 
 index = Mai2Servlet
 database = Mai2Data
 reader = Mai2Reader
+frontend = Mai2Frontend
 game_codes = [
     Mai2Constants.GAME_CODE_DX,
     Mai2Constants.GAME_CODE_FINALE,
@@ -15,4 +17,5 @@ game_codes = [
     Mai2Constants.GAME_CODE_ORANGE,
     Mai2Constants.GAME_CODE_GREEN,
     Mai2Constants.GAME_CODE,
+    Mai2Constants.GAME_CODE_DX_INT,
 ]

titles/mai2/base.py

@@ -4,6 +4,7 @@ import logging
 from base64 import b64decode
 from os import path, stat, remove
 from PIL import ImageFile
+from random import randint
 
 import pytz
 from core.config import CoreConfig
@@ -26,10 +27,10 @@ class Mai2Base:
         self.date_time_format = "%Y-%m-%d %H:%M:%S"
 
         if not self.core_config.server.is_using_proxy and Utils.get_title_port(self.core_config) != 80:
-            self.old_server = f"http://{self.core_config.server.hostname}:{Utils.get_title_port(cfg)}/197/MaimaiServlet/"
+            self.old_server = f"http://{self.core_config.server.hostname}:{Utils.get_title_port(cfg)}/SDEY/197/MaimaiServlet/"
 
         else:
-            self.old_server = f"http://{self.core_config.server.hostname}/197/MaimaiServlet/"
+            self.old_server = f"http://{self.core_config.server.hostname}/SDEY/197/MaimaiServlet/"
 
     async def handle_get_game_setting_api_request(self, data: Dict):
         # if reboot start/end time is not defined use the default behavior of being a few hours ago
@@ -238,8 +239,10 @@ class Mai2Base:
         user_id = data["userId"]
         upsert = data["upsertUserAll"]
         
-        if int(user_id) & 1000000000001 == 1000000000001:
-            self.logger.info("Guest play, ignoring.")
+        if int(user_id) & 0x1000000000001 == 0x1000000000001:
+            place_id = int(user_id) & 0xFFFC00000000
+            
+            self.logger.info("Guest play from place ID %d, ignoring.", place_id)
             return {"returnCode": 1, "apiName": "UpsertUserAllApi"}
 
         if "userData" in upsert and len(upsert["userData"]) > 0:
@@ -884,3 +887,45 @@ class Mai2Base:
                 self.logger.error(f"Failed to delete {out_name}.bin, please remove it manually")
 
         return {'returnCode': ret_code, 'apiName': 'UploadUserPhotoApi'}
+
+    async def handle_get_user_favorite_item_api_request(self, data: Dict) -> Dict:
+        user_id = data.get("userId", 0)
+        kind = data.get("kind", 0) # 1 is fav music, 2 is rival user IDs
+        next_index = data.get("nextIndex", 0)
+        max_ct = data.get("maxCount", 100) # always 100
+        is_all = data.get("isAllFavoriteItem", False) # always false
+        id_list: List[Dict] = []
+
+        if user_id:
+            if kind == 1:
+                fav_music = await self.data.item.get_fav_music(user_id)
+                if fav_music:
+                    for fav in fav_music:
+                        id_list.append({"orderId": 0, "id": fav["musicId"]})
+                        if len(id_list) >= 100: # Lazy but whatever
+                            break
+            
+            elif kind == 2:
+                rivals = await self.data.profile.get_rivals_game(user_id)
+                if rivals:
+                    for rival in rivals:
+                        id_list.append({"orderId": 0, "id": rival["rival"]})
+
+        return {
+            "userId": user_id,
+            "kind": kind,
+            "nextIndex": 0,
+            "userFavoriteItemList": id_list,
+        }
+
+    async def handle_get_user_recommend_rate_music_api_request(self, data: Dict) -> Dict:
+        """
+        userRecommendRateMusicIdList: list[int]
+        """
+        return {"userId": data["userId"], "userRecommendRateMusicIdList": []}
+
+    async def handle_get_user_recommend_select_music_api_request(self, data: Dict) -> Dict:
+        """
+        userRecommendSelectionMusicIdList: list[int]
+        """
+        return {"userId": data["userId"], "userRecommendSelectionMusicIdList": []}

titles/mai2/buddies.py

@@ -0,0 +1,19 @@
+from typing import Dict
+
+from core.config import CoreConfig
+from titles.mai2.festivalplus import Mai2FestivalPlus
+from titles.mai2.const import Mai2Constants
+from titles.mai2.config import Mai2Config
+
+
+class Mai2Buddies(Mai2FestivalPlus):
+    def __init__(self, cfg: CoreConfig, game_cfg: Mai2Config) -> None:
+        super().__init__(cfg, game_cfg)
+        self.version = Mai2Constants.VER_MAIMAI_DX_BUDDIES
+
+    async def handle_cm_get_user_preview_api_request(self, data: Dict) -> Dict:
+        user_data = await super().handle_cm_get_user_preview_api_request(data)
+
+        # hardcode lastDataVersion for CardMaker
+        user_data["lastDataVersion"] = "1.40.00"
+        return user_data

titles/mai2/config.py

@@ -1,3 +1,5 @@
+from typing import Dict
+
 from core.config import CoreConfig
 
 
@@ -70,8 +72,32 @@ class Mai2UploadsConfig:
         )
 
 
+class Mai2CryptoConfig:
+    def __init__(self, parent_config: "Mai2Config") -> None:
+        self.__config = parent_config
+
+    @property
+    def keys(self) -> Dict[int, list[str]]:
+        """
+        in the form of:
+        internal_version: [key, iv, salt]
+        key and iv are hex strings
+        salt is a normal UTF-8 string
+        """
+        return CoreConfig.get_config_field(
+            self.__config, "mai2", "crypto", "keys", default={}
+        )
+
+    @property
+    def encrypted_only(self) -> bool:
+        return CoreConfig.get_config_field(
+            self.__config, "mai2", "crypto", "encrypted_only", default=False
+        )
+
+
 class Mai2Config(dict):
     def __init__(self) -> None:
         self.server = Mai2ServerConfig(self)
         self.deliver = Mai2DeliverConfig(self)
         self.uploads = Mai2UploadsConfig(self)
+        self.crypto = Mai2CryptoConfig(self)

titles/mai2/const.py

@@ -28,6 +28,7 @@ class Mai2Constants:
     GAME_CODE_MILK = "SDDZ"
     GAME_CODE_FINALE = "SDEY"
     GAME_CODE_DX = "SDEZ"
+    GAME_CODE_DX_INT = "SDGA"
 
     CONFIG_NAME = "mai2.yaml"
 
@@ -53,6 +54,7 @@ class Mai2Constants:
     VER_MAIMAI_DX_UNIVERSE_PLUS = 18
     VER_MAIMAI_DX_FESTIVAL = 19
     VER_MAIMAI_DX_FESTIVAL_PLUS = 20
+    VER_MAIMAI_DX_BUDDIES = 21
 
     VERSION_STRING = (
         "maimai",
@@ -76,6 +78,7 @@ class Mai2Constants:
         "maimai DX UNiVERSE PLUS",
         "maimai DX FESTiVAL",
         "maimai DX FESTiVAL PLUS",
+        "maimai DX BUDDiES"
     )
 
     @classmethod

titles/mai2/dx.py

@@ -5,6 +5,7 @@ import json
 from random import randint
 
 from core.config import CoreConfig
+from core.utils import Utils
 from titles.mai2.base import Mai2Base
 from titles.mai2.config import Mai2Config
 from titles.mai2.const import Mai2Constants
@@ -15,19 +16,52 @@ class Mai2DX(Mai2Base):
         super().__init__(cfg, game_cfg)
         self.version = Mai2Constants.VER_MAIMAI_DX
 
+        # DX earlier version need a efficient old server uri to work
+        # game will auto add MaimaiServlet endpoint behind return uri
+        # so do not add "MaimaiServlet"
+        if not self.core_config.server.is_using_proxy and Utils.get_title_port(self.core_config) != 80:
+            self.old_server = f"http://{self.core_config.server.hostname}:{Utils.get_title_port(cfg)}/SDEY/197/"
+
+        else:
+            self.old_server = f"http://{self.core_config.server.hostname}/SDEY/197/"
+
     async def handle_get_game_setting_api_request(self, data: Dict):
+        # if reboot start/end time is not defined use the default behavior of being a few hours ago
+        if self.core_config.title.reboot_start_time == "" or self.core_config.title.reboot_end_time == "":
+            reboot_start = datetime.strftime(
+                datetime.utcnow() + timedelta(hours=6), self.date_time_format
+            )
+            reboot_end = datetime.strftime(
+                datetime.utcnow() + timedelta(hours=7), self.date_time_format
+            )
+        else:
+            # get current datetime in JST
+            current_jst = datetime.now(pytz.timezone('Asia/Tokyo')).date()
+
+            # parse config start/end times into datetime
+            reboot_start_time = datetime.strptime(self.core_config.title.reboot_start_time, "%H:%M")
+            reboot_end_time = datetime.strptime(self.core_config.title.reboot_end_time, "%H:%M")
+
+            # offset datetimes with current date/time
+            reboot_start_time = reboot_start_time.replace(year=current_jst.year, month=current_jst.month, day=current_jst.day, tzinfo=pytz.timezone('Asia/Tokyo'))
+            reboot_end_time = reboot_end_time.replace(year=current_jst.year, month=current_jst.month, day=current_jst.day, tzinfo=pytz.timezone('Asia/Tokyo'))
+
+            # create strings for use in gameSetting
+            reboot_start = reboot_start_time.strftime(self.date_time_format)
+            reboot_end = reboot_end_time.strftime(self.date_time_format)
+
         return {
             "gameSetting": {
                 "isMaintenance": False,
                 "requestInterval": 1800,
-                "rebootStartTime": "2020-01-01 07:00:00.0",
-                "rebootEndTime": "2020-01-01 07:59:59.0",
+                "rebootStartTime": reboot_start,
+                "rebootEndTime": reboot_end,
                 "movieUploadLimit": 100,
                 "movieStatus": 1,
-                "movieServerUri": self.old_server + "movie/",
-                "deliverServerUri": self.old_server + "deliver/" if self.can_deliver and self.game_config.deliver.enable else "",
-                "oldServerUri": self.old_server + "old",
-                "usbDlServerUri": self.old_server + "usbdl/" if self.can_deliver and self.game_config.deliver.udbdl_enable else "",
+                "movieServerUri": "",
+                "deliverServerUri": "",
+                "oldServerUri": self.old_server,
+                "usbDlServerUri": "",
                 "rebootInterval": 0,
             },
             "isAouAccession": False,
@@ -97,8 +131,10 @@ class Mai2DX(Mai2Base):
         user_id = data["userId"]
         upsert = data["upsertUserAll"]
         
-        if int(user_id) & 1000000000001 == 1000000000001:
-            self.logger.info("Guest play, ignoring.")
+        if int(user_id) & 0x1000000000001 == 0x1000000000001:
+            place_id = int(user_id) & 0xFFFC00000000
+            
+            self.logger.info("Guest play from place ID %d, ignoring.", place_id)
             return {"returnCode": 1, "apiName": "UpsertUserAllApi"}
 
         if "userData" in upsert and len(upsert["userData"]) > 0:
@@ -212,6 +248,9 @@ class Mai2DX(Mai2Base):
                     ),
                 )
                 await self.data.item.put_friend_season_ranking(user_id, fsr)
+        
+        if "user2pPlaylog" in upsert:
+            await self.data.score.put_playlog_2p(user_id, upsert["user2pPlaylog"])
 
         return {"returnCode": 1, "apiName": "UpsertUserAllApi"}
 
@@ -285,31 +324,6 @@ class Mai2DX(Mai2Base):
             "userCardList": card_list[start_idx:end_idx],
         }
 
-    async def handle_get_user_charge_api_request(self, data: Dict) -> Dict:
-        user_charges = await self.data.item.get_charges(data["userId"])
-        if user_charges is None:
-            return {"userId": data["userId"], "length": 0, "userChargeList": []}
-
-        user_charge_list = []
-        for charge in user_charges:
-            tmp = charge._asdict()
-            tmp.pop("id")
-            tmp.pop("user")
-            tmp["purchaseDate"] = datetime.strftime(
-                tmp["purchaseDate"], Mai2Constants.DATE_TIME_FORMAT
-            )
-            tmp["validDate"] = datetime.strftime(
-                tmp["validDate"], Mai2Constants.DATE_TIME_FORMAT
-            )
-
-            user_charge_list.append(tmp)
-
-        return {
-            "userId": data["userId"],
-            "length": len(user_charge_list),
-            "userChargeList": user_charge_list,
-        }
-
     async def handle_get_user_item_api_request(self, data: Dict) -> Dict:
         kind = int(data["nextIndex"] / 10000000000)
         next_idx = int(data["nextIndex"] % 10000000000)
@@ -549,33 +563,76 @@ class Mai2DX(Mai2Base):
         return {"userId": data["userId"], "length": 0, "userRegionList": []}
 
     async def handle_get_user_rival_data_api_request(self, data: Dict) -> Dict:
-        user_id = data["userId"]
-        rival_id = data["rivalId"]
+        user_id = data.get("userId", 0)
+        rival_id = data.get("rivalId", 0)
 
-        """
-        class UserRivalData:
-            rivalId: int
-            rivalName: str
-        """
-        return {"userId": user_id, "userRivalData": {}}
+        if not user_id or not rival_id: return {}
+
+        rival_pf = await self.data.profile.get_profile_detail(rival_id, self.version)
+        if not rival_pf: return {}
+
+        return {
+            "userId": user_id,
+            "userRivalData": {
+                "rivalId": rival_id,
+                "rivalName": rival_pf['userName']
+            }
+        }
 
     async def handle_get_user_rival_music_api_request(self, data: Dict) -> Dict:
+        user_id = data.get("userId", 0)
+        rival_id = data.get("rivalId", 0)
+        next_index = data.get("nextIndex", 0)
+        max_ct = 100
+        upper_lim = next_index + max_ct
+        rival_music_list: Dict[int, List] = {}
+
+        songs = await self.data.score.get_best_scores(rival_id)
+        if songs is None:
+            self.logger.debug("handle_get_user_rival_music_api_request: get_best_scores returned None!")
+            return {
+                "userId": user_id,
+                "rivalId": rival_id,
+                "nextIndex": 0,
+                "userRivalMusicList": [] # musicId userRivalMusicDetailList -> level achievement deluxscoreMax
+            }
+
+        num_user_songs = len(songs)
+
+        for x in range(next_index, upper_lim):
+            if x >= num_user_songs:
+                break
+
+            tmp = songs[x]._asdict()
+            if tmp['musicId'] in rival_music_list:
+                rival_music_list[tmp['musicId']].append([{"level": tmp['level'], 'achievement': tmp['achievement'], 'deluxscoreMax': tmp['deluxscoreMax']}])
+            
+            else:
+                if len(rival_music_list) >= max_ct:
+                    break
+                rival_music_list[tmp['musicId']] = [{"level": tmp['level'], 'achievement': tmp['achievement'], 'deluxscoreMax': tmp['deluxscoreMax']}]
+
+        next_index = 0 if len(rival_music_list) < max_ct or num_user_songs == upper_lim else upper_lim
+        self.logger.info(f"Send rival {rival_id} songs {next_index}-{upper_lim} ({len(rival_music_list)}) out of {num_user_songs} for user {user_id} (next idx {next_index})")
+
+        return {
+            "userId": user_id,
+            "rivalId": rival_id,
+            "nextIndex": next_index,
+            "userRivalMusicList": [{"musicId": x, "userRivalMusicDetailList": y} for x, y in rival_music_list.items()]
+        }
+
+    async def handle_get_user_new_item_api_request(self, data: Dict) -> Dict:
+        # TODO: Added in 1.41, implement this?
         user_id = data["userId"]
-        rival_id = data["rivalId"]
-        next_idx = data["nextIndex"]
-        rival_music_levels = data["userRivalMusicLevelList"]
-
-        """
-        class UserRivalMusicList:
-            class UserRivalMusicDetailList:
-                level: int
-                achievement: int
-                deluxscoreMax: int
-
-            musicId: int
-            userRivalMusicDetailList: list[UserRivalMusicDetailList]
-        """
-        return {"userId": user_id, "nextIndex": 0, "userRivalMusicList": []}
+        version = data.get("version", 1041000)
+        user_playlog_list = data.get("userPlaylogList", [])
+        
+        return {
+            "userId": user_id,
+            "itemKind": -1,
+            "itemId": -1,
+        }
 
     async def handle_get_user_music_api_request(self, data: Dict) -> Dict:
         user_id = data.get("userId", 0)        
@@ -622,3 +679,208 @@ class Mai2DX(Mai2Base):
             return ret
         ret['loginId'] = ret.get('loginCount', 0)
         return ret
+
+    # CardMaker support added in Universe
+    async def handle_cm_get_user_preview_api_request(self, data: Dict) -> Dict:
+        p = await self.data.profile.get_profile_detail(data["userId"], self.version)
+        if p is None:
+            return {}
+
+        return {
+            "userName": p["userName"],
+            "rating": p["playerRating"],
+            # hardcode lastDataVersion for CardMaker
+            "lastDataVersion": "1.20.00", # Future versiohs should replace this with the correct version
+            # checks if the user is still logged in
+            "isLogin": False,
+            "isExistSellingCard": True,
+        }
+
+    async def handle_cm_get_user_data_api_request(self, data: Dict) -> Dict:
+        # user already exists, because the preview checks that already
+        p = await self.data.profile.get_profile_detail(data["userId"], self.version)
+
+        cards = await self.data.card.get_user_cards(data["userId"])
+        if cards is None or len(cards) == 0:
+            # This should never happen
+            self.logger.error(
+                f"handle_get_user_data_api_request: Internal error - No cards found for user id {data['userId']}"
+            )
+            return {}
+
+        # get the dict representation of the row so we can modify values
+        user_data = p._asdict()
+
+        # remove the values the game doesn't want
+        user_data.pop("id")
+        user_data.pop("user")
+        user_data.pop("version")
+
+        return {"userId": data["userId"], "userData": user_data}
+
+    async def handle_cm_login_api_request(self, data: Dict) -> Dict:
+        return {"returnCode": 1}
+
+    async def handle_cm_logout_api_request(self, data: Dict) -> Dict:
+        return {"returnCode": 1}
+
+    async def handle_cm_get_selling_card_api_request(self, data: Dict) -> Dict:
+        selling_cards = await self.data.static.get_enabled_cards(self.version)
+        if selling_cards is None:
+            return {"length": 0, "sellingCardList": []}
+
+        selling_card_list = []
+        for card in selling_cards:
+            tmp = card._asdict()
+            tmp.pop("id")
+            tmp.pop("version")
+            tmp.pop("cardName")
+            tmp.pop("enabled")
+
+            tmp["startDate"] = datetime.strftime(
+                tmp["startDate"], Mai2Constants.DATE_TIME_FORMAT
+            )
+            tmp["endDate"] = datetime.strftime(
+                tmp["endDate"], Mai2Constants.DATE_TIME_FORMAT
+            )
+            tmp["noticeStartDate"] = datetime.strftime(
+                tmp["noticeStartDate"], Mai2Constants.DATE_TIME_FORMAT
+            )
+            tmp["noticeEndDate"] = datetime.strftime(
+                tmp["noticeEndDate"], Mai2Constants.DATE_TIME_FORMAT
+            )
+
+            selling_card_list.append(tmp)
+
+        return {"length": len(selling_card_list), "sellingCardList": selling_card_list}
+
+    async def handle_cm_get_user_card_api_request(self, data: Dict) -> Dict:
+        user_cards = await self.data.item.get_cards(data["userId"])
+        if user_cards is None:
+            return {"returnCode": 1, "length": 0, "nextIndex": 0, "userCardList": []}
+
+        max_ct = data["maxCount"]
+        next_idx = data["nextIndex"]
+        start_idx = next_idx
+        end_idx = max_ct + start_idx
+
+        if len(user_cards[start_idx:]) > max_ct:
+            next_idx += max_ct
+        else:
+            next_idx = 0
+
+        card_list = []
+        for card in user_cards:
+            tmp = card._asdict()
+            tmp.pop("id")
+            tmp.pop("user")
+
+            tmp["startDate"] = datetime.strftime(
+                tmp["startDate"], Mai2Constants.DATE_TIME_FORMAT
+            )
+            tmp["endDate"] = datetime.strftime(
+                tmp["endDate"], Mai2Constants.DATE_TIME_FORMAT
+            )
+            card_list.append(tmp)
+
+        return {
+            "returnCode": 1,
+            "length": len(card_list[start_idx:end_idx]),
+            "nextIndex": next_idx,
+            "userCardList": card_list[start_idx:end_idx],
+        }
+
+    async def handle_cm_get_user_item_api_request(self, data: Dict) -> Dict:
+        await self.handle_get_user_item_api_request(data)
+
+    async def handle_cm_get_user_character_api_request(self, data: Dict) -> Dict:
+        characters = await self.data.item.get_characters(data["userId"])
+
+        chara_list = []
+        for chara in characters:
+            chara_list.append(
+                {
+                    "characterId": chara["characterId"],
+                    # no clue why those values are even needed
+                    "point": 0,
+                    "count": 0,
+                    "level": chara["level"],
+                    "nextAwake": 0,
+                    "nextAwakePercent": 0,
+                    "favorite": False,
+                    "awakening": chara["awakening"],
+                    "useCount": chara["useCount"],
+                }
+            )
+
+        return {
+            "returnCode": 1,
+            "length": len(chara_list),
+            "userCharacterList": chara_list,
+        }
+
+    async def handle_cm_get_user_card_print_error_api_request(self, data: Dict) -> Dict:
+        return {"length": 0, "userPrintDetailList": []}
+
+    async def handle_cm_upsert_user_print_api_request(self, data: Dict) -> Dict:
+        user_id = data["userId"]
+        upsert = data["userPrintDetail"]
+
+        # set a random card serial number
+        serial_id = "".join([str(randint(0, 9)) for _ in range(20)])
+
+        # calculate start and end date of the card
+        start_date = datetime.utcnow()
+        end_date = datetime.utcnow() + timedelta(days=15)
+
+        user_card = upsert["userCard"]
+        await self.data.item.put_card(
+            user_id,
+            user_card["cardId"],
+            user_card["cardTypeId"],
+            user_card["charaId"],
+            user_card["mapId"],
+            # add the correct start date and also the end date in 15 days
+            start_date,
+            end_date,
+        )
+
+        # get the profile extend to save the new bought card
+        extend = await self.data.profile.get_profile_extend(user_id, self.version)
+        if extend:
+            extend = extend._asdict()
+            # parse the selectedCardList
+            # 6 = Freedom Pass, 4 = Gold Pass (cardTypeId)
+            selected_cards: List = extend["selectedCardList"]
+
+            # if no pass is already added, add the corresponding pass
+            if not user_card["cardTypeId"] in selected_cards:
+                selected_cards.insert(0, user_card["cardTypeId"])
+
+            extend["selectedCardList"] = selected_cards
+            await self.data.profile.put_profile_extend(user_id, self.version, extend)
+
+        # properly format userPrintDetail for the database
+        upsert.pop("userCard")
+        upsert.pop("serialId")
+        upsert["printDate"] = datetime.strptime(upsert["printDate"], "%Y-%m-%d")
+
+        await self.data.item.put_user_print_detail(user_id, serial_id, upsert)
+
+        return {
+            "returnCode": 1,
+            "orderId": 0,
+            "serialId": serial_id,
+            "startDate": datetime.strftime(start_date, Mai2Constants.DATE_TIME_FORMAT),
+            "endDate": datetime.strftime(end_date, Mai2Constants.DATE_TIME_FORMAT),
+        }
+
+    async def handle_cm_upsert_user_printlog_api_request(self, data: Dict) -> Dict:
+        return {
+            "returnCode": 1,
+            "orderId": 0,
+            "serialId": data["userPrintlog"]["serialId"],
+        }
+
+    async def handle_cm_upsert_buy_card_api_request(self, data: Dict) -> Dict:
+        return {"returnCode": 1}

titles/mai2/festival.py

@@ -20,18 +20,6 @@ class Mai2Festival(Mai2UniversePlus):
 
     async def handle_user_login_api_request(self, data: Dict) -> Dict:
         user_login = await super().handle_user_login_api_request(data)
-        # useless?
+        # TODO: Make use of this
         user_login["Bearer"] = "ARTEMiSTOKEN"
         return user_login
-
-    async def handle_get_user_recommend_rate_music_api_request(self, data: Dict) -> Dict:
-        """
-        userRecommendRateMusicIdList: list[int]
-        """
-        return {"userId": data["userId"], "userRecommendRateMusicIdList": []}
-
-    async def handle_get_user_recommend_select_music_api_request(self, data: Dict) -> Dict:
-        """
-        userRecommendSelectionMusicIdList: list[int]
-        """
-        return {"userId": data["userId"], "userRecommendSelectionMusicIdList": []}

titles/mai2/festivalplus.py

@@ -17,22 +17,3 @@ class Mai2FestivalPlus(Mai2Festival):
         # hardcode lastDataVersion for CardMaker
         user_data["lastDataVersion"] = "1.35.00"
         return user_data
-
-    async def handle_get_user_favorite_item_api_request(self, data: Dict) -> Dict:
-        user_id = data.get("userId", 0)
-        kind = data.get("kind", 2)
-        next_index = data.get("nextIndex", 0)
-        max_ct = data.get("maxCount", 100)
-        is_all = data.get("isAllFavoriteItem", False)
-
-        """
-        class userFavoriteItemList:
-            orderId: int
-            id: int
-        """
-        return {
-            "userId": user_id,
-            "kind": kind,
-            "nextIndex": 0,
-            "userFavoriteItemList": [],
-        }

titles/mai2/frontend.py

@@ -0,0 +1,190 @@
+from typing import List
+from starlette.routing import Route, Mount
+from starlette.requests import Request
+from starlette.responses import Response, RedirectResponse
+from os import path
+import yaml
+import jinja2
+
+from core.frontend import FE_Base, UserSession
+from core.config import CoreConfig
+from .database import Mai2Data
+from .config import Mai2Config
+from .const import Mai2Constants
+
+class Mai2Frontend(FE_Base):
+    def __init__(
+        self, cfg: CoreConfig, environment: jinja2.Environment, cfg_dir: str
+    ) -> None:
+        super().__init__(cfg, environment)
+        self.data = Mai2Data(cfg)
+        self.game_cfg = Mai2Config()
+        if path.exists(f"{cfg_dir}/{Mai2Constants.CONFIG_NAME}"):
+            self.game_cfg.update(
+                yaml.safe_load(open(f"{cfg_dir}/{Mai2Constants.CONFIG_NAME}"))
+            )
+        self.nav_name = "maimai"
+
+    def get_routes(self) -> List[Route]:
+        return [
+            Route("/", self.render_GET, methods=['GET']),
+            Mount("/playlog", routes=[
+                Route("/", self.render_GET_playlog, methods=['GET']),
+                Route("/{index}", self.render_GET_playlog, methods=['GET']),
+            ]),
+            Route("/update.name", self.update_name, methods=['POST']),
+            Route("/version.change", self.version_change, methods=['POST']),
+        ]
+
+    async def render_GET(self, request: Request) -> bytes:
+        template = self.environment.get_template(
+            "titles/mai2/templates/mai2_index.jinja"
+        )
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh:
+            usr_sesh = UserSession()
+
+        if usr_sesh.user_id > 0:
+            versions = await self.data.profile.get_all_profile_versions(usr_sesh.user_id)
+            profile = []
+            if versions:
+                # maimai_version is -1 means it is not initialized yet, select a default version from existing.
+                if usr_sesh.maimai_version < 0:
+                    usr_sesh.maimai_version = versions[0]['version']
+                profile = await self.data.profile.get_profile_detail(usr_sesh.user_id, usr_sesh.maimai_version)
+                versions = [x['version'] for x in versions]
+
+            resp = Response(template.render(
+                title=f"{self.core_config.server.name} | {self.nav_name}",
+                game_list=self.environment.globals["game_list"],
+                sesh=vars(usr_sesh),
+                user_id=usr_sesh.user_id,
+                profile=profile,
+                version_list=Mai2Constants.VERSION_STRING,
+                versions=versions,
+                cur_version=usr_sesh.maimai_version
+            ), media_type="text/html; charset=utf-8")
+
+            if usr_sesh.maimai_version >= 0:
+                encoded_sesh = self.encode_session(usr_sesh)
+                resp.delete_cookie("ARTEMIS_SESH")
+                resp.set_cookie("ARTEMIS_SESH", encoded_sesh)
+            return resp
+
+        else:
+            return RedirectResponse("/gate/", 303)
+
+    async def render_GET_playlog(self, request: Request) -> bytes:
+        template = self.environment.get_template(
+            "titles/mai2/templates/mai2_playlog.jinja"
+        )
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh:
+            print("wtf")
+            usr_sesh = UserSession()
+
+        if usr_sesh.user_id > 0:
+            if usr_sesh.maimai_version < 0:
+                print(usr_sesh.maimai_version)
+                return RedirectResponse("/game/mai2/", 303)
+            path_index = request.path_params.get('index')
+            if not path_index or int(path_index) < 1:
+                index = 0
+            else:
+                index = int(path_index) - 1 # 0 and 1 are 1st page
+            user_id = usr_sesh.user_id
+            playlog_count = await self.data.score.get_user_playlogs_count(user_id)
+            if playlog_count < index * 20 :
+                return Response(template.render(
+                    title=f"{self.core_config.server.name} | {self.nav_name}",
+                    game_list=self.environment.globals["game_list"],
+                    sesh=vars(usr_sesh),
+                    playlog_count=0
+                ), media_type="text/html; charset=utf-8")
+            playlog = await self.data.score.get_playlogs(user_id, index, 20)
+            playlog_with_title = []
+            for record in playlog:
+                music_chart = await self.data.static.get_music_chart(usr_sesh.maimai_version, record.musicId, record.level)
+                if music_chart:
+                    difficultyNum=music_chart.chartId
+                    difficulty=music_chart.difficulty
+                    artist=music_chart.artist
+                    title=music_chart.title
+                else:
+                    difficultyNum=0
+                    difficulty=0
+                    artist="unknown"
+                    title="musicid: " + str(record.musicId)
+                playlog_with_title.append({
+                    "raw": record,
+                    "title": title,
+                    "difficultyNum": difficultyNum,
+                    "difficulty": difficulty,
+                    "artist": artist,
+                })
+            return Response(template.render(
+                title=f"{self.core_config.server.name} | {self.nav_name}",
+                game_list=self.environment.globals["game_list"],
+                sesh=vars(usr_sesh),
+                user_id=usr_sesh.user_id,
+                playlog=playlog_with_title,
+                playlog_count=playlog_count
+            ), media_type="text/html; charset=utf-8")
+        else:
+            return RedirectResponse("/gate/", 303)
+
+    async def update_name(self, request: Request) -> bytes:
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh:
+            return RedirectResponse("/gate/", 303)
+
+        form_data = await request.form()
+        new_name: str  = form_data.get("new_name")
+        new_name_full = ""
+
+        if not new_name:
+            return RedirectResponse("/gate/?e=4", 303)
+
+        if len(new_name) > 8:
+            return RedirectResponse("/gate/?e=8", 303)
+
+        for x in new_name: # FIXME: This will let some invalid characters through atm
+            o = ord(x)
+            try:
+                if o == 0x20:
+                    new_name_full += chr(0x3000)
+                elif o < 0x7F and o > 0x20:
+                    new_name_full += chr(o + 0xFEE0)
+                elif o <= 0x7F:
+                    self.logger.warn(f"Invalid ascii character {o:02X}")
+                    return RedirectResponse("/gate/?e=4", 303)
+                else:
+                    new_name_full += x
+
+            except Exception as e:
+                self.logger.error(f"Something went wrong parsing character {o:04X} - {e}")
+                return RedirectResponse("/gate/?e=4", 303)
+
+        if not await self.data.profile.update_name(usr_sesh.user_id, new_name_full):
+            return RedirectResponse("/gate/?e=999", 303)
+
+        return RedirectResponse("/game/mai2/?s=1", 303)
+
+    async def version_change(self, request: Request):
+        usr_sesh = self.validate_session(request)
+        if not usr_sesh:
+            usr_sesh = UserSession()
+
+        if usr_sesh.user_id > 0:
+            form_data = await request.form()
+            maimai_version = form_data.get("version")
+            self.logger.info(f"version change to: {maimai_version}")
+            if(maimai_version.isdigit()):
+                usr_sesh.maimai_version=int(maimai_version)
+                encoded_sesh = self.encode_session(usr_sesh)
+                self.logger.info(f"Created session with JWT {encoded_sesh}")
+                resp = RedirectResponse("/game/mai2/", 303)
+                resp.set_cookie("ARTEMIS_SESH", encoded_sesh)
+            return resp
+        else:
+            return RedirectResponse("/gate/", 303)

titles/mai2/index.py

@@ -6,9 +6,13 @@ import inflection
 import yaml
 import logging, coloredlogs
 import zlib
+import string
 from logging.handlers import TimedRotatingFileHandler
 from os import path, mkdir
 from typing import Tuple, List, Dict
+from Crypto.Hash import MD5
+from Crypto.Cipher import AES
+from Crypto.Util.Padding import pad
 
 from core.config import CoreConfig
 from core.utils import Utils
@@ -25,12 +29,14 @@ from .universe import Mai2Universe
 from .universeplus import Mai2UniversePlus
 from .festival import Mai2Festival
 from .festivalplus import Mai2FestivalPlus
+from .buddies import Mai2Buddies
 
 
 class Mai2Servlet(BaseServlet):
     def __init__(self, core_cfg: CoreConfig, cfg_dir: str) -> None:
         super().__init__(core_cfg, cfg_dir)
         self.game_cfg = Mai2Config()
+        self.hash_table: Dict[int, Dict[str, str]] = {}
         if path.exists(f"{cfg_dir}/{Mai2Constants.CONFIG_NAME}"):
             self.game_cfg.update(
                 yaml.safe_load(open(f"{cfg_dir}/{Mai2Constants.CONFIG_NAME}"))
@@ -58,6 +64,7 @@ class Mai2Servlet(BaseServlet):
             Mai2UniversePlus,
             Mai2Festival,
             Mai2FestivalPlus,
+            Mai2Buddies
         ]
 
         self.logger = logging.getLogger("mai2")
@@ -84,6 +91,37 @@ class Mai2Servlet(BaseServlet):
                 level=self.game_cfg.server.loglevel, logger=self.logger, fmt=log_fmt_str
             )
             self.logger.initted = True
+        
+        for version, keys in self.game_cfg.crypto.keys.items():
+            if version < Mai2Constants.VER_MAIMAI_DX:
+                continue
+
+            if len(keys) < 3:
+                continue
+
+            self.hash_table[version] = {}
+            method_list = [
+                method
+                for method in dir(self.versions[version])
+                if not method.startswith("__")
+            ]
+
+            for method in method_list:
+                # handle_method_api_request -> HandleMethodApiRequest
+                # remove the first 6 chars and the final 7 chars to get the canonical
+                # endpoint name.
+                method_fixed = inflection.camelize(method)[6:-7]
+                hash = MD5.new((method_fixed + keys[2]).encode())
+
+                # truncate unused bytes like the game does
+                hashed_name = hash.hexdigest()
+                self.hash_table[version][hashed_name] = method_fixed
+
+                self.logger.debug(
+                    "Hashed v%s method %s with %s to get %s",
+                    version, method_fixed, keys[2], hashed_name
+                )
+
 
     @classmethod
     def is_game_enabled(
@@ -103,26 +141,26 @@ class Mai2Servlet(BaseServlet):
     
     def get_routes(self) -> List[Route]:
         return [
-            Route("/{version:int}/MaimaiServlet/api/movie/{endpoint:str}", self.handle_movie, methods=['GET', 'POST']),
-            Route("/{version:int}/MaimaiServlet/old/{endpoint:str}", self.handle_old_srv),
-            Route("/{version:int}/MaimaiServlet/old/{endpoint:str}/{placeid:str}/{keychip:str}/{userid:int}", self.handle_old_srv_userdata),
-            Route("/{version:int}/MaimaiServlet/old/{endpoint:str}/{userid:int}", self.handle_old_srv_userdata),
-            Route("/{version:int}/MaimaiServlet/old/{endpoint:str}/{userid:int}", self.handle_old_srv_userdata),
-            Route("/{version:int}/MaimaiServlet/usbdl/{endpoint:str}", self.handle_usbdl),
-            Route("/{version:int}/MaimaiServlet/deliver/{endpoint:str}", self.handle_deliver),
-            Route("/{version:int}/MaimaiServlet/{endpoint:str}", self.handle_mai, methods=['POST']),
-            Route("/{version:int}/Maimai2Servlet/{endpoint:str}", self.handle_mai2, methods=['POST']),
+            Route("/{game:str}/{version:int}/MaimaiServlet/api/movie/{endpoint:str}", self.handle_movie, methods=['GET', 'POST']),
+            Route("/{game:str}/{version:int}/MaimaiServlet/old/{endpoint:str}", self.handle_old_srv),
+            Route("/{game:str}/{version:int}/MaimaiServlet/old/{endpoint:str}/{placeid:str}/{keychip:str}/{userid:int}", self.handle_old_srv_userdata),
+            Route("/{game:str}/{version:int}/MaimaiServlet/old/{endpoint:str}/{userid:int}", self.handle_old_srv_userdata),
+            Route("/{game:str}/{version:int}/MaimaiServlet/old/{endpoint:str}/{userid:int}", self.handle_old_srv_userdata),
+            Route("/{game:str}/{version:int}/MaimaiServlet/usbdl/{endpoint:str}", self.handle_usbdl),
+            Route("/{game:str}/{version:int}/MaimaiServlet/deliver/{endpoint:str}", self.handle_deliver),
+            Route("/{game:str}/{version:int}/MaimaiServlet/{endpoint:str}", self.handle_mai, methods=['POST']),
+            Route("/{game:str}/{version:int}/Maimai2Servlet/{endpoint:str}", self.handle_mai2, methods=['POST']),
         ]
         
-    def get_allnet_info(self, game_code: str, game_ver: int, keychip: str) -> Tuple[str, str]:        
+    def get_allnet_info(self, game_code: str, game_ver: int, keychip: str) -> Tuple[str, str]:
         if not self.core_cfg.server.is_using_proxy and Utils.get_title_port(self.core_cfg) != 80:
             return (
-                f"http://{self.core_cfg.server.hostname}:{Utils.get_title_port(self.core_cfg)}/{game_ver}/",
+                f"http://{self.core_cfg.server.hostname}:{Utils.get_title_port(self.core_cfg)}/{game_code}/{game_ver}/",
                 f"{self.core_cfg.server.hostname}",
             )
 
         return (
-            f"http://{self.core_cfg.server.hostname}/{game_ver}/",
+            f"http://{self.core_cfg.server.hostname}/{game_code}/{game_ver}/",
             f"{self.core_cfg.server.hostname}",
         )
 
@@ -227,7 +265,7 @@ class Mai2Servlet(BaseServlet):
                 self.logger.error(f"Error handling v{version} method {endpoint} - {e}")
                 return Response(zlib.compress(b'{"returnCode": "0"}'))
 
-        if resp == None:
+        if resp is None:
             resp = {"returnCode": 1}
 
         self.logger.debug(f"Response {resp}")
@@ -237,42 +275,101 @@ class Mai2Servlet(BaseServlet):
     async def handle_mai2(self, request: Request) -> bytes:
         endpoint: str = request.path_params.get('endpoint')
         version: int = request.path_params.get('version')
+        game_code = request.path_params.get('game')
+
         if endpoint.lower() == "ping":
             return Response(zlib.compress(b'{"returnCode": "1"}'))
 
         req_raw = await request.body()
         internal_ver = 0
         client_ip = Utils.get_ip_addr(request)
-        if version < 105:  # 1.0
-            internal_ver = Mai2Constants.VER_MAIMAI_DX
-        elif version >= 105 and version < 110:  # PLUS
-            internal_ver = Mai2Constants.VER_MAIMAI_DX_PLUS
-        elif version >= 110 and version < 115:  # Splash
-            internal_ver = Mai2Constants.VER_MAIMAI_DX_SPLASH
-        elif version >= 115 and version < 120:  # Splash PLUS
-            internal_ver = Mai2Constants.VER_MAIMAI_DX_SPLASH_PLUS
-        elif version >= 120 and version < 125:  # UNiVERSE
-            internal_ver = Mai2Constants.VER_MAIMAI_DX_UNIVERSE
-        elif version >= 125 and version < 130:  # UNiVERSE PLUS
-            internal_ver = Mai2Constants.VER_MAIMAI_DX_UNIVERSE_PLUS
-        elif version >= 130 and version < 135:  # FESTiVAL
-            internal_ver = Mai2Constants.VER_MAIMAI_DX_FESTIVAL
-        elif version >= 135:  # FESTiVAL PLUS
-            internal_ver = Mai2Constants.VER_MAIMAI_DX_FESTIVAL_PLUS
+        encrypted = False
+        
+        if game_code == "SDEZ": # JP
+          if version < 110:  # 1.0
+              internal_ver = Mai2Constants.VER_MAIMAI_DX
+          elif version >= 110 and version < 114:  # PLUS
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_PLUS
+          elif version >= 114 and version < 117:  # Splash
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_SPLASH
+          elif version >= 117 and version < 120:  # Splash PLUS
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_SPLASH_PLUS
+          elif version >= 120 and version < 125:  # UNiVERSE
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_UNIVERSE
+          elif version >= 125 and version < 130:  # UNiVERSE PLUS
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_UNIVERSE_PLUS
+          elif version >= 130 and version < 135:  # FESTiVAL
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_FESTIVAL
+          elif version >= 135 and version < 140:  # FESTiVAL PLUS
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_FESTIVAL_PLUS
+          elif version >= 140:  # BUDDiES
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_BUDDIES
+        elif game_code == "SDGA": # Int
+          if version < 105:  # 1.0
+              internal_ver = Mai2Constants.VER_MAIMAI_DX
+          elif version >= 105 and version < 110:  # PLUS
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_PLUS
+          elif version >= 110 and version < 115:  # Splash
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_SPLASH
+          elif version >= 115 and version < 120:  # Splash PLUS
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_SPLASH_PLUS
+          elif version >= 120 and version < 125:  # UNiVERSE
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_UNIVERSE
+          elif version >= 125 and version < 130:  # UNiVERSE PLUS
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_UNIVERSE_PLUS
+          elif version >= 130 and version < 135:  # FESTiVAL
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_FESTIVAL
+          elif version >= 135 and version < 140:  # FESTiVAL PLUS
+              internal_ver = Mai2Constants.VER_MAIMAI_DX_FESTIVAL_PLUS
 
+        if all(c in string.hexdigits for c in endpoint) and len(endpoint) == 32:
+            # If we get a 32 character long hex string, it's a hash and we're
+            # dealing with an encrypted request. False positives shouldn't happen
+            # as long as requests are suffixed with `Api`.
+            if internal_ver not in self.hash_table:
+                self.logger.error(
+                    "v%s does not support encryption or no keys entered",
+                    version,
+                )
+                return Response(zlib.compress(b'{"stat": "0"}'))
+            elif endpoint.lower() not in self.hash_table[internal_ver]:
+                self.logger.error(
+                    "No hash found for v%s endpoint %s",
+                    version, endpoint
+                )
+                return Response(zlib.compress(b'{"stat": "0"}'))
+            
+            endpoint = self.hash_table[internal_ver][endpoint.lower()]
+
+            try:
+                crypt = AES.new(
+                    bytes.fromhex(self.game_cfg.crypto.keys[internal_ver][0]),
+                    AES.MODE_CBC,
+                    bytes.fromhex(self.game_cfg.crypto.keys[internal_ver][1]),
+                )
+
+                req_raw = crypt.decrypt(req_raw)
+
+            except Exception as e:
+                self.logger.error(
+                    "Failed to decrypt v%s request to %s",
+                    version, endpoint,
+                    exc_info=e,
+                )
+                return Response(zlib.compress(b'{"stat": "0"}'))
+
+            encrypted = True
+        
         if (
-            request.headers.get("Mai-Encoding") is not None
-            or request.headers.get("X-Mai-Encoding") is not None
+            not encrypted
+            and self.game_cfg.crypto.encrypted_only
+            and version >= 110
         ):
-            # The has is some flavor of MD5 of the endpoint with a constant bolted onto the end of it.
-            # See cake.dll's Obfuscator function for details. Hopefully most DLL edits will remove
-            # these two(?) headers to not cause issues, but given the general quality of SEGA data...
-            enc_ver = request.headers.get("Mai-Encoding")
-            if enc_ver is None:
-                enc_ver = request.headers.get("X-Mai-Encoding")
-            self.logger.debug(
-                f"Encryption v{enc_ver} - User-Agent: {request.headers.get('User-Agent')}"
+            self.logger.error(
+                "Unencrypted v%s %s request, but config is set to encrypted only: %r",
+                version, endpoint, req_raw
             )
+            return Response(zlib.compress(b'{"stat": "0"}'))            
 
         try:
             unzip = zlib.decompress(req_raw)
@@ -288,6 +385,11 @@ class Mai2Servlet(BaseServlet):
         self.logger.info(f"v{version} {endpoint} request from {client_ip}")
         self.logger.debug(req_data)
 
+        endpoint = (
+            endpoint.replace("MaimaiExp", "")
+            if game_code == Mai2Constants.GAME_CODE_DX_INT
+            else endpoint
+        )
         func_to_find = "handle_" + inflection.underscore(endpoint) + "_request"
         handler_cls = self.versions[internal_ver](self.core_cfg, self.game_cfg)
 
@@ -304,12 +406,26 @@ class Mai2Servlet(BaseServlet):
                 self.logger.error(f"Error handling v{version} method {endpoint} - {e}")
                 return Response(zlib.compress(b'{"stat": "0"}'))
 
-        if resp == None:
+        if resp is None:
             resp = {"returnCode": 1}
 
         self.logger.debug(f"Response {resp}")
 
-        return Response(zlib.compress(json.dumps(resp, ensure_ascii=False).encode("utf-8")))
+        zipped = zlib.compress(json.dumps(resp, ensure_ascii=False).encode("utf-8"))
+
+        if not encrypted or version < 110:
+            return Response(zipped)
+        
+        padded = pad(zipped, 16)
+
+        crypt = AES.new(
+            bytes.fromhex(self.game_cfg.crypto.keys[internal_ver][0]),
+            AES.MODE_CBC,
+            bytes.fromhex(self.game_cfg.crypto.keys[internal_ver][1]),
+        )
+
+        return Response(crypt.encrypt(padded))
+
 
     async def handle_old_srv(self, request: Request) -> bytes:
         endpoint = request.path_params.get('endpoint')

titles/mai2/schema/item.py

@@ -134,6 +134,20 @@ favorite = Table(
     mysql_charset="utf8mb4",
 )
 
+fav_music = Table(
+    "mai2_item_favorite_music",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column("musicId", Integer, nullable=False),
+    UniqueConstraint("user", "musicId", name="mai2_item_favorite_music_uk"),
+    mysql_charset="utf8mb4",
+)
+
 charge = Table(
     "mai2_item_charge",
     metadata,
@@ -451,6 +465,30 @@ class Mai2ItemData(BaseData):
             return None
         return result.fetchall()
 
+    async def get_fav_music(self, user_id: int) -> Optional[List[Row]]:
+        result = await self.execute(fav_music.select(fav_music.c.user == user_id))
+        if result:
+            return result.fetchall()
+
+    async def add_fav_music(self, user_id: int, music_id: int) -> Optional[int]:
+        sql = insert(fav_music).values(
+            user = user_id,
+            musicId = music_id
+        )
+        
+        conflict = sql.on_duplicate_key_do_nothing()
+        
+        result = await self.execute(conflict)
+        if result:
+            return result.lastrowid
+        
+        self.logger.error(f"Failed to add music {music_id} as favorite for user {user_id}!")
+        
+    async def remove_fav_music(self, user_id: int, music_id: int) -> None:
+        result = await self.execute(fav_music.delete(and_(fav_music.c.user == user_id, fav_music.c.musicId == music_id)))
+        if not result:
+            self.logger.error(f"Failed to remove music {music_id} as favorite for user {user_id}!")
+
     async def put_card(
         self,
         user_id: int,

titles/mai2/schema/profile.py

@@ -40,6 +40,8 @@ detail = Table(
     Column("charaLockSlot", JSON),
     Column("contentBit", BigInteger),
     Column("playCount", Integer),
+    Column("currentPlayCount", Integer), # new with buddies
+    Column("renameCredit", Integer), # new with buddies
     Column("mapStock", Integer),  # new with fes+
     Column("eventWatchedDate", String(25)),
     Column("lastGameId", String(25)),
@@ -489,8 +491,31 @@ consec_logins = Table(
     mysql_charset="utf8mb4",
 )
 
+rival = Table(
+    "mai2_user_rival",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column(
+        "rival",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column("show", Boolean, nullable=False, server_default="0"),
+    UniqueConstraint("user", "rival", name="mai2_user_rival_uk"),
+    mysql_charset="utf8mb4",
+)
 
 class Mai2ProfileData(BaseData):
+    async def get_all_profile_versions(self, user_id: int) -> Optional[List[Row]]:
+        result = await self.execute(detail.select(detail.c.user == user_id))
+        if result:
+            return result.fetchall()
+
     async def put_profile_detail(
         self, user_id: int, version: int, detail_data: Dict, is_dx: bool = True
     ) -> Optional[Row]:
@@ -841,3 +866,52 @@ class Mai2ProfileData(BaseData):
         if result is None:
             return None
         return result.fetchone()
+
+    async def get_rivals(self, user_id: int) -> Optional[List[Row]]:
+        result = await self.execute(rival.select(rival.c.user == user_id))
+        if result:
+            return result.fetchall()
+
+    async def get_rivals_game(self, user_id: int) -> Optional[List[Row]]:
+        result = await self.execute(rival.select(and_(rival.c.user == user_id, rival.c.show == True)).limit(3))
+        if result:
+            return result.fetchall()
+    
+    async def set_rival_shown(self, user_id: int, rival_id: int, is_shown: bool) -> None:
+        sql = rival.update(and_(rival.c.user == user_id, rival.c.rival == rival_id)).values(
+            show = is_shown
+        )
+
+        result = await self.execute(sql)
+        if not result:
+            self.logger.error(f"Failed to set rival {rival_id} shown status to {is_shown} for user {user_id}")
+
+    async def add_rival(self, user_id: int, rival_id: int) -> Optional[int]:
+        sql = insert(rival).values(
+            user = user_id,
+            rival = rival_id
+        )
+        
+        conflict = sql.on_duplicate_key_do_nothing()
+        
+        result = await self.execute(conflict)
+        if result:
+            return result.lastrowid
+        
+        self.logger.error(f"Failed to add music {rival_id} as favorite for user {user_id}!")
+        
+    async def remove_rival(self, user_id: int, rival_id: int) -> None:
+        result = await self.execute(rival.delete(and_(rival.c.user == user_id, rival.c.rival == rival_id)))
+        if not result:
+            self.logger.error(f"Failed to remove rival {rival_id} for user {user_id}!")
+
+    async def update_name(self, user_id: int, new_name: str) -> bool:
+        sql = detail.update(detail.c.user == user_id).values(
+            userName=new_name
+        )
+        result = await self.execute(sql)
+
+        if result is None:
+            self.logger.warning(f"Failed to set user {user_id} name to {new_name}")
+            return False
+        return True

titles/mai2/schema/score.py

@@ -145,11 +145,34 @@ playlog = Table(
     Column("isNewFree", Boolean),
     Column("extNum1", Integer),
     Column("extNum2", Integer),
-    Column("extNum4", Integer, server_default="0"),
+    Column("extNum4", Integer),
+    Column("extBool1", Boolean), # new with buddies
     Column("trialPlayAchievement", Integer),
     mysql_charset="utf8mb4",
 )
 
+# new with buddies
+playlog_2p = Table(
+    "mai2_playlog_2p",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    # TODO: ForeignKey to aime_user?
+    Column("userId1", Integer),
+    Column("userId2", Integer),
+    # TODO: ForeignKey to mai2_profile_detail?
+    Column("userName1", String(25)),
+    Column("userName2", String(25)),
+    Column("regionId", Integer),
+    Column("placeId", Integer),
+    Column("user2pPlaylogDetailList", JSON),
+    mysql_charset="utf8mb4",
+)
+
 course = Table(
     "mai2_score_course",
     metadata,
@@ -296,16 +319,16 @@ class Mai2ScoreData(BaseData):
             sql = best_score.select(
                 and_(
                     best_score.c.user == user_id,
-                    (best_score.c.song_id == song_id) if song_id is not None else True,
+                    (best_score.c.musicId == song_id) if song_id is not None else True,
                 )
-            )
+            ).order_by(best_score.c.musicId).order_by(best_score.c.level)
         else:
             sql = best_score_old.select(
                 and_(
                     best_score_old.c.user == user_id,
-                    (best_score_old.c.song_id == song_id) if song_id is not None else True,
+                    (best_score_old.c.musicId == song_id) if song_id is not None else True,
                 )
-            )
+            ).order_by(best_score.c.musicId).order_by(best_score.c.level)
 
         result = await self.execute(sql)
         if result is None:
@@ -343,6 +366,18 @@ class Mai2ScoreData(BaseData):
             self.logger.error(f"put_playlog:  Failed to insert! user_id {user_id} is_dx {is_dx}")
             return None
         return result.lastrowid
+    
+    async def put_playlog_2p(self, user_id: int, playlog_2p_data: Dict) -> Optional[int]:
+        playlog_2p_data["user"] = user_id
+        sql = insert(playlog_2p).values(**playlog_2p_data)
+
+        conflict = sql.on_duplicate_key_update(**playlog_2p_data)
+
+        result = await self.execute(conflict)
+        if result is None:
+            self.logger.error(f"put_playlog_2p:  Failed to insert! user_id {user_id}")
+            return None
+        return result.lastrowid
 
     async def put_course(self, user_id: int, course_data: Dict) -> Optional[int]:
         course_data["user"] = user_id
@@ -363,3 +398,23 @@ class Mai2ScoreData(BaseData):
         if result is None:
             return None
         return result.fetchall()
+
+    async def get_playlogs(self, user_id: int, idx: int = 0, limit: int = 0) -> Optional[List[Row]]:
+        sql = playlog.select(playlog.c.user == user_id)
+        
+        if limit:
+            sql = sql.limit(limit)
+            if idx:
+                sql = sql.offset(idx * limit)
+        
+        result = await self.execute(sql)
+        if result:
+            return result.fetchall()
+
+    async def get_user_playlogs_count(self, aime_id: int) -> Optional[Row]:
+        sql = select(func.count()).where(playlog.c.user == aime_id)
+        result = await self.execute(sql)
+        if result is None:
+            self.logger.warning(f"aime_id {aime_id} has no playlog ")
+            return None
+        return result.scalar()

titles/mai2/templates/css/mai2_style.css

@@ -0,0 +1,195 @@
+.mai2-header {
+    text-align: center;
+}
+
+ul.mai2-navi {
+    list-style-type: none;
+    padding: 0;
+    overflow: hidden;
+    background-color: #333;
+    text-align: center;
+    display: inline-block;
+}
+
+ul.mai2-navi li {
+    display: inline-block;
+}
+
+ul.mai2-navi li a {
+    display: block;
+    color: white;
+    text-align: center;
+    padding: 14px 16px;
+    text-decoration: none;
+}
+
+ul.mai2-navi li a:hover:not(.active) {
+    background-color: #111;
+}
+
+ul.mai2-navi li a.active {
+    background-color: #4CAF50;
+}
+
+ul.mai2-navi li.right {
+    float: right;
+}
+
+@media screen and (max-width: 600px) {
+
+    ul.mai2-navi li.right,
+    ul.mai2-navi li {
+        float: none;
+        display: block;
+        text-align: center;
+    }
+}
+
+table {
+    border-collapse: collapse;
+    border-spacing: 0;
+    border-collapse: separate;
+    overflow: hidden;
+    background-color: #555555;
+
+}
+
+th, td {
+    text-align: left;
+    border: none;
+
+}
+
+th {
+    color: white;
+}
+
+.table-rowdistinct tr:nth-child(even) {
+    background-color: #303030;
+}
+
+.table-rowdistinct tr:nth-child(odd) {
+    background-color: #555555;
+}
+
+caption {
+    text-align: center;
+    color: white;
+    font-size: 18px;
+    font-weight: bold;
+}
+
+.table-large {
+    margin: 16px;
+}
+
+.table-large th,
+.table-large td {
+    padding: 8px;
+}
+
+.table-small {
+    width: 100%;
+    margin: 4px;
+}
+
+.table-small th,
+.table-small td {
+    padding: 2px;
+}
+
+.bg-card {
+    background-color: #555555;
+}
+
+.card-hover {
+    transition: all 0.2s ease-in-out;
+}
+
+.card-hover:hover {
+    transform: scale(1.02);
+}
+
+.basic {
+    color: #28a745;
+    font-weight: bold;
+}
+
+.hard {
+    color: #ffc107;
+
+    font-weight: bold;
+}
+
+.expert {
+    color: #dc3545;
+    font-weight: bold;
+}
+
+.master {
+    color: #dd09e8;
+    font-weight: bold;
+}
+
+.ultimate {
+    color: #000000;
+    font-weight: bold;
+}
+
+.score {
+    color: #ffffff;
+    font-weight: bold;
+}
+
+.rainbow {
+    background: linear-gradient(to right, red, yellow, lime, aqua, blue, fuchsia) 0 / 5em;
+    background-clip: text;
+    -webkit-background-clip: text;
+    -webkit-text-fill-color: transparent;
+    font-weight: bold;
+}
+
+.platinum {
+    color: #FFFF00;
+    font-weight: bold;
+}
+
+.gold {
+    color: #FFFF00;
+    font-weight: bold;
+}
+
+.scrolling-text {
+    overflow: hidden;
+}
+
+.scrolling-text p {
+    white-space: nowrap;
+    display: inline-block;
+
+}
+
+.scrolling-text h6 {
+    white-space: nowrap;
+    display: inline-block;
+
+}
+
+.scrolling-text h5 {
+    white-space: nowrap;
+    display: inline-block;
+
+}
+
+.scrolling {
+    animation: scroll 10s linear infinite;
+}
+
+@keyframes scroll {
+    0% {
+        transform: translateX(100%);
+    }
+    100% {
+        transform: translateX(-100%);
+    }
+}

titles/mai2/templates/mai2_header.jinja

@@ -0,0 +1,17 @@
+<div class="mai2-header">
+  <h1>maimai</h1>
+  <ul class="mai2-navi">
+    <li><a class="nav-link" href="/game/mai2/">PROFILE</a></li>
+    <li><a class="nav-link" href="/game/mai2/playlog/">RECORD</a></li>
+  </ul>
+</div>
+<script>
+  $(document).ready(function () {
+    var currentPath = window.location.pathname;
+    if (currentPath === '/game/mai2/') {
+      $('.nav-link[href="/game/mai2/"]').addClass('active');
+    } else if (currentPath.startsWith('/game/mai2/playlog/')) {
+      $('.nav-link[href="/game/mai2/playlog/"]').addClass('active');
+    }
+  });
+</script>

titles/mai2/templates/mai2_index.jinja

@@ -0,0 +1,134 @@
+{% extends "core/templates/index.jinja" %}
+{% block content %}
+<style>
+  {% include 'titles/mai2/templates/css/mai2_style.css' %}
+</style>
+<div class="container">
+  {% include 'titles/mai2/templates/mai2_header.jinja' %}
+  {% if profile is defined and profile is not none and profile|length > 0 %}
+  <div class="row">
+    <div class="col-lg-8 m-auto mt-3">
+      <div class="card bg-card rounded">
+        <table class="table-large table-rowdistinct">
+          <caption align="top">OVERVIEW</caption>
+          <tr>
+            <th>{{ profile.userName }}</th>
+            <th>
+              <button type="button" class="btn btn-primary btn-sm" data-bs-toggle="modal" data-bs-target="#name_change">Edit</button>
+            </th>
+          </tr>
+          <tr>
+            <td>version:</td>
+            <td>
+              <select name="version" id="version" onChange="changeVersion(this)">
+                {% for ver in versions %}
+                {% if ver == cur_version %}
+                <option value="{{ ver }}" selected>{{ version_list[ver] }}</option>
+                {% else %}
+                <option value="{{ ver }}">{{ version_list[ver] }}</option>
+                {% endif %}
+                {% endfor %}
+              </select>
+              {% if versions | length > 1 %}
+              <p style="margin-block-end: 0;">You have {{ versions | length }} versions.</p>
+              {% endif %}
+            </td>
+          </tr>
+          <tr>
+            <td>Rating:</td>
+            <td>
+              <span class="{% if profile.playerRating >= 15000 %}rainbow{% elif profile.playerRating < 15000 and profile.playerRating >= 14500 %}platinum{% elif profile.playerRating < 14500 and profile.playerRating >=14000 %}platinum{% endif %}">
+                {{ profile.playerRating }}
+              </span>
+              <span>
+                (highest: {{ profile.highestRating }})
+              </span>
+            </td>
+          </tr>
+          <tr>
+            <td>Play Counts:</td>
+            <td>{{ profile.playCount }}</td>
+          </tr>
+          <tr>
+            <td>Last Play Date:</td>
+            <td>{{ profile.lastPlayDate }}</td>
+          </tr>
+        </table>
+      </div>
+    </div>
+    <div class="col-lg-8 m-auto mt-3"> 
+      <div class="card bg-card rounded">
+          <table class="table-large table-rowdistinct">
+            <caption align="top">SCORE</caption>
+            <tr>
+              <td>Total Delux Score:</td>
+              <td>{{ profile.totalDeluxscore }}</td>
+            </tr>
+            <tr>
+              <td>Total Basic Delux Score:</td>
+              <td>{{ profile.totalBasicDeluxscore }}</td>
+            </tr>
+            <tr>
+              <td>Total Advanced Delux Score:</td>
+              <td>{{ profile.totalAdvancedDeluxscore }}</td>
+            </tr>
+            <tr>
+              <td>Total Expert Delux Score:</td>
+              <td>{{ profile.totalExpertDeluxscore }}</td>
+            </tr>
+            <tr>
+              <td>Total Master Delux Score:</td>
+              <td>{{ profile.totalMasterDeluxscore }}</td>
+            </tr>
+            <tr>
+              <td>Total ReMaster Delux Score:</td>
+              <td>{{ profile.totalReMasterDeluxscore }}</td>
+            </tr>
+          </table>
+      </div>
+    </div>
+  </div>
+  {% if error is defined %}
+  {% include "core/templates/widgets/err_banner.jinja" %}
+  {% endif %}
+  {% elif sesh is defined and sesh is not none and sesh.user_id > 0 %}
+  No profile information found for this account.
+  {% else %}
+  Login to view profile information.
+  {% endif %}
+</div>
+<div class="modal fade" id="name_change" tabindex="-1" aria-labelledby="name_change_label" data-bs-theme="dark"
+  aria-hidden="true">
+  <div class="modal-dialog modal-dialog-centered">
+    <div class="modal-content">
+      <div class="modal-header">
+        <h5 class="modal-title">Name change</h5>
+      </div>
+      <div class="modal-body">
+        <form id="new_name_form" action="/game/mai2/update.name" method="post" style="outline: 0;">
+          <label class="form-label" for="new_name">new name:</label>
+          <input class="form-control" aria-describedby="newNameHelp" form="new_name_form" id="new_name"
+            name="new_name" maxlength="14" type="text" required>
+          <div id="newNameHelp" class="form-text">name must be full-width character string.
+          </div>
+        </form>
+      </div>
+      <div class="modal-footer">
+        <input type=submit class="btn btn-primary" type="button" form="new_name_form">
+        <button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Close</button>
+      </div>
+    </div>
+  </div>
+</div>
+<script>
+  function changeVersion(sel) {
+    $.post("/game/mai2/version.change", { version: sel.value })
+      .done(function (data) {
+        location.reload();
+      })
+      .fail(function () {
+        alert("Failed to update version.");
+      });
+  }
+</script>
+{% endblock content %}

titles/mai2/templates/mai2_playlog.jinja

@@ -0,0 +1,225 @@
+{% extends "core/templates/index.jinja" %}
+{% block content %}
+<style>
+  {% include 'titles/mai2/templates/css/mai2_style.css' %}
+</style>
+<div class="container">
+  {% include 'titles/mai2/templates/mai2_header.jinja' %}
+  {% if playlog is defined and playlog is not none %}
+  <div class="row">
+    <h4 style="text-align: center;">Playlog counts: {{ playlog_count }}</h4>
+    {% set rankName = ['D', 'C', 'B', 'BB', 'BBB', 'A', 'AA', 'AAA', 'S', 'S+', 'SS', 'SS+', 'SSS', 'SSS+'] %}
+    {% set difficultyName = ['basic', 'hard', 'expert', 'master', 'ultimate'] %}
+    {% for record in playlog %}
+    <div class="col-lg-6 mt-3">
+      <div class="card bg-card rounded card-hover">
+        <div class="card-header row">
+          <div class="col-8 scrolling-text">
+            <h5 class="card-text"> {{ record.title }} </h5>
+            <br>
+            <h6 class="card-text"> {{ record.artist }} </h6>
+          </div>
+          <div class="col-4">
+            <h6 class="card-text">{{ record.raw.userPlayDate }}</h6>
+            <h6 class="card-text">TRACK {{ record.raw.trackNo }}</h6>
+          </div>
+        </div>
+        <div class="card-body row">
+          <div class="col-3" style="text-align: center;">
+            <h4 class="card-text">{{ record.raw.deluxscore }}</h4>
+            <h2>{{ rankName[record.raw.rank] }}</h2>
+            <h6
+              class="{% if record.raw.level == 0 %}basic{% elif record.raw.level == 1 %}advanced{% elif record.raw.level == 2 %}expert{% elif record.raw.level == 3 %}master{% elif record.raw.level == 4 %}remaster{% endif %}">
+              {{ difficultyName[record.raw.level] }}&nbsp&nbsp{{ record.difficulty }}
+            </h6>
+          </div>
+          <div class="col-6" style="text-align: center;">
+            <table class="table-small table-rowdistinc">
+              <tr>
+                <td>CRITICAL PERFECT</td>
+                <td>
+                  Tap: {{ record.raw.tapCriticalPerfect }}<br>
+                  Hold: {{ record.raw.holdCriticalPerfect }}<br>
+                  Slide: {{ record.raw.slideCriticalPerfect }}<br>
+                  Touch: {{ record.raw.touchCriticalPerfect }}<br>
+                  Break: {{ record.raw.breakCriticalPerfect }}
+                </td>
+              </tr>
+              <tr>
+                <td>PERFECT</td>
+                <td>
+                  Tap: {{ record.raw.tapPerfect }}<br>
+                  Hold: {{ record.raw.holdPerfect }}<br>
+                  Slide: {{ record.raw.slidePerfect }}<br>
+                  Touch: {{ record.raw.touchPerfect }}<br>
+                  Break: {{ record.raw.breakPerfect }}
+                </td>
+              </tr>
+              <tr>
+                <td>GREAT</td>
+                <td>
+                  Tap: {{ record.raw.tapGreat }}<br>
+                  Hold: {{ record.raw.holdGreat }}<br>
+                  Slide: {{ record.raw.slideGreat }}<br>
+                  Touch: {{ record.raw.touchGreat }}<br>
+                  Break: {{ record.raw.breakGreat }}
+                </td>
+              </tr>
+              <tr>
+                <td>GOOD</td>
+                <td>
+                  Tap: {{ record.raw.tapGood }}<br>
+                  Hold: {{ record.raw.holdGood }}<br>
+                  Slide: {{ record.raw.slideGood }}<br>
+                  Touch: {{ record.raw.touchGood }}<br>
+                  Break: {{ record.raw.breakGood }}
+                </td>
+              </tr>
+              <tr>
+                <td>MISS</td>
+                <td>
+                  Tap: {{ record.raw.tapMiss }}<br>
+                  Hold: {{ record.raw.holdMiss }}<br>
+                  Slide: {{ record.raw.slideMiss }}<br>
+                  Touch: {{ record.raw.touchMiss }}<br>
+                  Break: {{ record.raw.breakMiss }}
+                </td>
+              </tr>
+            </table>
+          </div>
+          <div class="col-3" style="text-align: center;">
+            {%if record.raw.comboStatus == 1 %}
+            <h6>FULL COMBO</h6>
+            {% endif %}
+            {%if record.raw.comboStatus == 2 %}
+            <h6>FULL COMBO +</h6>
+            {% endif %}
+            {%if record.raw.comboStatus == 3 %}
+            <h6>ALL PERFECT</h6>
+            {% endif %}
+            {%if record.raw.comboStatus == 4 %}
+            <h6>ALL PERFECT +</h6>
+            {% endif %}
+            {%if record.raw.syncStatus == 1 %}
+            <h6>FULL SYNC</h6>
+            {% endif %}
+            {%if record.raw.syncStatus == 2 %}
+            <h6>FULL SYNC +</h6>
+            {% endif %}
+            {%if record.raw.syncStatus == 3 %}
+            <h6>FULL SYNC DX</h6>
+            {% endif %}
+            {%if record.raw.syncStatus == 4 %}
+            <h6>FULL SYNC DX +</h6>
+            {% endif %}
+            {%if record.raw.isAchieveNewRecord == 1 or record.raw.isDeluxscoreNewRecord == 1 %}
+            <h6>NEW RECORD</h6>
+            {% endif %}
+          </div>
+        </div>
+      </div>
+    </div>
+    {% endfor %}
+  </div>
+  {% set playlog_pages = playlog_count // 20 + 1 %}
+  {% elif sesh is defined and sesh is not none and sesh.user_id > 0 %}
+  No Playlog information found for this account.
+  {% else %}
+  Login to view profile information.
+  {% endif %}
+</div>
+<footer class="navbar-fixed-bottom">
+  <nav aria-label="Playlog page navigation">
+    <ul class="pagination justify-content-center mt-3">
+      <li class="page-item"><a id="prev_page" class="page-link" href="#">Previous</a></li>
+      <li class="page-item"><a id="first_page" class="page-link" href="/game/mai2/playlog/">1</a></li>
+      <li class="page-item"><a id="prev_3_page" class="page-link" href="">...</a></li>
+      <li class="page-item"><a id="front_page" class="page-link" href="">2</a></li>
+      <li class="page-item"><a id="cur_page" class="page-link active" href="">3</a></li>
+      <li class="page-item"><a id="back_page" class="page-link" href="">4</a></li>
+      <li class="page-item"><a id="next_3_page" class="page-link" href="">...</a></li>
+      <li class="page-item"><a id="last_page" class="page-link" href="/game/mai2/playlog/{{ playlog_pages }}">{{
+          playlog_pages }}</a></li>
+      <li class="page-item"><a id="next_page" class="page-link" href="#">Next</a></li>
+      &nbsp
+    </ul>
+  </nav>
+  <div class="row">
+    <div class="col-5"></div>
+    <div class="col-2">
+      <div class="input-group rounded">
+        <input id="page_input" type="text" class="form-control" placeholder="go to page">
+        <span class="input-group-btn">
+          <button id="go_button" class="btn btn-light" type="button">
+            Go!
+          </button>
+        </span>
+      </div>
+    </div>
+    <div class="col-5"></div>
+  </div>
+</footer>
+
+<script>
+  $(document).ready(function () {
+    $('.scrolling-text p, .scrolling-text h1, .scrolling-text h2, .scrolling-text h3, .scrolling-text h4, .scrolling-text h5, .scrolling-text h6').each(function () {
+      var parentWidth = $(this).parent().width();
+      var elementWidth = $(this).outerWidth();
+      var elementWidthWithPadding = $(this).outerWidth(true);
+
+      if (elementWidthWithPadding > parentWidth) {
+        $(this).addClass('scrolling');
+      }
+    });
+
+    var currentUrl = window.location.pathname;
+    var currentPage = parseInt(currentUrl.split('/').pop());
+    var rootUrl = '/game/mai2/playlog/';
+    var playlogPages = {{ playlog_pages }};
+  if (Number.isNaN(currentPage)) {
+    currentPage = 1;
+  }
+  $('#cur_page').text(currentPage);
+  $('#prev_page').attr('href', rootUrl + (currentPage - 1))
+  $('#next_page').attr('href', rootUrl + (currentPage + 1))
+  $('#front_page').attr('href', rootUrl + (currentPage - 1))
+  $('#front_page').text(currentPage - 1);
+  $('#back_page').attr('href', rootUrl + (currentPage + 1))
+  $('#back_page').text(currentPage + 1);
+  $('#prev_3_page').attr('href', rootUrl + (currentPage - 3))
+  $('#next_3_page').attr('href', rootUrl + (currentPage + 3))
+  if ((currentPage - 1) < 3) {
+    $('#prev_3_page').hide();
+    if ((currentPage - 1) < 2) {
+      $('#front_page').hide();
+      if (currentPage === 1) {
+        $('#first_page').hide();
+        $('#prev_page').addClass('disabled');
+      }
+    }
+  }
+  if ((playlogPages - currentPage) < 3) {
+    $('#next_3_page').hide();
+    if ((playlogPages - currentPage) < 2) {
+      $('#back_page').hide();
+      if (currentPage === playlogPages) {
+        $('#last_page').hide();
+        $('#next_page').addClass('disabled');
+      }
+    }
+  }
+
+  $('#go_button').click(function () {
+    var pageNumber = parseInt($('#page_input').val());
+
+    if (!Number.isNaN(pageNumber) && pageNumber <= playlogPages && pageNumber >= 0) {
+      var url = '/game/mai2/playlog/' + pageNumber;
+      window.location.href = url;
+    } else {
+      $('#page_input').val('');
+      $('#page_input').attr('placeholder', 'invalid input!');
+    }
+  });
+  });
+</script>
+{% endblock content %}

titles/mai2/universe.py

@@ -1,8 +1,6 @@
 from typing import Any, List, Dict
 from random import randint
 from datetime import datetime, timedelta
-import pytz
-import json
 
 from core.config import CoreConfig
 from titles.mai2.splashplus import Mai2SplashPlus
@@ -14,207 +12,3 @@ class Mai2Universe(Mai2SplashPlus):
     def __init__(self, cfg: CoreConfig, game_cfg: Mai2Config) -> None:
         super().__init__(cfg, game_cfg)
         self.version = Mai2Constants.VER_MAIMAI_DX_UNIVERSE
-
-    async def handle_cm_get_user_preview_api_request(self, data: Dict) -> Dict:
-        p = await self.data.profile.get_profile_detail(data["userId"], self.version)
-        if p is None:
-            return {}
-
-        return {
-            "userName": p["userName"],
-            "rating": p["playerRating"],
-            # hardcode lastDataVersion for CardMaker
-            "lastDataVersion": "1.20.00",
-            # checks if the user is still logged in
-            "isLogin": False,
-            "isExistSellingCard": True,
-        }
-
-    async def handle_cm_get_user_data_api_request(self, data: Dict) -> Dict:
-        # user already exists, because the preview checks that already
-        p = await self.data.profile.get_profile_detail(data["userId"], self.version)
-
-        cards = await self.data.card.get_user_cards(data["userId"])
-        if cards is None or len(cards) == 0:
-            # This should never happen
-            self.logger.error(
-                f"handle_get_user_data_api_request: Internal error - No cards found for user id {data['userId']}"
-            )
-            return {}
-
-        # get the dict representation of the row so we can modify values
-        user_data = p._asdict()
-
-        # remove the values the game doesn't want
-        user_data.pop("id")
-        user_data.pop("user")
-        user_data.pop("version")
-
-        return {"userId": data["userId"], "userData": user_data}
-
-    async def handle_cm_login_api_request(self, data: Dict) -> Dict:
-        return {"returnCode": 1}
-
-    async def handle_cm_logout_api_request(self, data: Dict) -> Dict:
-        return {"returnCode": 1}
-
-    async def handle_cm_get_selling_card_api_request(self, data: Dict) -> Dict:
-        selling_cards = await self.data.static.get_enabled_cards(self.version)
-        if selling_cards is None:
-            return {"length": 0, "sellingCardList": []}
-
-        selling_card_list = []
-        for card in selling_cards:
-            tmp = card._asdict()
-            tmp.pop("id")
-            tmp.pop("version")
-            tmp.pop("cardName")
-            tmp.pop("enabled")
-
-            tmp["startDate"] = datetime.strftime(
-                tmp["startDate"], Mai2Constants.DATE_TIME_FORMAT
-            )
-            tmp["endDate"] = datetime.strftime(
-                tmp["endDate"], Mai2Constants.DATE_TIME_FORMAT
-            )
-            tmp["noticeStartDate"] = datetime.strftime(
-                tmp["noticeStartDate"], Mai2Constants.DATE_TIME_FORMAT
-            )
-            tmp["noticeEndDate"] = datetime.strftime(
-                tmp["noticeEndDate"], Mai2Constants.DATE_TIME_FORMAT
-            )
-
-            selling_card_list.append(tmp)
-
-        return {"length": len(selling_card_list), "sellingCardList": selling_card_list}
-
-    async def handle_cm_get_user_card_api_request(self, data: Dict) -> Dict:
-        user_cards = await self.data.item.get_cards(data["userId"])
-        if user_cards is None:
-            return {"returnCode": 1, "length": 0, "nextIndex": 0, "userCardList": []}
-
-        max_ct = data["maxCount"]
-        next_idx = data["nextIndex"]
-        start_idx = next_idx
-        end_idx = max_ct + start_idx
-
-        if len(user_cards[start_idx:]) > max_ct:
-            next_idx += max_ct
-        else:
-            next_idx = 0
-
-        card_list = []
-        for card in user_cards:
-            tmp = card._asdict()
-            tmp.pop("id")
-            tmp.pop("user")
-
-            tmp["startDate"] = datetime.strftime(
-                tmp["startDate"], Mai2Constants.DATE_TIME_FORMAT
-            )
-            tmp["endDate"] = datetime.strftime(
-                tmp["endDate"], Mai2Constants.DATE_TIME_FORMAT
-            )
-            card_list.append(tmp)
-
-        return {
-            "returnCode": 1,
-            "length": len(card_list[start_idx:end_idx]),
-            "nextIndex": next_idx,
-            "userCardList": card_list[start_idx:end_idx],
-        }
-
-    async def handle_cm_get_user_item_api_request(self, data: Dict) -> Dict:
-        super().handle_get_user_item_api_request(data)
-
-    async def handle_cm_get_user_character_api_request(self, data: Dict) -> Dict:
-        characters = await self.data.item.get_characters(data["userId"])
-
-        chara_list = []
-        for chara in characters:
-            chara_list.append(
-                {
-                    "characterId": chara["characterId"],
-                    # no clue why those values are even needed
-                    "point": 0,
-                    "count": 0,
-                    "level": chara["level"],
-                    "nextAwake": 0,
-                    "nextAwakePercent": 0,
-                    "favorite": False,
-                    "awakening": chara["awakening"],
-                    "useCount": chara["useCount"],
-                }
-            )
-
-        return {
-            "returnCode": 1,
-            "length": len(chara_list),
-            "userCharacterList": chara_list,
-        }
-
-    async def handle_cm_get_user_card_print_error_api_request(self, data: Dict) -> Dict:
-        return {"length": 0, "userPrintDetailList": []}
-
-    async def handle_cm_upsert_user_print_api_request(self, data: Dict) -> Dict:
-        user_id = data["userId"]
-        upsert = data["userPrintDetail"]
-
-        # set a random card serial number
-        serial_id = "".join([str(randint(0, 9)) for _ in range(20)])
-
-        # calculate start and end date of the card
-        start_date = datetime.utcnow()
-        end_date = datetime.utcnow() + timedelta(days=15)
-
-        user_card = upsert["userCard"]
-        await self.data.item.put_card(
-            user_id,
-            user_card["cardId"],
-            user_card["cardTypeId"],
-            user_card["charaId"],
-            user_card["mapId"],
-            # add the correct start date and also the end date in 15 days
-            start_date,
-            end_date,
-        )
-
-        # get the profile extend to save the new bought card
-        extend = await self.data.profile.get_profile_extend(user_id, self.version)
-        if extend:
-            extend = extend._asdict()
-            # parse the selectedCardList
-            # 6 = Freedom Pass, 4 = Gold Pass (cardTypeId)
-            selected_cards: List = extend["selectedCardList"]
-
-            # if no pass is already added, add the corresponding pass
-            if not user_card["cardTypeId"] in selected_cards:
-                selected_cards.insert(0, user_card["cardTypeId"])
-
-            extend["selectedCardList"] = selected_cards
-            await self.data.profile.put_profile_extend(user_id, self.version, extend)
-
-        # properly format userPrintDetail for the database
-        upsert.pop("userCard")
-        upsert.pop("serialId")
-        upsert["printDate"] = datetime.strptime(upsert["printDate"], "%Y-%m-%d")
-
-        await self.data.item.put_user_print_detail(user_id, serial_id, upsert)
-
-        return {
-            "returnCode": 1,
-            "orderId": 0,
-            "serialId": serial_id,
-            "startDate": datetime.strftime(start_date, Mai2Constants.DATE_TIME_FORMAT),
-            "endDate": datetime.strftime(end_date, Mai2Constants.DATE_TIME_FORMAT),
-        }
-
-    async def handle_cm_upsert_user_printlog_api_request(self, data: Dict) -> Dict:
-        return {
-            "returnCode": 1,
-            "orderId": 0,
-            "serialId": data["userPrintlog"]["serialId"],
-        }
-
-    async def handle_cm_upsert_buy_card_api_request(self, data: Dict) -> Dict:
-        return {"returnCode": 1}

titles/ongeki/base.py

@@ -293,7 +293,9 @@ class OngekiBase:
 
     async def handle_upsert_user_gplog_api_request(self, data: Dict) -> Dict:
         user = data["userId"]
-        if user >= 200000000000000:  # Account for guest play
+
+        # If playing as guest, the user ID is or(0x1000000000001, (placeId & 65535) << 32)
+        if user & 0x1000000000001 == 0x1000000000001:
             user = None
 
         await self.data.log.put_gp_log(
@@ -426,9 +428,10 @@ class OngekiBase:
         userTechCountList = []
 
         for tc in utcl:
-            tc.pop("id")
-            tc.pop("user")
-            userTechCountList.append(tc)
+            tmp = tc._asdict()
+            tmp.pop("id")
+            tmp.pop("user")
+            userTechCountList.append(tmp)
 
         return {
             "userId": data["userId"],
@@ -497,7 +500,7 @@ class OngekiBase:
         }
 
     async def handle_get_user_music_api_request(self, data: Dict) -> Dict:
-        song_list = self.util_generate_music_list(data["userId"])
+        song_list = await self.util_generate_music_list(data["userId"])
         max_ct = data["maxCount"]
         next_idx = data["nextIndex"]
         start_idx = next_idx
@@ -944,6 +947,12 @@ class OngekiBase:
         upsert = data["upsertUserAll"]
         user_id = data["userId"]
 
+        if user_id & 0x1000000000001 == 0x1000000000001:
+            place_id = int(user_id) & 0xFFFC00000000
+            
+            self.logger.info("Guest play from place ID %d, ignoring.", place_id)
+            return {"returnCode": 1, "apiName": "UpsertUserAllApi"}
+
         # The isNew fields are new as of Red and up. We just won't use them for now.
 
         if "userData" in upsert and len(upsert["userData"]) > 0:
@@ -1067,6 +1076,24 @@ class OngekiBase:
         if "userKopList" in upsert:
             for x in upsert["userKopList"]:
                 await self.data.profile.put_kop(user_id, x)
+            
+        for rating_type in {
+            "userRatingBaseBestList",
+            "userRatingBaseBestNewList",
+            "userRatingBaseHotList",
+            "userRatingBaseNextList",
+            "userRatingBaseNextNewList",
+            "userRatingBaseHotNextList",
+        }:
+            if rating_type not in upsert:
+                continue
+
+            await self.data.profile.put_profile_rating(
+                user_id,
+                self.version,
+                rating_type,
+                upsert[rating_type],
+            )
 
         return {"returnCode": 1, "apiName": "upsertUserAll"}
 

titles/ongeki/bright.py

@@ -144,7 +144,7 @@ class OngekiBright(OngekiBase):
         }
 
     async def handle_cm_get_user_item_api_request(self, data: Dict) -> Dict:
-        return self.handle_get_user_item_api_request(data)
+        return await self.handle_get_user_item_api_request(data)
 
     async def handle_cm_get_user_gacha_supply_api_request(self, data: Dict) -> Dict:
         # not used for now? not sure what it even does

titles/ongeki/frontend.py

@@ -31,7 +31,8 @@ class OngekiFrontend(FE_Base):
     
     def get_routes(self) -> List[Route]:
         return [
-            Route("/", self.render_GET)
+            Route("/", self.render_GET),
+            Route("/version.change", self.render_POST, methods=['POST'])
         ]
 
     async def render_GET(self, request: Request) -> bytes:
@@ -44,7 +45,7 @@ class OngekiFrontend(FE_Base):
 
         self.version = usr_sesh.ongeki_version
         if usr_sesh.user_id > 0:
-            profile_data =self.data.profile.get_profile_data(usr_sesh.user_id, self.version)
+            profile_data =await self.data.profile.get_profile_data(usr_sesh.user_id, self.version)
             rival_list = await self.data.profile.get_rivals(usr_sesh.user_id)
             rival_data = {
                 "userRivalList": rival_list,
@@ -64,34 +65,39 @@ class OngekiFrontend(FE_Base):
                 version_list=self.version_list,
                 version=self.version,
                 sesh=vars(usr_sesh)
-            ))
+            ), media_type="text/html; charset=utf-8")
         else:
             return RedirectResponse("/gate/", 303)
     
     async def render_POST(self, request: Request):
-        uri = request.uri.decode()
+        uri = request.url.path
+        frm = await request.form()
         usr_sesh = self.validate_session(request)
         if not usr_sesh:
             usr_sesh = UserSession()
 
         if usr_sesh.user_id > 0:
             if uri == "/game/ongeki/rival.add":
-                rival_id = request.args[b"rivalUserId"][0].decode()
+                rival_id = frm.get("rivalUserId")
                 await self.data.profile.put_rival(usr_sesh.user_id, rival_id)
                 # self.logger.info(f"{usr_sesh.user_id} added a rival")
                 return RedirectResponse(b"/game/ongeki/", 303)
             
             elif uri == "/game/ongeki/rival.delete":
-                rival_id = request.args[b"rivalUserId"][0].decode()
+                rival_id = frm.get("rivalUserId")
                 await self.data.profile.delete_rival(usr_sesh.user_id, rival_id)
                 # self.logger.info(f"{response}")
                 return RedirectResponse(b"/game/ongeki/", 303)
             
             elif uri == "/game/ongeki/version.change":
-                ongeki_version=request.args[b"version"][0].decode()
+                ongeki_version=frm.get("version")
                 if(ongeki_version.isdigit()):
                     usr_sesh.ongeki_version=int(ongeki_version)
-                return RedirectResponse("/game/ongeki/", 303)
+                enc = self.encode_session(usr_sesh)
+                resp = RedirectResponse("/game/ongeki/", 303)
+                resp.delete_cookie('ARTEMIS_SESH')
+                resp.set_cookie('ARTEMIS_SESH', enc)
+                return resp
             
             else:
                 Response("Something went wrong", status_code=500)

titles/ongeki/index.py

@@ -139,11 +139,10 @@ class OngekiServlet(BaseServlet):
             t_port = f":{title_port_int}" if title_port_int != 80 else ""
 
         return (
-            f"{proto}://{self.core_cfg.title.hostname}{t_port}/{game_code}/{game_ver}/",
-            f"{self.core_cfg.title.hostname}{t_port}/",
+            f"{proto}://{self.core_cfg.server.hostname}{t_port}/{game_code}/{game_ver}/",
+            f"{self.core_cfg.server.hostname}{t_port}/",
         )
 
-
     async def render_POST(self, request: Request) -> bytes:
         endpoint: str = request.path_params.get('endpoint', '')
         version: int = request.path_params.get('version', 0)
@@ -169,7 +168,7 @@ class OngekiServlet(BaseServlet):
             internal_ver = OngekiConstants.VER_ONGEKI_RED_PLUS
         elif version >= 130 and version < 135:  # Bright
             internal_ver = OngekiConstants.VER_ONGEKI_BRIGHT
-        elif version >= 135 and version < 140:  # Bright Memory
+        elif version >= 135 and version < 145:  # Bright Memory
             internal_ver = OngekiConstants.VER_ONGEKI_BRIGHT_MEMORY
 
         if all(c in string.hexdigits for c in endpoint) and len(endpoint) == 32:

titles/ongeki/schema/profile.py

@@ -246,6 +246,26 @@ rival = Table(
     mysql_charset="utf8mb4",
 )
 
+rating = Table(
+    "ongeki_profile_rating",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column("version", Integer, nullable=False),
+    Column("type", String(255), nullable=False),
+    Column("index", Integer, nullable=False),
+    Column("musicId", Integer),
+    Column("difficultId", Integer),
+    Column("romVersionCode", Integer),
+    Column("score", Integer),
+    UniqueConstraint("user", "version", "type", "index", name="ongeki_profile_rating_best_uk"),
+    mysql_charset="utf8mb4",
+)
+
 
 class OngekiProfileData(BaseData):
     def __init__(self, cfg: CoreConfig, conn: Connection) -> None:
@@ -508,10 +528,35 @@ class OngekiProfileData(BaseData):
             )
             return None
         return result.lastrowid
+
     async def delete_rival(self, aime_id: int, rival_id: int) -> Optional[int]:
         sql = delete(rival).where(rival.c.user==aime_id, rival.c.rivalUserId==rival_id)
         result = await self.execute(sql)
         if result is None:
             self.logger.error(f"delete_rival: failed to delete! aime_id: {aime_id}, rival_id: {rival_id}")
         else:
-            return result.rowcount
+            return result.rowcount
+    
+    async def put_profile_rating(
+        self,
+        aime_id: int,
+        version: int,
+        rating_type: str,
+        rating_data: List[Dict],
+    ):
+        inserted_values = [
+            {"user": aime_id, "version": version, "type": rating_type, "index": i, **x}
+            for (i, x) in enumerate(rating_data)
+        ]
+        sql = insert(rating).values(inserted_values)
+        update_dict = {x.name: x for x in sql.inserted if x.name != "id"}
+        sql = sql.on_duplicate_key_update(**update_dict)
+        result = await self.execute(sql)
+
+        if result is None:
+            self.logger.warn(
+                f"put_profile_rating_{rating_type}: Could not insert rating entries, aime_id: {aime_id}",
+            )
+            return
+        
+        return result.lastrowid

titles/ongeki/schema/score.py

@@ -30,7 +30,7 @@ score_best = Table(
     Column("isFullCombo", Boolean, nullable=False),
     Column("isAllBreake", Boolean, nullable=False),
     Column("isLock", Boolean, nullable=False),
-    Column("clearStatus", Boolean, nullable=False),
+    Column("clearStatus", Integer, nullable=False),
     Column("isStoryWatched", Boolean, nullable=False),
     Column("platinumScoreMax", Integer),
     UniqueConstraint("user", "musicId", "level", name="ongeki_best_score_uk"),
@@ -129,7 +129,13 @@ tech_count = Table(
 
 class OngekiScoreData(BaseData):
     async def get_tech_count(self, aime_id: int) -> Optional[List[Dict]]:
-        return []
+        sql = select(tech_count).where(tech_count.c.user == aime_id)
+        
+        result = await self.execute(sql)
+        
+        if result is None:
+            return None
+        return result.fetchall()
 
     async def put_tech_count(self, aime_id: int, tech_count_data: Dict) -> Optional[int]:
         tech_count_data["user"] = aime_id

titles/pokken/base.py

@@ -1,9 +1,7 @@
-from datetime import datetime, timedelta
+from datetime import datetime
 import json, logging
 from typing import Any, Dict, List
-import random
 
-from core.data import Data
 from core import CoreConfig
 from .config import PokkenConfig
 from .proto import jackal_pb2
@@ -18,7 +16,6 @@ class PokkenBase:
         self.version = 0
         self.logger = logging.getLogger("pokken")
         self.data = PokkenData(core_cfg)
-        self.SUPPORT_SET_NONE = 4294967295
 
     async def handle_noop(self, request: Any) -> bytes:
         res = jackal_pb2.Response()
@@ -38,7 +35,30 @@ class PokkenBase:
         res = jackal_pb2.Response()
         res.result = 1
         res.type = jackal_pb2.MessageType.REGISTER_PCB
-        self.logger.info(f"Register PCB {request.register_pcb.pcb_id}")
+        pcbid = request.register_pcb.pcb_id
+        if not pcbid.isdigit() or len(pcbid) != 12 or \
+            not pcbid.startswith(f"{PokkenConstants.SERIAL_IDENT[0]}{PokkenConstants.SERIAL_REGIONS[0]}{PokkenConstants.SERIAL_ROLES[0]}{PokkenConstants.SERIAL_CAB_IDENTS[0]}"):
+            self.logger.warn(f"Bad PCBID {pcbid}")
+            res.result = 0
+            return res
+        
+        netid = PokkenConstants.NETID_PREFIX[0] + pcbid[5:]
+        
+        self.logger.info(f"Register PCB {pcbid} (netID {netid})")
+        
+        minfo = await self.data.arcade.get_machine(netid)
+        
+        if not minfo and not self.core_cfg.server.allow_unregistered_serials:
+            self.logger.warn(f"netID {netid} does not belong to any shop!")
+            res.result = 0
+            return res
+        
+        elif not minfo:
+            self.logger.warn(f"Orphaned netID {netid} allowed to connect")
+            locid = 0
+        
+        else:
+            locid = minfo['arcade']
 
         regist_pcb = jackal_pb2.RegisterPcbResponseData()
         regist_pcb.server_time = int(datetime.now().timestamp())
@@ -46,7 +66,7 @@ class PokkenBase:
             "MatchingServer": {
                 "host": f"https://{self.game_cfg.server.hostname}",
                 "port": self.game_cfg.ports.game,
-                "url": "/SDAK/100/matching",
+                "url": "/pokken/matching",
             },
             "StunServer": {
                 "addr": self.game_cfg.server.stun_server_host,
@@ -56,10 +76,10 @@ class PokkenBase:
                 "addr": self.game_cfg.server.stun_server_host,
                 "port": self.game_cfg.server.stun_server_port,
             },
-            "AdmissionUrl": f"ws://{self.game_cfg.server.hostname}:{self.game_cfg.ports.admission}",
-            "locationId": 123, # FIXME: Get arcade's ID from the database
-            "logfilename": "JackalMatchingLibrary.log",
-            "biwalogfilename": "./biwa.log",
+            "AdmissionUrl": f"ws://{self.game_cfg.server.hostname}:{self.game_cfg.ports.admission}/pokken/admission",
+            "locationId": locid,
+            "logfilename": "J:\\JackalMatchingLibrary.log",
+            "biwalogfilename": "J:\\biwa_log.log",
         }
         regist_pcb.bnp_baseuri = f"{self.core_cfg.server.hostname}/bna"
         regist_pcb.biwa_setting = json.dumps(biwa_setting)
@@ -95,12 +115,11 @@ class PokkenBase:
         res.type = jackal_pb2.MessageType.LOAD_CLIENT_SETTINGS
         settings = jackal_pb2.LoadClientSettingsResponseData()
 
-        # TODO: Make configurable
         settings.money_magnification = 1
         settings.continue_bonus_exp = 100
         settings.continue_fight_money = 100
         settings.event_bonus_exp = 100
-        settings.level_cap = 999
+        settings.level_cap = 100
         settings.op_movie_flag = 0xFFFFFFFF
         settings.lucky_bonus_rate = 1
         settings.fail_support_num = 10
@@ -132,9 +151,13 @@ class PokkenBase:
         res.type = jackal_pb2.MessageType.LOAD_USER
         access_code = request.load_user.access_code
         load_usr = jackal_pb2.LoadUserResponseData()
-        user_id = await self.data.card.get_user_id_from_card(access_code)
+        load_usr.load_hash = 1
+        load_usr.access_code = access_code        
+        load_usr.precedent_release_flag = 0xFFFFFFFF
+        load_usr.cardlock_status = False
+        card = await self.data.card.get_card_by_access_code(access_code)
 
-        if user_id is None and self.game_cfg.server.auto_register:
+        if card is None and self.game_cfg.server.auto_register:
             user_id = await self.data.user.create_user()
             card_id = await self.data.card.create_card(user_id, access_code)
 
@@ -142,54 +165,34 @@ class PokkenBase:
                 f"Register new card {access_code} (UserId {user_id}, CardId {card_id})"
             )
 
-        elif user_id is None:
+        elif card is None:
             self.logger.info(f"Registration of card {access_code} blocked!")
             res.load_user.CopyFrom(load_usr)
             return res.SerializeToString()
+        
+        else:
+            user_id = card['user']
+            card_id = card['id']
 
         """ 
-        TODO: Add repeated values
-        tutorial_progress_flag
-        rankmatch_progress
+        TODO: Unlock all supports? Probably
         support_pokemon_list
-        support_set_1
-        support_set_2
-        support_set_3
-        aid_skill_list
-        achievement_flag
-        event_achievement_flag
-        event_achievement_param
         """
         profile = await self.data.profile.get_profile(user_id)
         load_usr.commidserv_result = 1
-        load_usr.load_hash = 1
-        load_usr.cardlock_status = False
         load_usr.banapass_id = user_id
-        load_usr.access_code = access_code
-        load_usr.precedent_release_flag = 0xFFFFFFFF
 
-        if profile is None:
+        if profile is None or profile['trainer_name'] is None:
             profile_id = await self.data.profile.create_profile(user_id)
+            self.logger.info(f"Create new profile {profile_id} for user {user_id}")
             profile_dict = {"id": profile_id, "user": user_id}
             pokemon_data = []
-            tutorial_progress = []
-            rankmatch_progress = []
-            achievement_flag = []
-            event_achievement_flag = []
-            event_achievement_param = []
             load_usr.new_card_flag = True
 
         else:
             profile_dict = {k: v for k, v in profile._asdict().items() if v is not None}
-            self.logger.info(
-                f"Card-in user {user_id} (Trainer name {profile_dict.get('trainer_name', '')})"
-            )
+            self.logger.info(f"Card-in user {user_id} (Trainer name {profile_dict.get('trainer_name', '')})")
             pokemon_data = await self.data.profile.get_all_pokemon_data(user_id)
-            tutorial_progress = []
-            rankmatch_progress = []
-            achievement_flag = []
-            event_achievement_flag = []
-            event_achievement_param = []
             load_usr.new_card_flag = False
 
         load_usr.navi_newbie_flag = profile_dict.get("navi_newbie_flag", True)
@@ -201,9 +204,9 @@ class PokkenBase:
         load_usr.trainer_name = profile_dict.get(
             "trainer_name", f"Newb{str(user_id).zfill(4)}"
         )
-        load_usr.trainer_rank_point = profile_dict.get("trainer_rank_point", 0)
-        load_usr.wallet = profile_dict.get("wallet", 0)
-        load_usr.fight_money = profile_dict.get("fight_money", 0)
+        load_usr.trainer_rank_point = profile_dict.get("trainer_rank_point", 0) # determines rank
+        load_usr.wallet = profile_dict.get("wallet", 0) # pg count
+        load_usr.fight_money = profile_dict.get("fight_money", 0) # ?
         load_usr.score_point = profile_dict.get("score_point", 0)
         load_usr.grade_max_num = profile_dict.get("grade_max_num", 0)
         load_usr.extra_counter = profile_dict.get("extra_counter", 0)
@@ -218,18 +221,18 @@ class PokkenBase:
         load_usr.rank_event = profile_dict.get("rank_event", 0)
         load_usr.awake_num = profile_dict.get("awake_num", 0)
         load_usr.use_support_num = profile_dict.get("use_support_num", 0)
-        load_usr.rankmatch_flag = profile_dict.get("rankmatch_flag", 0)
+        load_usr.rankmatch_flag = profile_dict.get("rankmatch_flag", 0) # flags that next rank match will be rank up
         load_usr.rankmatch_max = profile_dict.get("rankmatch_max", 0)
         load_usr.rankmatch_success = profile_dict.get("rankmatch_success", 0)
         load_usr.beat_num = profile_dict.get("beat_num", 0)
-        load_usr.title_text_id = profile_dict.get("title_text_id", 0)
-        load_usr.title_plate_id = profile_dict.get("title_plate_id", 0)
-        load_usr.title_decoration_id = profile_dict.get("title_decoration_id", 0)
+        load_usr.title_text_id = profile_dict.get("title_text_id", 2)
+        load_usr.title_plate_id = profile_dict.get("title_plate_id", 1)
+        load_usr.title_decoration_id = profile_dict.get("title_decoration_id", 1)
         load_usr.navi_trainer = profile_dict.get("navi_trainer", 0)
         load_usr.navi_version_id = profile_dict.get("navi_version_id", 0)
         load_usr.aid_skill = profile_dict.get("aid_skill", 0)
-        load_usr.comment_text_id = profile_dict.get("comment_text_id", 0)
-        load_usr.comment_word_id = profile_dict.get("comment_word_id", 0)
+        load_usr.comment_text_id = profile_dict.get("comment_text_id", 1)
+        load_usr.comment_word_id = profile_dict.get("comment_word_id", 1)
         load_usr.latest_use_pokemon = profile_dict.get("latest_use_pokemon", 0)
         load_usr.ex_ko_num = profile_dict.get("ex_ko_num", 0)
         load_usr.wko_num = profile_dict.get("wko_num", 0)
@@ -237,11 +240,11 @@ class PokkenBase:
         load_usr.cool_ko_num = profile_dict.get("cool_ko_num", 0)
         load_usr.perfect_ko_num = profile_dict.get("perfect_ko_num", 0)
         load_usr.record_flag = profile_dict.get("record_flag", 0)
-        load_usr.site_register_status = profile_dict.get("site_register_status", 0)
+        load_usr.site_register_status = profile_dict.get("site_register_status", 1)
         load_usr.continue_num = profile_dict.get("continue_num", 0)
 
         load_usr.avatar_body = profile_dict.get("avatar_body", 0)
-        load_usr.avatar_gender = profile_dict.get("avatar_gender", 0)
+        load_usr.avatar_gender = profile_dict.get("avatar_gender", 1)
         load_usr.avatar_background = profile_dict.get("avatar_background", 0)
         load_usr.avatar_head = profile_dict.get("avatar_head", 0)
         load_usr.avatar_battleglass = profile_dict.get("avatar_battleglass", 0)
@@ -283,6 +286,31 @@ class PokkenBase:
                 pkm.bp_point_sp = pkmn_d.get('bp_point_sp', 0)
 
                 load_usr.pokemon_data.append(pkm)
+        
+        for x in profile_dict.get("tutorial_progress_flag", []):
+            load_usr.tutorial_progress_flag.append(x)
+        
+        for x in profile_dict.get("achievement_flag", []):
+            load_usr.achievement_flag.append(x)
+
+        for x in profile_dict.get("aid_skill_list", []):
+            load_usr.aid_skill_list.append(x)
+
+        for x in profile_dict.get("rankmatch_progress", []):
+            load_usr.rankmatch_progress.append(x)
+
+        for x in profile_dict.get("event_achievement_flag", []):
+            load_usr.event_achievement_flag.append(x)
+
+        for x in profile_dict.get("event_achievement_param", []):
+            load_usr.event_achievement_param.append(x)
+        
+        load_usr.support_set_1.append(profile_dict.get("support_set_1_1", 4294967295))
+        load_usr.support_set_1.append(profile_dict.get("support_set_1_2", 4294967295))
+        load_usr.support_set_2.append(profile_dict.get("support_set_2_1", 4294967295))
+        load_usr.support_set_2.append(profile_dict.get("support_set_2_2", 4294967295))
+        load_usr.support_set_3.append(profile_dict.get("support_set_3_1", 4294967295))
+        load_usr.support_set_3.append(profile_dict.get("support_set_3_2", 4294967295))
 
         res.load_user.CopyFrom(load_usr)
         return res.SerializeToString()
@@ -300,6 +328,8 @@ class PokkenBase:
 
         req = request.save_user
         user_id = req.banapass_id
+
+        self.logger.info(f"Save user data for {user_id}")
         
         tut_flgs: List[int] = []
         ach_flgs: List[int] = []
@@ -339,7 +369,7 @@ class PokkenBase:
         for ach_flg in req.achievement_flag:
             ach_flgs.append(ach_flg)
         
-        await self.data.profile.update_profile_tutorial_flags(user_id, ach_flg)
+        await self.data.profile.update_profile_achievement_flags(user_id, ach_flgs)
 
         for evt_flg in req.event_achievement_flag:
             evt_flgs.append(evt_flg)
@@ -353,18 +383,23 @@ class PokkenBase:
             await self.data.item.add_reward(user_id, reward.get_category_id, reward.get_content_id, reward.get_type_id)
         
         await self.data.profile.add_profile_points(user_id, get_rank_pts, get_money, get_score_pts, grade_max)
+
+        # Inconsistant underscore use AND a typo??
+        await self.data.profile.update_rankmatch_data(user_id, req.rankmatch_flag, req.rank_match_max, req.rank_match_success, req.rank_match_process)
         
         await self.data.profile.update_support_team(user_id, 1, req.support_set_1[0], req.support_set_1[1])
         await self.data.profile.update_support_team(user_id, 2, req.support_set_2[0], req.support_set_2[1])
         await self.data.profile.update_support_team(user_id, 3, req.support_set_3[0], req.support_set_3[1])
 
         await self.data.profile.put_pokemon(user_id, mon.char_id, mon.illustration_book_no, mon.bp_point_atk, mon.bp_point_res, mon.bp_point_def, mon.bp_point_sp)
-        await self.data.profile.add_pokemon_xp(user_id, mon.char_id, mon.get_pokemon_exp)
+        await self.data.profile.add_pokemon_xp(user_id, mon.illustration_book_no, mon.get_pokemon_exp)
+        await self.data.profile.set_latest_mon(user_id, mon.illustration_book_no)
         
         for x in range(len(battle.play_mode)):
+            self.logger.info(f"Save {PokkenConstants.BATTLE_TYPE(battle.play_mode[x]).name} battle {PokkenConstants.BATTLE_RESULT(battle.result[x]).name} for {user_id} with mon {mon.illustration_book_no}")
             await self.data.profile.put_pokemon_battle_result(
                 user_id, 
-                mon.char_id, 
+                mon.illustration_book_no, 
                 PokkenConstants.BATTLE_TYPE(battle.play_mode[x]), 
                 PokkenConstants.BATTLE_RESULT(battle.result[x])
             )
@@ -391,7 +426,6 @@ class PokkenBase:
             last_evt
         )
 
-
         return res.SerializeToString()
 
     async def handle_save_ingame_log(self, data: jackal_pb2.Request) -> bytes:
@@ -419,6 +453,13 @@ class PokkenBase:
     async def handle_matching_is_matching(
         self, data: Dict = {}, client_ip: str = "127.0.0.1"
     ) -> Dict:
+        """
+                "sessionId":"12345678",
+                "A":{
+                    "pcb_id": data["data"]["must"]["pcb_id"],
+                    "gip": client_ip
+                },
+        """
         return {
             "data": {
                 "sessionId":"12345678",
@@ -435,6 +476,11 @@ class PokkenBase:
     ) -> Dict:
         return {}
 
+    async def handle_matching_obtain_matching(
+        self, data: Dict = {}, client_ip: str = "127.0.0.1"
+    ) -> Dict:
+        return {}
+
     async def handle_admission_noop(self, data: Dict, req_ip: str = "127.0.0.1") -> Dict:
         return {}
     

titles/pokken/frontend.py

@@ -74,7 +74,7 @@ class PokkenFrontend(FE_Base):
             profile=pf,
             error=err,
             success=succ
-        ))
+        ), media_type="text/html; charset=utf-8")
     
     async def change_name(self, request: Request) -> RedirectResponse:
         usr_sesh = self.validate_session(request)

titles/pokken/schema/profile.py

@@ -1,11 +1,11 @@
 from typing import Optional, Dict, List, Union
 from sqlalchemy import Table, Column, UniqueConstraint, PrimaryKeyConstraint, and_, case
-from sqlalchemy.types import Integer, String, TIMESTAMP, Boolean, JSON
+from sqlalchemy.types import Integer, String, TIMESTAMP, Boolean, JSON, INTEGER
 from sqlalchemy.schema import ForeignKey
 from sqlalchemy.sql import func, select, update, delete
 from sqlalchemy.sql.functions import coalesce
 from sqlalchemy.engine import Row
-from sqlalchemy.dialects.mysql import insert
+from sqlalchemy.dialects.postgresql import insert
 
 from core.data.schema import BaseData, metadata
 from ..const import PokkenConstants
@@ -16,13 +16,8 @@ profile = Table(
     "pokken_profile",
     metadata,
     Column("id", Integer, primary_key=True, nullable=False),
-    Column(
-        "user",
-        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
-        nullable=False,
-        unique=True,
-    ),
-    Column("trainer_name", String(16)),  # optional
+    Column("user", Integer, ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"), nullable=False, unique=True),
+    Column("trainer_name", String(14)),  # optional
     Column("home_region_code", Integer),
     Column("home_loc_name", String(255)),
     Column("pref_code", Integer),
@@ -105,20 +100,15 @@ profile = Table(
     Column("battle_num_vs_cpu", Integer),  # 2
     Column("win_cpu", Integer),
     Column("battle_num_tutorial", Integer),  # 1?
-    mysql_charset="utf8mb4",
 )
 
 pokemon_data = Table(
-    "pokken_pokemon_data",
+    "pokken_pokemon",
     metadata,
     Column("id", Integer, primary_key=True, nullable=False),
-    Column(
-        "user",
-        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
-        nullable=False,
-    ),
-    Column("char_id", Integer, nullable=False),
-    Column("illustration_book_no", Integer),
+    Column("user", Integer, ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"), nullable=False),
+    Column("char_id", Integer),
+    Column("illustration_book_no", Integer, nullable=False), # This is the fucking pokedex number????
     Column("pokemon_exp", Integer),
     Column("battle_num_vs_wan", Integer),  # 4?
     Column("win_vs_wan", Integer),
@@ -132,8 +122,7 @@ pokemon_data = Table(
     Column("bp_point_res", Integer),
     Column("bp_point_def", Integer),
     Column("bp_point_sp", Integer),
-    UniqueConstraint("user", "char_id", name="pokken_pokemon_data_uk"),
-    mysql_charset="utf8mb4",
+    UniqueConstraint("user", "illustration_book_no", name="pokken_pokemon_uk"),
 )
 
 
@@ -157,8 +146,8 @@ class PokkenProfileData(BaseData):
         return result.lastrowid
 
     async def set_profile_name(self, user_id: int, new_name: str, gender: Union[int, None] = None) -> None:
-        sql = update(profile).where(profile.c.user == user_id).values(
-            trainer_name=new_name,
+        sql = profile.update(profile.c.user == user_id).values(
+            trainer_name=new_name if new_name else profile.c.trainer_name,
             avatar_gender=gender if gender is not None else profile.c.avatar_gender
         )
         result = await self.execute(sql)
@@ -179,12 +168,12 @@ class PokkenProfileData(BaseData):
         aid_skill: int,
         last_evt: int
     ) -> None:
-        sql = update(profile).where(profile.c.user == user_id).values(
+        sql = profile.update(profile.c.user == user_id).values(
             extra_counter=extra_counter,
             event_reward_get_flag=evt_reward_get_flg,
-            total_play_days=total_play_days,
-            awake_num=awake_num,
-            use_support_num=use_support_ct,
+            total_play_days=coalesce(profile.c.total_play_days, 0) + total_play_days,
+            awake_num=coalesce(profile.c.awake_num, 0) + awake_num,
+            use_support_num=coalesce(profile.c.use_support_num, 0) + use_support_ct,
             beat_num=beat_num,
             aid_skill=aid_skill,
             last_play_event_id=last_evt
@@ -195,7 +184,7 @@ class PokkenProfileData(BaseData):
             self.logger.error(f"Failed to put extra data for user {user_id}")
 
     async def update_profile_tutorial_flags(self, user_id: int, tutorial_flags: List) -> None:
-        sql = update(profile).where(profile.c.user == user_id).values(
+        sql = profile.update(profile.c.user == user_id).values(
             tutorial_progress_flag=tutorial_flags,
         )
         result = await self.execute(sql)
@@ -205,7 +194,7 @@ class PokkenProfileData(BaseData):
             )
 
     async def update_profile_achievement_flags(self, user_id: int, achievement_flags: List) -> None:
-        sql = update(profile).where(profile.c.user == user_id).values(
+        sql = profile.update(profile.c.user == user_id).values(
             achievement_flag=achievement_flags,
         )
         result = await self.execute(sql)
@@ -215,7 +204,7 @@ class PokkenProfileData(BaseData):
             )
 
     async def update_profile_event(self, user_id: int, event_state: List, event_flags: List[int], event_param: List[int], last_evt: int = None) -> None:
-        sql = update(profile).where(profile.c.user == user_id).values(
+        sql = profile.update(profile.c.user == user_id).values(
             event_state=event_state,
             event_achievement_flag=event_flags,
             event_achievement_param=event_param,
@@ -230,12 +219,16 @@ class PokkenProfileData(BaseData):
     async def add_profile_points(
         self, user_id: int, rank_pts: int, money: int, score_pts: int, grade_max: int
     ) -> None:
-        sql = update(profile).where(profile.c.user == user_id).values(
-            trainer_rank_point = profile.c.trainer_rank_point + rank_pts,
-            fight_money = profile.c.fight_money + money,
-            score_point = profile.c.score_point + score_pts,
+        sql = profile.update(profile.c.user == user_id).values(
+            trainer_rank_point = coalesce(profile.c.trainer_rank_point, 0) + rank_pts,
+            wallet = coalesce(profile.c.wallet, 0) + money,
+            score_point = coalesce(profile.c.score_point, 0) + score_pts,
             grade_max_num = grade_max
         )
+        
+        result = await self.execute(sql)
+        if result is None:
+            return None
 
     async def get_profile(self, user_id: int) -> Optional[Row]:
         sql = profile.select(profile.c.user == user_id)
@@ -248,7 +241,7 @@ class PokkenProfileData(BaseData):
         self,
         user_id: int,
         pokemon_id: int,
-        illust_no: int,
+        pokedex_number: int,
         atk: int,
         res: int,
         defe: int,
@@ -257,7 +250,7 @@ class PokkenProfileData(BaseData):
         sql = insert(pokemon_data).values(
             user=user_id,
             char_id=pokemon_id,
-            illustration_book_no=illust_no,
+            illustration_book_no=pokedex_number,
             pokemon_exp=0,
             battle_num_vs_wan=0,
             win_vs_wan=0,
@@ -274,7 +267,7 @@ class PokkenProfileData(BaseData):
         )
 
         conflict = sql.on_duplicate_key_update(
-            illustration_book_no=illust_no,
+            illustration_book_no=pokedex_number,
             bp_point_atk=pokemon_data.c.bp_point_atk + atk,
             bp_point_res=pokemon_data.c.bp_point_res + res,
             bp_point_def=pokemon_data.c.bp_point_def + defe,
@@ -293,7 +286,7 @@ class PokkenProfileData(BaseData):
         pokemon_id: int,
         xp: int
     ) -> None:
-        sql = update(pokemon_data).where(and_(pokemon_data.c.user==user_id, pokemon_data.c.char_id==pokemon_id)).values(
+        sql = pokemon_data.update(and_(pokemon_data.c.user==user_id, pokemon_data.c.illustration_book_no==pokemon_id)).values(
             pokemon_exp=coalesce(pokemon_data.c.pokemon_exp, 0) + xp
         )
 
@@ -315,6 +308,14 @@ class PokkenProfileData(BaseData):
             return None
         return result.fetchall()
 
+    async def set_latest_mon(self, user_id: int, pokedex_no: int) -> None:
+        sql = profile.update(profile.c.user == user_id).values(
+            latest_use_pokemon=pokedex_no
+        )
+        result = await self.execute(sql)
+        if result is None:
+            self.logger.warning(f"Failed to update user {user_id}'s last used pokemon {pokedex_no}")
+
     async def put_pokemon_battle_result(
         self, user_id: int, pokemon_id: int, match_type: PokkenConstants.BATTLE_TYPE, match_result: PokkenConstants.BATTLE_RESULT
     ) -> None:
@@ -322,7 +323,7 @@ class PokkenProfileData(BaseData):
         Records the match stats (type and win/loss) for the pokemon and profile
         coalesce(pokemon_data.c.win_vs_wan, 0)
         """
-        sql = update(pokemon_data).where(and_(pokemon_data.c.user==user_id, pokemon_data.c.char_id==pokemon_id)).values(
+        sql = pokemon_data.update(and_(pokemon_data.c.user==user_id, pokemon_data.c.char_id==pokemon_id)).values(
             battle_num_tutorial=coalesce(pokemon_data.c.battle_num_tutorial, 0) + 1 if match_type==PokkenConstants.BATTLE_TYPE.TUTORIAL else coalesce(pokemon_data.c.battle_num_tutorial, 0),
             battle_all_num_tutorial=coalesce(pokemon_data.c.battle_all_num_tutorial, 0) + 1 if match_type==PokkenConstants.BATTLE_TYPE.TUTORIAL else coalesce(pokemon_data.c.battle_all_num_tutorial, 0),
 
@@ -353,7 +354,7 @@ class PokkenProfileData(BaseData):
         """
         Records profile stats
         """
-        sql = update(profile).where(profile.c.user==user_id).values(
+        sql = profile.update(profile.c.user==user_id).values(
             ex_ko_num=coalesce(profile.c.ex_ko_num, 0) + exkos,
             wko_num=coalesce(profile.c.wko_num, 0) + wkos,
             timeup_win_num=coalesce(profile.c.timeup_win_num, 0) + timeout_wins,
@@ -367,7 +368,12 @@ class PokkenProfileData(BaseData):
             self.logger.warning(f"Failed to update stats for user {user_id}")
 
     async def update_support_team(self, user_id: int, support_id: int, support1: int = None, support2: int = None) -> None:
-        sql = update(profile).where(profile.c.user==user_id).values(
+        if support1 == 4294967295:
+            support1 = None
+        
+        if support2 == 4294967295:
+            support2 = None
+        sql = profile.update(profile.c.user==user_id).values(
             support_set_1_1=support1 if support_id == 1 else profile.c.support_set_1_1,
             support_set_1_2=support2 if support_id == 1 else profile.c.support_set_1_2,
             support_set_2_1=support1 if support_id == 2 else profile.c.support_set_2_1,
@@ -379,3 +385,15 @@ class PokkenProfileData(BaseData):
         result = await self.execute(sql)
         if result is None:
             self.logger.warning(f"Failed to update support team {support_id} for user {user_id}")
+
+    async def update_rankmatch_data(self, user_id: int, flag: int, rm_max: Optional[int], success: Optional[int], progress: List[int]) -> None:
+        sql = profile.update(profile.c.user==user_id).values(
+            rankmatch_flag=flag,
+            rankmatch_max=rm_max,
+            rankmatch_progress=progress,
+            rankmatch_success=success,
+        )
+
+        result = await self.execute(sql)
+        if result is None:
+            self.logger.warning(f"Failed to update rankmatch data for user {user_id}")