use SQL's limit/offset pagination for nextIndex/maxCount requests (#185)

Instead of retrieving the entire list of items/characters/scores/etc. at once (and even store them in memory), use SQL's `LIMIT ... OFFSET ...` pagination so we only take what we need.

Currently only CHUNITHM uses this, but this will also affect maimai DX and O.N.G.E.K.I. once the PR is ready.

Also snuck in a fix for CHUNITHM/maimai DX's `GetUserRivalMusicApi` to respect the `userRivalMusicLevelList` sent by the client.

### How this works

Say we have a `GetUserCharacterApi` request:

```json
{
    "userId": 10000,
    "maxCount": 700,
    "nextIndex": 0
}
```

Instead of getting the entire character list from the database (which can be very large if the user force unlocked everything), add limit/offset to the query:

```python
select(character)
.where(character.c.user == user_id)
.order_by(character.c.id.asc())
.limit(max_count + 1)
.offset(next_index)
```

The query takes `maxCount + 1` items from the database to determine if there is more items than can be returned:

```python
rows = ...

if len(rows) > max_count:
    # return only max_count rows
    next_index += max_count
else:
    # return everything left
    next_index = -1
```

This has the benefit of not needing to load everything into memory (and also having to store server state, as seen in the [`SCORE_BUFFER` list](2274b42358/titles/chuni/base.py (L13)).)

Reviewed-on: Hay1tsme/artemis#185
Co-authored-by: beerpsi <beerpsi@duck.com>
Co-committed-by: beerpsi <beerpsi@duck.com>
This commit is contained in:
2024-11-16 19:10:29 +00:00
committed by Hay1tsme
parent cb009f6e23
commit 58a5177a30
18 changed files with 1410 additions and 713 deletions

View File

@ -1,4 +1,4 @@
from enum import Enum
from enum import Enum, IntEnum
class ChuniConstants:
@ -81,12 +81,31 @@ class ChuniConstants:
return cls.VERSION_NAMES[ver]
class MapAreaConditionType(Enum):
UNLOCKED = 0
class MapAreaConditionType(IntEnum):
"""Condition types for the GetGameMapAreaConditionApi endpoint. Incomplete.
For the MAP_CLEARED/MAP_AREA_CLEARED/TROPHY_OBTAINED conditions, the conditionId
is the map/map area/trophy.
For the RANK_*/ALL_JUSTICE conditions, the conditionId is songId * 100 + difficultyId.
For example, Halcyon [ULTIMA] would be 173 * 100 + 4 = 17304.
"""
ALWAYS_UNLOCKED = 0
MAP_CLEARED = 1
MAP_AREA_CLEARED = 2
TROPHY_OBTAINED = 3
RANK_SSS = 19
RANK_SSP = 20
RANK_SS = 21
RANK_SP = 22
RANK_S = 23
ALL_JUSTICE = 28
class MapAreaConditionLogicalOperator(Enum):
AND = 1
@ -102,11 +121,36 @@ class AvatarCategory(Enum):
FRONT = 6
BACK = 7
class ItemKind(Enum):
class ItemKind(IntEnum):
NAMEPLATE = 1
FRAME = 2
"""
"Frame" is the background for the gauge/score/max combo display
shown during gameplay. This item cannot be equipped (as of LUMINOUS)
and is hardcoded to the current game's version.
"""
TROPHY = 3
SKILL = 4
TICKET = 5
"""A statue is also a ticket."""
PRESENT = 6
MUSIC_UNLOCK = 7
MAP_ICON = 8
SYSTEM_VOICE = 9
AVATAR_ACCESSORY = 11
SYMBOL_CHAT = 10
AVATAR_ACCESSORY = 11
ULTIMA_UNLOCK = 12
"""This only applies to ULTIMA difficulties that are *not* unlocked by
SS-ing EXPERT+MASTER.
"""
class FavoriteItemKind(IntEnum):
MUSIC = 1
RIVAL = 2
CHARACTER = 3