Reader - Reader crashes after reading some special characters #55

New Issue

DSRLIN · 2023-10-26T05:57:31Z

DSRLIN commented

2023-10-26 05:57:31 +00:00

Currently, the reader in artemis is using the default parser provided by ElementTree.

This will cause the reader to crash after reading some special characters like &.

e.g. 曲：矢鴇つかさ & 脇眞富（Arte Refact）／歌：星咲あかり(CV：赤尾ひかる)、高瀬梨緒(CV：久保ユリカ)

If we use reader on this line, then it will raise an error.

The solution is to init a parser for each tree_root first.

    parser = etree.XMLParser(recover=True)
    troot = ET.fromstring(f.read(), parser=parser)

I've tested on "Victor Kong & Yukino"、"Jehezukiel (feat.Sagi & KURORAK)"、"矢鴇つかさ & 脇眞富"、"Katzeohr & Spiegel" and other strings and it works fine.

Currently, the reader in artemis is using the default parser provided by ElementTree. This will cause the reader to crash after reading some special characters like &. e.g. 曲：矢鴇つかさ & 脇眞富（Arte Refact）／歌：星咲あかり(CV：赤尾ひかる)、高瀬梨緒(CV：久保ユリカ) If we use reader on this line, then it will raise an error. The solution is to init a parser for each tree_root first. ```python parser = etree.XMLParser(recover=True) troot = ET.fromstring(f.read(), parser=parser) ``` I've tested on "Victor Kong & Yukino"、"Jehezukiel (feat.Sagi & KURORAK)"、"矢鴇つかさ & 脇眞富"、"Katzeohr & Spiegel" and other strings and it works fine.

Midorica commented

2023-11-06 02:47:20 +00:00

I do remember seeing this issue indeed, it happened with Chunithm and Ongeki only i think

Feel free to push a PR and ill give it a test

I do remember seeing this issue indeed, it happened with Chunithm and Ongeki only i think Feel free to push a PR and ill give it a test

DesktopMan commented

2023-11-12 20:46:18 +00:00

I might be seeing the same issue, importing Chunithm SUN SDHD_2.10.01_20220913130916_0. With clean DB:

python read.py --series SDHD --version 13 --binfolder d:\ChunithmSUN --optfolder d:\ChunithmSUN\Option

[2023-11-12 21:42:10] Reader | INFO | Inserted event 1376
Traceback (most recent call last):
  File "D:\services\artemis\read.py", line 139, in <module>
    handler.read()
  File "D:\services\artemis\titles\chuni\read.py", line 41, in read
    self.read_events(f"{dir}/event")
  File "D:\services\artemis\titles\chuni\read.py", line 128, in read_events
    xml_root = ET.fromstring(strdata)
  File "C:\Program Files\Python310\lib\xml\etree\ElementTree.py", line 1342, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: mismatched tag: line 70, column 6

I might be seeing the same issue, importing Chunithm SUN SDHD_2.10.01_20220913130916_0. With clean DB: ``` python read.py --series SDHD --version 13 --binfolder d:\ChunithmSUN --optfolder d:\ChunithmSUN\Option [2023-11-12 21:42:10] Reader | INFO | Inserted event 1376 Traceback (most recent call last): File "D:\services\artemis\read.py", line 139, in <module> handler.read() File "D:\services\artemis\titles\chuni\read.py", line 41, in read self.read_events(f"{dir}/event") File "D:\services\artemis\titles\chuni\read.py", line 128, in read_events xml_root = ET.fromstring(strdata) File "C:\Program Files\Python310\lib\xml\etree\ElementTree.py", line 1342, in XML parser.feed(text) xml.etree.ElementTree.ParseError: mismatched tag: line 70, column 6 ```

Midorica added the

bug

label 2023-11-13 14:54:49 +00:00

Sign in to join this conversation.