View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0015350 | Scribus | Import / Export | public | 2018-06-05 15:41 | 2019-06-04 20:56 |
Reporter | ralfD | Assigned To | jghali | ||
Priority | normal | Severity | major | Reproducibility | always |
Status | closed | Resolution | fixed | ||
Product Version | 1.5.4.svn | ||||
Fixed in Version | 1.5.5.svn | ||||
Summary | 0015350: xtg import broken for multibyte encoded input files | ||||
Description | xtg import for multibyte encoded input files (UTF-8, UTF-16 etc.) is broken. The function XtgScanner::nextSymbol in file xtgscanner.cpp read characters and feeds them to m_codec->toUnicode which will fail for all mutibyte input. | ||||
Steps To Reproduce | Save the following text to an (UTF-8 encoded) file: x------------------------------[file: sample.xtg]------------------------------------------------------x <v8.00><e9> @mystyle:Fürchtet euch nicht, gössere Kläffer! x----------------------------------------------------------------------------------------------------------x Create a text frame and import the file as a 'Quark Xpress Tags Document'. | ||||
Additional Information | The fix for this would be to switch to Qt's stateful decoder. In a quick/hacky test that does fix the issue. | ||||
Tags | No tags attached. | ||||
Patch | No | ||||
|
Can you please provide a sample xtg file? |
|
Yes, see the following minimal example. |
|
Just to note: lookAhead() suffers from the same shotcomings. To really fix this problem the scanner should operate on characters/on a character buffer and not on bytes/a bytearray. |
|
>> To really fix this problem the scanner should operate on characters/on a character buffer and not on bytes/a bytearray. Yes, that's the easy solution. However the xtg format makes it theoretically possible to switch between encodings inside a single file. The xtg documentation is not clear about that. So I'm not quite sure that we can use a character buffer. |
|
Yes, I was wondering about that myself. The "specs" are rather unspecific about where the encoding tag is allowed (and doesn't even mention compinations of BOM and encoding tag). The whole format seems to be rather ad-hoc (at least when I last looked at it while writing a parser almost 20 years ago ...). But realistically I never encountered a file with more then one encoding marker (n.b.: all those files where exports from Quark XPress or related products). |
Date Modified | Username | Field | Change |
---|---|---|---|
2018-06-05 15:41 | ralfD | New Issue | |
2018-06-05 18:45 | jghali | Note Added: 0045302 | |
2018-06-05 19:45 | ralfD | File Added: multibyte.xtg | |
2018-06-05 19:45 | ralfD | Note Added: 0045303 | |
2018-06-06 08:54 | ralfD | Note Added: 0045306 | |
2018-06-06 10:18 | jghali | Note Added: 0045307 | |
2018-06-06 10:40 | ralfD | Note Added: 0045308 | |
2018-06-17 23:17 | jghali | Assigned To | => jghali |
2018-06-17 23:17 | jghali | Status | new => resolved |
2018-06-17 23:17 | jghali | Resolution | open => fixed |
2018-06-17 23:17 | jghali | Fixed in Version | => 1.5.5.svn |
2019-06-04 20:56 | cbradney | Status | resolved => closed |