View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0002728ScribusInternalpublic2005-10-16 16:312005-11-05 12:47
ReporterringercAssigned Toringerc 
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionfixed 
Platformx86 LinuxOSFedora CoreOS Version3
Product Version1.3.2cvs 
Fixed in Version1.3.2cvs 
Summary0002728: Fix bad encoding handling in FileLoader, ScribusXML's .sla loading
DescriptionFileLoader and ScribusXML use the old, unsafe util.cpp:loadText(...) helper function. This function produces a corrupt QString that we then corrupt again in such a way as to end up with a valid utf-8 QString.

The original string is corrupt because we shove multibyte utf-8 characters into multiple QString characters. Since QString is a real text string that knows its own encoding, that data appears to be corrupt. A file containing "©" in utf-8 represents that as \xc2\xa9 ; we shove the bytes "\xc2" and "\xa9" into the QString, producing the QString "©" instead of "©" like we intended. This is better explained in the coding style document, and elsewhere on the 'net.

This is inefficient - we convert the QString from a byte stream to a corrupt QSTring, then back to a byte string temporarily when converting back to a QString - and unsafe.

I've converted FileLoader and ScribusXML to use the new loadRawText() helper in util.cpp that uses a QCString to load the data. This reduces the number of conversions we do on the data and eliminates the two-wrongs-make-a-right bug in the load code.

I also changed the file load path for old format docs so they also go through loadRawText(...) instead of being slurped in directly. This solves some issues with Qt mis-detecting the encoding on old documents. Old docs are now always loaded using the local 8-bit encoding.
TagsNo tags attached.
Patch

Relationships

related to 0002215 closedjghali Audit and fix text encoding handling 

Activities

ringerc

2005-10-16 18:09

reporter   ~0007117

Changes are now in CVS.

Issue History

Date Modified Username Field Change
2005-10-16 16:31 ringerc New Issue
2005-10-16 16:35 ringerc Relationship added related to 0002215
2005-10-16 16:36 ringerc Description Updated
2005-10-16 17:44 ringerc Description Updated
2005-10-16 18:09 ringerc Status assigned => resolved
2005-10-16 18:09 ringerc Fixed in Version => 1.3.2cvs
2005-10-16 18:09 ringerc Resolution open => fixed
2005-10-16 18:09 ringerc Note Added: 0007117
2005-11-05 12:47 cbradney Status resolved => closed