View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0010292 | Scribus | Story Editor / Text Frames | public | 2011-10-01 02:56 | 2016-04-11 00:03 |
Reporter | Sherwood Wang | Assigned To | jghali | ||
Priority | normal | Severity | major | Reproducibility | always |
Status | closed | Resolution | fixed | ||
Platform | x86_64 | OS | Debian | OS Version | Squeeze |
Product Version | 1.5.0svn | ||||
Fixed in Version | 1.5.0svn | ||||
Summary | 0010292: [PATCH] CJK formating issue | ||||
Description | People who write in CJK don't add spaces between characters. So Scribus will regard a whole paragraph of CJK characters as one word. It will not break lines between two CJK characters if there is any space at other place. And it can't justify such a line for the same reason. This patch will identify CJK characters and add implicit breakable points between them to correct line-breaking problem add implicit spaces to correct line-justifying problem. | ||||
Steps To Reproduce | Just copy any Chinese text (especially with some spaces in it) into the text frame. | ||||
Tags | No tags attached. | ||||
Patch | |||||
|
cjkpatch (5,419 bytes)
Index: scribus/pageitem_textframe.cpp =================================================================== --- scribus/pageitem_textframe.cpp (revision 16866) +++ scribus/pageitem_textframe.cpp (working copy) @@ -724,8 +724,68 @@ double lineCorr; }; +static int checkCJK(QChar ch) { + unsigned int code = ch.unicode(); + if ( (0x2E80 < code && code < 0x2EFF) || // CJK Radicals Supplement + (0x3000 < code && code < 0x303F) || // CJK Symbols and Punctuation + (0x31C0 < code && code < 0x31EF) || // CJK Strokes + (0x3200 < code && code < 0x32FF) || // Enclosed CJK Letters and Months + (0x3300 < code && code < 0x33FF) || // CJK Compatibility + (0x3400 < code && code < 0x4DBF) || // CJK Unified Ideographs Extension A + (0x4E00 < code && code < 0x9FFF) || // CJK Unified Ideographs + (0xF900 < code && code < 0xFAFF) || // CJK Compatibility Ideographs + (0xFE30 < code && code < 0xFE4F) || // CJK Compatibility Forms + (0x20000 < code && code < 0x2A6DF) || // CJK Unified Ideographs Extension B + (0x2A700 < code && code < 0x2B73F) || // CJK Unified Ideographs Extension C + (0x2B740 < code && code < 0x2B81F) || // CJK Unified Ideographs Extension D + (0x2F800 < code && code < 0x2FA1F) || // CJK Compatibility Ideographs Supplement + (0xFF01 < code && code < 0xFF0F) || + (0xFF1A < code && code < 0xFF20) || + (0xFF58 < code && code < 0xFFDC) || + (code == 0x3000) || + (code == 0x3002) || + (code == 0x201C) || + (code == 0x201D)) + return 1; + else + return 0; +} +static int checkCJKBreakAfter(QChar ch) { + unsigned int code[] = {0x201C, 0xFF08, 0xFF3B, 0xFF5B, 0xFF5F, 0xFF62, 0xFF0D, 0}; + for (int i = 0; code[i]; ++i) + if (code[i] == ch.unicode()) + return 0; + return 1; +} +static int checkCJKBreakBefore(QChar ch) { + unsigned int code[] = + {0x201D, 0x3002, 0xFF01, 0xFF09, 0xFF0C, 0xFF0E, 0xFF1A, + 0xFF1B, 0xFF1F, 0xFF3D, 0xFF5D, 0xFF60, 0xFF63, 0xFF64, 0}; + for (int i = 0; code[i]; ++i) + if (code[i] == ch.unicode()) + return 0; + return 1; +} + +static int implicitSpace(ScText *f, ScText *s) { + if (checkCJK(f->ch) && checkCJK(s->ch)) + return 1; + else + return 0; +} + +static int implicitBreak(ScText *f, ScText *s) { + if (checkCJK(f->ch) && checkCJK(s->ch)) { + if (!checkCJKBreakAfter(f->ch) + || !checkCJKBreakBefore(s->ch)) + return 0; + return 1; + } else + return 0; +} + /// called when line length is known and line is to be justified static void justifyLine(StoryText& itemText, LineSpec& line) { @@ -734,6 +794,8 @@ double spaceNatural = 0; double glyphExtension; double spaceExtension; + int spaceInsertion = 0; + double imSpace = -1; const ParagraphStyle& style(itemText.paragraphStyle(line.firstItem)); @@ -747,28 +809,53 @@ else if ( (itemText.item(sof)->effects() & ScStyle_SuppressSpace) == 0) { spaceNatural += itemText.item(sof)->glyph.wide(); + if (imSpace < 0.0 || imSpace > itemText.item(sof)->glyph.wide()) + imSpace = itemText.item(sof)->glyph.wide(); } + if (sof != line.firstItem && implicitSpace(itemText.item(sof - 1), itemText.item(sof))) { + spaceInsertion += 1; + } } - // decision: prio 1: stretch glyph; prio 2: stretch spaces + imSpace /= 2; + + // decision: prio 1: stretch glyph; prio 2: insert spaces; prio 3: stretch spaces if (line.width < spaceNatural + glyphNatural * style.minGlyphExtension() && spaceNatural > 0) { glyphExtension = style.minGlyphExtension() - 1; spaceExtension = (line.width - glyphNatural * (1+glyphExtension) ) / spaceNatural - 1; + imSpace = 0; } else if (line.width < spaceNatural + glyphNatural * style.maxGlyphExtension() && glyphNatural > 0) { spaceExtension = 0; glyphExtension = (line.width - spaceNatural) / glyphNatural - 1; + imSpace = 0; } else { glyphExtension = style.maxGlyphExtension() - 1; - if (spaceNatural > 0) - spaceExtension = (line.width - glyphNatural * (1+glyphExtension) ) / spaceNatural - 1; - else - spaceExtension = 0; + if (spaceInsertion) { + double remaining = line.width - glyphNatural * (1 + glyphExtension) - spaceNatural; + if (imSpace > 0) { + if (remaining / spaceInsertion < imSpace) { + imSpace = remaining / spaceInsertion; + spaceExtension = 0; + } else { + spaceExtension = (remaining + spaceNatural) / (spaceNatural + spaceInsertion * imSpace) - 1; + imSpace *= spaceExtension + 1; + } + } else { + imSpace = remaining / spaceInsertion; + spaceExtension = 0; + } + } else { + if (spaceNatural > 0) + spaceExtension = (line.width - glyphNatural * (1+glyphExtension) ) / spaceNatural - 1; + else + spaceExtension = 0; + } } double glyphScale = 1 + glyphExtension; @@ -799,6 +886,9 @@ { itemText.item(yof)->glyph.last()->xadvance += wide * spaceExtension; } + if (yof != line.firstItem && implicitSpace(itemText.item(yof - 1), itemText.item(yof))) { + itemText.item(yof - 1)->glyph.last()->xadvance += imSpace; + } } } @@ -1651,6 +1741,10 @@ if ((hl->ch == SpecialChars::COLBREAK) && (Cols > 1)) goNextColumn = true; + if (a != firstInFrame() && implicitBreak(itemText.item(a - 1), itemText.item(a))) { + current.rememberBreak(a - 1, breakPos); + } + // remember possible break // #5783 : comment out "&& !outs" as this make it possible to perform a line break // before a space which contradicts unicode line breaking rules |
|
jiero tested it and said that it is really useful. (and works) the patch looks very straightforward and clean. still, i'm not sure it's a good idea to put such language specific exceptions in the middle of pagitem_textframe... is there any alternative? what i also wonder is: can full OTF support will help "cleanly" solving this issue? ... just wondering... |
|
I think it's inevitable to put some language specific codes into pagitem_textframe. In Latin, a character is a letter. In CJK, a character can be a word. It's almost impossible to layout CJK scripts like layouting Latin scripts. So XeTeX, which is developed by SIL International, adopted a similar method to break lines and add glues with CJK. I have no idea about solving this issue with OTF. As I know, OTF doesn't define anything about lines. It only interests in the relation between characters. |
|
ok, if xatex does it that way, it must be the good way! could you just check if other asiatic languages need a similar check (but with other characters) and eventually implement it, too? we don't have any chinese devoper (yet!) but we would really like to be able to support asiatic languages! |
|
Chinese (including some dialect of it, such as Yue, Kejiahua), Japanese and Korean are similar. Thanks to Unicode, all characters of this style are put into CJK sections, even some special characters are only used in the derivatives of these languages. So problems about them are solved by this patch. I'm interested in Scribus. And I will try to make contributions. |
|
Committed! Many thanks! |
|
@Sherwood Wang: we need testers and test files for ScribusCTL that could support Vertical Text: https://github.com/HOST-Oman/scribus/issues/143 |
Date Modified | Username | Field | Change |
---|---|---|---|
2011-10-01 02:56 | Sherwood Wang | New Issue | |
2011-10-01 02:56 | Sherwood Wang | File Added: cjkpatch | |
2011-10-02 07:11 | ale | Note Added: 0026945 | |
2011-10-03 07:40 | Sherwood Wang | Note Added: 0026948 | |
2011-10-03 07:40 | Sherwood Wang | Note Edited: 0026948 | |
2011-10-03 13:39 | ale | Note Added: 0026950 | |
2011-10-04 17:37 | Sherwood Wang | Note Added: 0026967 | |
2011-10-04 20:54 | jghali | Note Added: 0026971 | |
2011-10-04 20:54 | jghali | Status | new => resolved |
2011-10-04 20:54 | jghali | Fixed in Version | => 1.5.0svn |
2011-10-04 20:54 | jghali | Resolution | open => fixed |
2011-10-04 20:54 | jghali | Assigned To | => jghali |
2011-10-05 20:04 | cbradney | Status | resolved => closed |
2015-09-17 20:08 | Kunda | Category | Story Editor / Text Frames => Story Ed/Txt Frames |
2015-09-17 20:12 | Kunda | Category | Story Ed/Txt Frames => Story Editor / Text Frames |
2016-04-11 00:03 | Kunda | Note Added: 0040035 |