View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0011389 | Scribus | Language Tools | public | 2013-01-21 13:41 | 2014-07-07 15:42 |
Reporter | JLuc | Assigned To | |||
Priority | normal | Severity | minor | Reproducibility | always |
Status | closed | Resolution | fixed | ||
OS | ubuntu | OS Version | 12.04 | ||
Product Version | 1.4.2 | ||||
Summary | 0011389: spellchecker find wrong mistakes | ||||
Description | With new 1.4.2 on ubuntu 12.04, with libhunspell 1.3.2-4 and hunspell-fr 1:3.3.0-2ubuntu3, The spell checker find wrong errors, all related to "qu'" words : lorsqu', quoiqu', qu' | ||||
Steps To Reproduce | - create new doc - F3 and change default character style langage to french - T so as to create text frame, - insert default french false text (french Lorem Ipsum) in text frame, - F2 and set this text style to default paragraphe style and default character style - select text frame - SHIFT F7 so as to launch spellchecker see the wrong errors. | ||||
Additional Information | Same spellchecker in libreoffice with same pasted text does not raise these non-mistakes. Both scribus and libreoffice use same dictionnary (checked in uninstalling it). Maybe related to some local issue ? | ||||
Tags | No tags attached. | ||||
Patch | |||||
|
|
|
I uploaded test file. |
|
Pasting this in for reference: http://blogs.msdn.com/b/correcteurorthographiqueoffice/archive/2005/12/07/identifying-tokens-is-word-breaking-so-easy.aspx Right now we break words on !isLetter(), but changing that will have all kinds of deep rooted effects, so this is not just a spelling issue. |
|
Basically: int StoryText::endOfWord(int pos) const { int len = length(); while (pos < len) { if(text(pos).isLetter()) ++pos; else break; } return pos; } It makes no consideration of language or script |
|
Amof, there are no problems with other uses of ' as in "c'est" or "L'oiseau" : no error is detected and that's fine. It's only with "anythinQU'xxx" that an error is mistakenly rised. So i'm afraid that changing the ' status for french will cure the "qu'" issue but might result in problems for other situations like "c'est" where as is code does fine now. |
|
It seems, it is a (1) tokenization and/or (2) Unicode (older Hunspell dictionary) problem. (1) Hunspell with the French dictionary recognizes the qu' words only in their full forms, eg. in the command line: $ echo "lorsqu'un" | hunspell -d fr + un The newest French dictionary recognizes the ASCII and the true apostrophe (U+2019 = ’), too: $ echo "lorsqu’un" | hunspell -d fr + un Older hunspell-fr dictionaries don't specify the apostrophe as a WORDCHARS in its affix file, but you can check with the following input with tabulator: $ echo "lorsqu'un " | hunspell -d fr -1 + un) (2) Older Hunspell dictionaries doesn't recognize the words with true (U+2019) apostrophes. It is not problem with LibreOffice, because it converts them to ASCII apostrophes before calling Hunspell. The recent French Hunspell dictionary of LibreOffice has already contained the following line in the ICONV section of the aff file: ICONV ’ ' So if Scribus calls Hunspell with the right tokenization, but U+2019 apostrophes, system-wide French Hunspell dictionary may need an update. Other option to convert U+2019 characters to ASCII characters in Scribus. By the way, spell checking component of LibreOffice just got a similar fix to handle poor man's Unicode f-ligatures and zero width non-joiner Unicode characters (ZWNJ could be important to fix German typography with automatic f-ligature replacements): http://cgit.freedesktop.org/libreoffice/core/commit/?id=98029f1625663609d670f79eea61f7547bfc8123 |
|
> Right now we break words on !isLetter(), but changing that will have all kinds of deep rooted effects, so this is not just a spelling issue. If the apostrophe has to be considered as a word break, you have to patch the dictionary to add the following words (elided forms): - lorsqu - puisqu - jusqu - puisqu - quoiqu - aujourd That would solve the main issues. (Maybe I will add these elided forms in the upstream dictionary.) Then you could also add splitted words from every word that contains an apostrophe. |
|
I added in the Hunspell dictionary the following elided forms: — lorsqu : http://www.dicollecte.org/entry.php?prj=fr&id=222035 — puisqu : http://www.dicollecte.org/entry.php?prj=fr&id=222036 — jusqu : http://www.dicollecte.org/entry.php?prj=fr&id=222037 — quoiqu : http://www.dicollecte.org/entry.php?prj=fr&id=222038 — quelqu : http://www.dicollecte.org/entry.php?prj=fr&id=222039 That should solve most of the issues with apostrophes. I did it only because they are valid elided forms, but I won’t add other word fragments in the dictionary. The new Hunspell dictionary will be released in march or april 2013. |
|
jluc, not able to reproduce on 1.5svn r19322 |
|
Hmmm i still reproduce. Try with following sentence : "le bug se produit mais l'affichage est foireux lorsqu'on met le mot au tout début du frame" |
|
my hunspell-fr version is 1.3.3.0-2ubuntu3 i have to try with newer version wait and see - status for this report is now 'commentaire' |
|
Fixed with new hunspell-fr version found http://www.dicollecte.org/download.php?prj=fr |
|
As fixed we can close it :) |
Date Modified | Username | Field | Change |
---|---|---|---|
2013-01-21 13:41 | JLuc | New Issue | |
2013-01-21 13:57 | JLuc | File Added: test spellchecker.sla | |
2013-01-21 13:57 | JLuc | Note Added: 0029748 | |
2013-02-05 20:37 | cbradney | Note Added: 0029793 | |
2013-02-05 20:38 | cbradney | Note Added: 0029794 | |
2013-02-12 19:28 | JLuc | Note Added: 0029833 | |
2013-02-12 20:37 | nemeth | Note Added: 0029834 | |
2013-02-13 00:27 | olivierr | Note Added: 0029835 | |
2013-02-17 09:45 | olivierr | Note Added: 0029850 | |
2013-02-17 09:50 | olivierr | Note Edited: 0029850 | |
2014-07-06 23:17 | Kunda | Note Added: 0032518 | |
2014-07-06 23:17 | Kunda | Status | new => feedback |
2014-07-07 09:27 | JLuc | Note Added: 0032531 | |
2014-07-07 09:27 | JLuc | Status | feedback => new |
2014-07-07 12:01 | JLuc | Note Added: 0032533 | |
2014-07-07 12:12 | JLuc | Note Added: 0032535 | |
2014-07-07 12:12 | JLuc | Status | new => resolved |
2014-07-07 12:12 | JLuc | Resolution | open => fixed |
2014-07-07 12:12 | JLuc | Assigned To | => JLuc |
2014-07-07 15:42 | FirasH | Note Added: 0032547 | |
2014-07-07 15:42 | FirasH | Status | resolved => closed |
2014-07-07 15:42 | FirasH | Assigned To | JLuc => |