View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0008157 | Scribus | Language Tools | public | 2009-06-15 13:20 | 2014-10-24 22:57 |
Reporter | caolan | Assigned To | cbradney | ||
Priority | normal | Severity | feature | Reproducibility | always |
Status | closed | Resolution | won't fix | ||
Product Version | 1.3.5svn | ||||
Summary | 0008157: [PATCH] Support for arbitrary seperately packaged hyphenators | ||||
Description | Scribus includes a copy of a number of hyphenators that were originally created for use by OOo. But there are a lot more hyphenators for a lot more languages available, and e.g. they are generally now packaged separately in Fedora and Debian (see /usr/share/hyphen and hyphen-af through to hyphen-zu at http://fedoraproject.org/wiki/OpenOffice.org/LinguisticComponents) It would be ideal for those distros to be use the system hyphenators to reduce duplication and to allow scribus to support these other languages. A simple link from /usr/share/scribus/dicts to /usr/share/hyphen shows a few problems with that "simple" approach. Attached is a patch that allows this to be at least possible. | ||||
Additional Information | a) scribus/scribus.cpp There's probably no reason that not process links within the /usr/lib/scribus/dicts dir itself, so remove | QDir::NoSymLinks b) scribus/hyphenator.cpp Make this code safe against, e.g. releasing a NULL hyphenator or attempts to load a hyphenator that doesn't exist c) scribus/hysettings.cpp The names of languages that scribus knows about are in a hard-coded table. So to support arbitrary languages which scribus doesn't know about we take the locale string as a fallback in the UI d) scribus/langmgr.cpp This bit is admittedly a bit messy given the existing table of language abbreviations that are in use. But the patch should be good enough for the majority of situations as a starting point. (as an aside, there are some odd entries in LanguageManager::generateLangList, e.g. langList.insert("ga", langPair("Galician", as "ga" is the language code for Irish (Gaelic), and langList.insert("es_LA", langPair("Spanish (Latin)", which suggest Spanish as spoken in Lao by its name) | ||||
Tags | No tags attached. | ||||
Patch | Yes | ||||
2009-06-15 13:20
|
scribus-1.3.5.rc2.arbitraryhyphenators.patch (5,009 bytes)
diff -ru scribus-1.3.5.rc2.orig/scribus/scribus.cpp scribus-1.3.5.rc2/scribus/scribus.cpp --- scribus-1.3.5.rc2.orig/scribus/scribus.cpp 2009-06-15 10:15:07.000000000 +0100 +++ scribus-1.3.5.rc2/scribus/scribus.cpp 2009-06-15 10:15:54.000000000 +0100 @@ -8841,7 +8841,7 @@ //Grab the language abbreviation from it, get the full language text //Insert the name as key and a new string list into the map QString hyphDirName = QDir::convertSeparators(ScPaths::instance().dictDir()); - QDir hyphDir(hyphDirName, "*.dic", QDir::Name, QDir::Files | QDir::NoSymLinks); + QDir hyphDir(hyphDirName, "*.dic", QDir::Name, QDir::Files); if ((hyphDir.exists()) && (hyphDir.count() != 0)) { // LanguageManager langmgr; diff -ru scribus-1.3.5.rc2.orig/scribus/hyphenator.cpp scribus-1.3.5.rc2/scribus/hyphenator.cpp --- scribus-1.3.5.rc2.orig/scribus/hyphenator.cpp 2009-06-15 10:15:19.000000000 +0100 +++ scribus-1.3.5.rc2/scribus/hyphenator.cpp 2009-06-15 12:49:16.000000000 +0100 @@ -61,23 +61,26 @@ Language = PrefsManager::instance()->appPrefs.Language; doc->Language = Language; } + // pfad += ScCore->primaryMainWindow()->Sprachen[Language]; - pfad += lmgr->getHyphFilename( doc->Language, false ); - QFile f(pfad); - if (f.open(QIODevice::ReadOnly)) + QString fileName = lmgr->getHyphFilename( doc->Language, false ); + if (!fileName.isEmpty()) { - QTextStream st(&f); - QString line; - line = st.readLine(); - codec = QTextCodec::codecForName(line.toUtf8()); - f.close(); + pfad += fileName; + QFile f(pfad); + if (f.open(QIODevice::ReadOnly)) + { + QTextStream st(&f); + QString line; + line = st.readLine(); + f.close(); + useAble = true; + } } - else - { - useAble = false; - hdict = NULL; + + if (!useAble) return; - } + QByteArray fn = pfad.toLocal8Bit(); const char * filename = fn.data(); hdict = hnj_hyphen_load(filename); @@ -90,7 +93,8 @@ Hyphenator::~Hyphenator() { - hnj_hyphen_free(hdict); + if (hdict != NULL) + hnj_hyphen_free(hdict); } void Hyphenator::NewDict(const QString& name) @@ -110,22 +114,29 @@ if (hdict != NULL) hnj_hyphen_free(hdict); - pfad += LanguageManager::instance()->getHyphFilename(Language, false) ; - QFile f(pfad); - if (f.open(QIODevice::ReadOnly)) + useAble = false; + hdict = NULL; + + QString fileName = LanguageManager::instance()->getHyphFilename( doc->Language, false ); + if (!fileName.isEmpty()) { - QTextStream st(&f); - QString line; - line = st.readLine(); - codec = QTextCodec::codecForName(line.toUtf8()); - f.close(); + pfad += fileName; + QFile f(pfad); + if (f.open(QIODevice::ReadOnly)) + { + QTextStream st(&f); + QString line; + line = st.readLine(); + codec = QTextCodec::codecForName(line.toUtf8()); + f.close(); + useAble = true; + } } - else - { - useAble = false; - hdict = NULL; + + + if (!useAble) return; - } + QByteArray fn = pfad.toLocal8Bit(); filename = fn.data(); hdict = hnj_hyphen_load(filename); diff -ru scribus-1.3.5.rc2.orig/scribus/hysettings.cpp scribus-1.3.5.rc2/scribus/hysettings.cpp --- scribus-1.3.5.rc2.orig/scribus/hysettings.cpp 2009-06-15 10:15:20.000000000 +0100 +++ scribus-1.3.5.rc2/scribus/hysettings.cpp 2009-06-15 12:53:18.000000000 +0100 @@ -27,7 +27,17 @@ language->setInsertPolicy(QComboBox::InsertAlphabetically); foreach(QString hlang, lmg->hyphLangs()) { - language->addItem( lmg->getLangFromAbbrev(hlang), lmg->getLangFromAbbrev(hlang,false) ); + QString uiname = lmg->getLangFromAbbrev(hlang); + QString lang = lmg->getLangFromAbbrev(hlang,false); + + //Fallback to use lang code as name if otherwise unknown + if (uiname.isEmpty()) + uiname = hlang; + + if (lang.isEmpty()) + lang = hlang; + + language->addItem( uiname, lang ); } buttonExceptAdd->setIcon(QIcon(loadIcon("16/list-add.png"))); diff -ru scribus-1.3.5.rc2.orig/scribus/langmgr.cpp scribus-1.3.5.rc2/scribus/langmgr.cpp --- scribus-1.3.5.rc2.orig/scribus/langmgr.cpp 2009-06-15 10:15:15.000000000 +0100 +++ scribus-1.3.5.rc2/scribus/langmgr.cpp 2009-06-15 12:48:39.000000000 +0100 @@ -315,7 +314,31 @@ { if(langIsAbbreviated) return hyphLangList.value(lang); - return hyphLangList.value(getAbbrevFromLang(lang, false)); + + QString abbrev = getAbbrevFromLang(lang, false); + + //If we don't have the language in our hard-coded table, + //then try it directly + if (abbrev.isEmpty()) + abbrev = lang; + QString ret = hyphLangList.value(abbrev); + if (ret.isEmpty()) + { + //As a hack, if the language was munged into an abbreviated + //generic version through our table, e.g. German to "de" + //then hope for the best and pick the first of e.g. + //"de_AT", "de_DE" if they exist + foreach(QString hlang, hyphLangs()) + { + QStringList items = hlang.split("_"); + if (!items.empty() && items[0] == abbrev) + { + ret = hyphLangList.value(hlang); + break; + } + } + } + return ret; } const QStringList LanguageManager::hyphLangs() |
|
scribus-1.3.8-arbitrary-hyphenators.patch (5,354 bytes)
From 0ccfe17b41250d1c5f21f46309b60bab31ab0d7e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dan=20Hor=C3=A1k?= <dan@danny.cz> Date: Tue, 5 Oct 2010 16:45:49 +0200 Subject: [PATCH] support arbitrary seperately packaged hyphenators --- Scribus/scribus/hyphenator.cpp | 62 +++++++++++++++++++++++---------------- Scribus/scribus/hysettings.cpp | 12 +++++++- Scribus/scribus/langmgr.cpp | 26 ++++++++++++++++- Scribus/scribus/scribus.cpp | 2 +- 4 files changed, 73 insertions(+), 29 deletions(-) diff --git a/Scribus/scribus/hyphenator.cpp b/Scribus/scribus/hyphenator.cpp index 3e0d3d8..38cf5d2 100644 --- a/Scribus/scribus/hyphenator.cpp +++ b/Scribus/scribus/hyphenator.cpp @@ -61,23 +61,26 @@ Hyphenator::Hyphenator(QWidget* parent, ScribusDoc *dok) : QObject( parent ), Language = PrefsManager::instance()->appPrefs.Language; doc->Language = Language; } + // pfad += ScCore->primaryMainWindow()->Sprachen[Language]; - pfad += lmgr->getHyphFilename( doc->Language, false ); - QFile f(pfad); - if (f.open(QIODevice::ReadOnly)) + QString fileName = lmgr->getHyphFilename( doc->Language, false ); + if (!fileName.isEmpty()) { - QTextStream st(&f); - QString line; - line = st.readLine(); - codec = QTextCodec::codecForName(line.toUtf8()); - f.close(); + pfad += fileName; + QFile f(pfad); + if (f.open(QIODevice::ReadOnly)) + { + QTextStream st(&f); + QString line; + line = st.readLine(); + f.close(); + useAble = true; + } } - else - { - useAble = false; - hdict = NULL; + + if (!useAble) return; - } + QByteArray fn = pfad.toLocal8Bit(); const char * filename = fn.data(); hdict = hnj_hyphen_load(filename); @@ -111,22 +114,29 @@ void Hyphenator::NewDict(const QString& name) if (hdict != NULL) hnj_hyphen_free(hdict); - pfad += LanguageManager::instance()->getHyphFilename(Language, false) ; - QFile f(pfad); - if (f.open(QIODevice::ReadOnly)) + useAble = false; + hdict = NULL; + + QString fileName = LanguageManager::instance()->getHyphFilename( doc->Language, false ); + if (!fileName.isEmpty()) { - QTextStream st(&f); - QString line; - line = st.readLine(); - codec = QTextCodec::codecForName(line.toUtf8()); - f.close(); + pfad += fileName; + QFile f(pfad); + if (f.open(QIODevice::ReadOnly)) + { + QTextStream st(&f); + QString line; + line = st.readLine(); + codec = QTextCodec::codecForName(line.toUtf8()); + f.close(); + useAble = true; + } } - else - { - useAble = false; - hdict = NULL; + + + if (!useAble) return; - } + QByteArray fn = pfad.toLocal8Bit(); filename = fn.data(); hdict = hnj_hyphen_load(filename); diff --git a/Scribus/scribus/hysettings.cpp b/Scribus/scribus/hysettings.cpp index 27dc7b0..ebd0407 100644 --- a/Scribus/scribus/hysettings.cpp +++ b/Scribus/scribus/hysettings.cpp @@ -27,7 +27,17 @@ HySettings::HySettings( QWidget* parent/*, QMap<QString,QString>* langs*/ ) : QW language->setInsertPolicy(QComboBox::InsertAlphabetically); foreach(QString hlang, lmg->hyphLangs()) { - language->addItem( lmg->getLangFromAbbrev(hlang), lmg->getLangFromAbbrev(hlang,false) ); + QString uiname = lmg->getLangFromAbbrev(hlang); + QString lang = lmg->getLangFromAbbrev(hlang,false); + + //Fallback to use lang code as name if otherwise unknown + if (uiname.isEmpty()) + uiname = hlang; + + if (lang.isEmpty()) + lang = hlang; + + language->addItem( uiname, lang ); } buttonExceptAdd->setIcon(QIcon(loadIcon("16/list-add.png"))); diff --git a/Scribus/scribus/langmgr.cpp b/Scribus/scribus/langmgr.cpp index 6167c50..cd510b1 100644 --- a/Scribus/scribus/langmgr.cpp +++ b/Scribus/scribus/langmgr.cpp @@ -315,7 +315,31 @@ const QString LanguageManager::getHyphFilename(const QString & lang, bool langIs { if(langIsAbbreviated) return hyphLangList.value(lang); - return hyphLangList.value(getAbbrevFromLang(lang, false)); + + QString abbrev = getAbbrevFromLang(lang, false); + + //If we don't have the language in our hard-coded table, + //then try it directly + if (abbrev.isEmpty()) + abbrev = lang; + QString ret = hyphLangList.value(abbrev); + if (ret.isEmpty()) + { + //As a hack, if the language was munged into an abbreviated + //generic version through our table, e.g. German to "de" + //then hope for the best and pick the first of e.g. + //"de_AT", "de_DE" if they exist + foreach(QString hlang, hyphLangs()) + { + QStringList items = hlang.split("_"); + if (!items.empty() && items[0] == abbrev) + { + ret = hyphLangList.value(hlang); + break; + } + } + } + return ret; } const QStringList LanguageManager::hyphLangs() diff --git a/Scribus/scribus/scribus.cpp b/Scribus/scribus/scribus.cpp index 6b8fb9c..5f3aef2 100644 --- a/Scribus/scribus/scribus.cpp +++ b/Scribus/scribus/scribus.cpp @@ -8899,7 +8899,7 @@ void ScribusMainWindow::initHyphenator() //Grab the language abbreviation from it, get the full language text //Insert the name as key and a new string list into the map QString hyphDirName = QDir::convertSeparators(ScPaths::instance().dictDir()); - QDir hyphDir(hyphDirName, "*.dic", QDir::Name, QDir::Files | QDir::NoSymLinks); + QDir hyphDir(hyphDirName, "*.dic", QDir::Name, QDir::Files); if ((hyphDir.exists()) && (hyphDir.count() != 0)) { // LanguageManager langmgr; -- 1.7.2.3 |
|
Currently testing building with this patch. I have removed the incorrect Galician references. Once working, this needs fwd porting to 1.5.0svn |
|
Does the hyphen directory get created by an Mozilla install or some independent package? I am wondering for other platforms like OSX or Windows.. |
|
The hyphenation dict, spelling dict and the applications/libraries are installed (at least in Fedora/RHEL/CentOS) as a part of "XY Language support" during the OS installation. Additional languages can be installed later using the "Add/Remove software" tool or directly via package management from the hyphen-* or aspell-*/hunspell-* packages. I'd expect other distributions use similar solution. In my opinion Scribus can use a check in ScPaths::dictDir() - if m_shareDir + "dicts/" exists then use this as dictDir, otherwise use the system dicst from "/usr/share/hyphen" (distribution package will remove scribus provided dicts) |
|
I am still confused as to how we can provide for these hyphenation dictionaries on other platforms.. dicts will always exist as we always install to it. It is not at all clear how to get hunspell dicts for OSX or Windows unless we just download them from somewhere like http://wiki.services.openoffice.org/wiki/Dictionaries. With Macports version of hunspell, hunspell-dict-af_ZA @2006-01-17 (textproc) hunspell-dict-ca_ES @2002-10-15 (textproc) hunspell-dict-cs_CZ @2003-01-01 (textproc) hunspell-dict-cy_GB @2004-04-25 (textproc) hunspell-dict-de_DE @2006-02-07 (textproc) hunspell-dict-el_GR @2004-12-20 (textproc) hunspell-dict-en_CA @2002-03-15 (textproc) hunspell-dict-en_NZ @2002-05-18 (textproc) hunspell-dict-en_US @2006-02-07 (textproc) hunspell-dict-en_ZA @2006-01-20 (textproc) hunspell-dict-eo_EO @2005-07-10 (textproc) hunspell-dict-es_ES @2005-05-10 (textproc) hunspell-dict-es_MX @2005-05-05 (textproc) hunspell-dict-fo_FO @2005-03-07 (textproc) hunspell-dict-fy_NL @2007-11-28 (textproc) hunspell-dict-ga_IE @2007-10-29 (textproc) hunspell-dict-gd_GB @2005-01-08 (textproc) hunspell-dict-gsc_FR @2007-08-16 (textproc) hunspell-dict-gu_IN @2006-10-15 (textproc) hunspell-dict-he_IL @2005-01-12 (textproc) hunspell-dict-hi_IN @2007-02-19 (textproc) hunspell-dict-hr_HR @2006-06-07 (textproc) hunspell-dict-id_ID @2004-08-12 (textproc) hunspell-dict-ku_TR @2005-01-21 (textproc) hunspell-dict-lt_LT @2003-12-31 (textproc) hunspell-dict-mg_MG @2005-01-09 (textproc) hunspell-dict-mk_MK @2005-11-26 (textproc) hunspell-dict-ms_MY @2005-01-17 (textproc) hunspell-dict-nb_NO @2006-05-08 (textproc) hunspell-dict-nl_NL @2007-06-07 (textproc) hunspell-dict-nn_NO @2006-05-08 (textproc) hunspell-dict-nr_ZA @2006-01-20 (textproc) hunspell-dict-ns_ZA @2006-01-20 (textproc) hunspell-dict-ny_MW @2005-01-08 (textproc) hunspell-dict-oc_FR @2006-09-22 (textproc) hunspell-dict-rw_RW @2005-01-08 (textproc) hunspell-dict-sl_SI @2007-01-27 (textproc) hunspell-dict-ss_ZA @2006-07-05 (textproc) hunspell-dict-st_ZA @2006-01-20 (textproc) hunspell-dict-sw_KE @2004-05-16 (textproc) hunspell-dict-tet_ID @2005-01-08 (textproc) hunspell-dict-th_TH @2006-12-12 (textproc) hunspell-dict-tl_PH @2005-01-08 (textproc) hunspell-dict-tn_ZA @2004-05-16 (textproc) hunspell-dict-ts_ZA @2006-01-23 (textproc) hunspell-dict-uk_UA @2009-01-25 (textproc) hunspell-dict-ve_ZA @2006-07-06 (textproc) hunspell-dict-xh_ZA @2006-01-23 (textproc) hunspell-dict-zu_ZA @2006-01-20 (textproc) |
Date Modified | Username | Field | Change |
---|---|---|---|
2009-06-15 13:20 | caolan | New Issue | |
2009-06-15 13:20 | caolan | File Added: scribus-1.3.5.rc2.arbitraryhyphenators.patch | |
2009-07-30 23:06 |
|
Status | new => assigned |
2009-07-30 23:06 |
|
Assigned To | => cbradney |
2010-10-05 16:47 | sharkcz | File Added: scribus-1.3.8-arbitrary-hyphenators.patch | |
2010-10-07 20:52 | cbradney | Note Added: 0024621 | |
2010-10-07 20:58 | cbradney | Note Added: 0024622 | |
2010-10-08 15:45 | sharkcz | Note Added: 0024627 | |
2010-10-09 12:25 | cbradney | Note Added: 0024633 | |
2010-10-09 12:29 | cbradney | Note Edited: 0024633 | |
2014-08-31 03:09 | Kunda | Summary | Support for arbitrary seperately packaged hyphenators => [PATCH[ Support for arbitrary seperately packaged hyphenators |
2014-08-31 03:10 | Kunda | Summary | [PATCH[ Support for arbitrary seperately packaged hyphenators => [PATCH] Support for arbitrary seperately packaged hyphenators |
2014-09-03 17:33 | cbradney | Status | assigned => closed |
2014-09-03 17:33 | cbradney | Resolution | open => won't fix |
2014-10-24 22:57 | Kunda | Patch | => Yes |