View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0017367 | Scribus | Typography | public | 2025-01-08 17:36 | 2025-01-09 05:58 |
Reporter | jirib | Assigned To | |||
Priority | normal | Severity | minor | Reproducibility | always |
Status | new | Resolution | open | ||
Product Version | 1.6.3.svn | ||||
Summary | 0017367: [hyphenator.cpp] hyphenation works differently even when using same hyphenation dict | ||||
Description | This is how is the Czech word 'orgánu' hyphenated in Scribus: $ $ grep -Po ' \K(or.*nu)\s+' pl-reprint-01.sla | xxd 00000000: 6f72 c2ad 67c3 a16e 7520 0a or..g..nu . # in literal: or-gánu ('-' is soft hyphen) A proof that same hyphen dict is used: $ md5sum ~/.local/share/scribus/dicts/hyph/hyph_cs_CZ.dic /home/jiri/.venv3/lib/python3.11/site-packages/pyphen/dictionaries/hyph_cs_CZ.dic c9cbd906b7884d88b00a3a0a6a7773c7 /home/jiri/.local/share/scribus/dicts/hyph/hyph_cs_CZ.dic c9cbd906b7884d88b00a3a0a6a7773c7 /home/jiri/.venv3/lib/python3.11/site-packages/pyphen/dictionaries/hyph_cs_CZ.dic With pyphen: >>> import pyphen >>> dic = pyphen.Pyphen(lang='cs_CZ') >>> text = 'orgánu' >>> ' '.join(dic.inserted(word) for word in text.split()) 'or-gá-nu' There's a TODO note in the code regarding 'hnj_hyphen_hyphenate2()' - https://github.com/scribusproject/scribus/blob/master/scribus/hyphenator.cpp#L116 . Is that the reason? The "impact" is unpleasant rivers of white :-( | ||||
Steps To Reproduce | See example above. | ||||
Tags | No tags attached. | ||||
Patch | No | ||||
|
Eh, pyphen is python-only implementation, it does not use hunspell library. So, I don't know if this is a bug or not :/ |
|
s/hunspell/hyphen/ - i'm very sorry, i'm new to this. a test via C code from hyphen: $ gcc -o hyphen_test /tmp/hyphen.c -lhyphen $ ./hyphen_test Hyphenation points for 'orgánu': Hyphenation point at index 1 Hyphenation point at index 4 Hyphenated word: or-gá-nu |
|
C code for hyphen_test hyphen.c (1,361 bytes)
#include <stdio.h> #include <string.h> #include "hyphen.h" int main() { HyphenDict *hd = hnj_hyphen_load("hyph_cs_CZ.dic"); if (!hd) { fprintf(stderr, "Failed to load hyphenation patterns\n"); return 1; } const char *word = "orgánu"; int word_size = strlen(word); char hyphens[word_size + 5]; memset(hyphens, 0, sizeof(hyphens)); char hyphenated_word[word_size * 2]; memset(hyphenated_word, 0, sizeof(hyphenated_word)); char **rep = NULL; int *pos = NULL; int *cut = NULL; int result = hnj_hyphen_hyphenate2(hd, word, word_size, hyphens, hyphenated_word, &rep, &pos, &cut); if (result != 0) { fprintf(stderr, "Hyphenation failed\n"); hnj_hyphen_free(hd); return 1; } int index = 0; for (int i = 0; i < word_size; i++) { hyphenated_word[index++] = word[i]; if (hyphens[i] % 2 != 0) { hyphenated_word[index++] = '-'; } } hyphenated_word[index] = '\0'; printf("Hyphenation points for '%s':\n", word); for (int i = 0; i < word_size; i++) { if (hyphens[i] % 2 != 0) { printf("Hyphenation point at index %d\n", i); } } printf("Hyphenated word: %s\n", hyphenated_word); hnj_hyphen_free(hd); return 0; } |
|
Scribus uses an included hyphenation codebase as you've seen and clearly can produce different results to a python based hyphenator. |
|
Yes, the point to pyphen was wrong in detail but the attached C code produces "better" hyphenation, thus I think there's an area for improvement in Scribus - the attached code uses hnj_hyphen_hyphenate2(). And, there's hnj_hyphen_hyphenate3(), i'm not sure but seems like a new way to achieve hyphenation ??? |
Date Modified | Username | Field | Change |
---|---|---|---|
2025-01-08 17:36 | jirib | New Issue | |
2025-01-08 18:57 | jirib | Note Added: 0051889 | |
2025-01-08 19:15 | jirib | Note Added: 0051890 | |
2025-01-08 19:15 | jirib | Note Added: 0051891 | |
2025-01-08 19:15 | jirib | File Added: hyphen.c | |
2025-01-08 21:59 | cbradney | Note Added: 0051892 | |
2025-01-09 05:58 | jirib | Note Added: 0051893 |