View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0017367 | Scribus | Typography | public | 2025-01-08 17:36 | 2025-01-09 21:28 |
Reporter | jirib | Assigned To | |||
Priority | normal | Severity | minor | Reproducibility | always |
Status | new | Resolution | open | ||
Product Version | 1.6.3.svn | ||||
Summary | 0017367: [hyphenator.cpp] hyphenation works differently even when using same hyphenation dict | ||||
Description | This is how is the Czech word 'orgánu' hyphenated in Scribus: $ $ grep -Po ' \K(or.*nu)\s+' pl-reprint-01.sla | xxd 00000000: 6f72 c2ad 67c3 a16e 7520 0a or..g..nu . # in literal: or-gánu ('-' is soft hyphen) A proof that same hyphen dict is used: $ md5sum ~/.local/share/scribus/dicts/hyph/hyph_cs_CZ.dic /home/jiri/.venv3/lib/python3.11/site-packages/pyphen/dictionaries/hyph_cs_CZ.dic c9cbd906b7884d88b00a3a0a6a7773c7 /home/jiri/.local/share/scribus/dicts/hyph/hyph_cs_CZ.dic c9cbd906b7884d88b00a3a0a6a7773c7 /home/jiri/.venv3/lib/python3.11/site-packages/pyphen/dictionaries/hyph_cs_CZ.dic With pyphen: >>> import pyphen >>> dic = pyphen.Pyphen(lang='cs_CZ') >>> text = 'orgánu' >>> ' '.join(dic.inserted(word) for word in text.split()) 'or-gá-nu' There's a TODO note in the code regarding 'hnj_hyphen_hyphenate2()' - https://github.com/scribusproject/scribus/blob/master/scribus/hyphenator.cpp#L116 . Is that the reason? The "impact" is unpleasant rivers of white :-( | ||||
Steps To Reproduce | See example above. | ||||
Tags | No tags attached. | ||||
Patch | No | ||||
|
Eh, pyphen is python-only implementation, it does not use hunspell library. So, I don't know if this is a bug or not :/ |
|
s/hunspell/hyphen/ - i'm very sorry, i'm new to this. a test via C code from hyphen: $ gcc -o hyphen_test /tmp/hyphen.c -lhyphen $ ./hyphen_test Hyphenation points for 'orgánu': Hyphenation point at index 1 Hyphenation point at index 4 Hyphenated word: or-gá-nu |
|
C code for hyphen_test hyphen.c (1,361 bytes)
#include <stdio.h> #include <string.h> #include "hyphen.h" int main() { HyphenDict *hd = hnj_hyphen_load("hyph_cs_CZ.dic"); if (!hd) { fprintf(stderr, "Failed to load hyphenation patterns\n"); return 1; } const char *word = "orgánu"; int word_size = strlen(word); char hyphens[word_size + 5]; memset(hyphens, 0, sizeof(hyphens)); char hyphenated_word[word_size * 2]; memset(hyphenated_word, 0, sizeof(hyphenated_word)); char **rep = NULL; int *pos = NULL; int *cut = NULL; int result = hnj_hyphen_hyphenate2(hd, word, word_size, hyphens, hyphenated_word, &rep, &pos, &cut); if (result != 0) { fprintf(stderr, "Hyphenation failed\n"); hnj_hyphen_free(hd); return 1; } int index = 0; for (int i = 0; i < word_size; i++) { hyphenated_word[index++] = word[i]; if (hyphens[i] % 2 != 0) { hyphenated_word[index++] = '-'; } } hyphenated_word[index] = '\0'; printf("Hyphenation points for '%s':\n", word); for (int i = 0; i < word_size; i++) { if (hyphens[i] % 2 != 0) { printf("Hyphenation point at index %d\n", i); } } printf("Hyphenated word: %s\n", hyphenated_word); hnj_hyphen_free(hd); return 0; } |
|
Scribus uses an included hyphenation codebase as you've seen and clearly can produce different results to a python based hyphenator. |
|
Yes, the point to pyphen was wrong in detail but the attached C code produces "better" hyphenation, thus I think there's an area for improvement in Scribus - the attached code uses hnj_hyphen_hyphenate2(). And, there's hnj_hyphen_hyphenate3(), i'm not sure but seems like a new way to achieve hyphenation ??? |
|
Looking at the code at https://github.com/hunspell/hyphen/blob/master/hyphen.c, it looks like the 2nd function is the default. Our hyphen code has not been touched much in years. We'd need to see what we would need to change and understand how it would be different. It is something we could change in Scribus 1.7.0. |
Date Modified | Username | Field | Change |
---|---|---|---|
2025-01-08 17:36 | jirib | New Issue | |
2025-01-08 18:57 | jirib | Note Added: 0051889 | |
2025-01-08 19:15 | jirib | Note Added: 0051890 | |
2025-01-08 19:15 | jirib | Note Added: 0051891 | |
2025-01-08 19:15 | jirib | File Added: hyphen.c | |
2025-01-08 21:59 | cbradney | Note Added: 0051892 | |
2025-01-09 05:58 | jirib | Note Added: 0051893 | |
2025-01-09 21:28 | cbradney | Note Added: 0051903 |