View Issue Details

IDProjectCategoryView StatusLast Update
0011060ScribusImport / Exportpublic2015-11-21 17:19
Reporterjzeleny Assigned To 
PrioritynormalSeverityminorReproducibilityalways
Status newResolutionopen 
Summary0011060: Rewrite plugin for HTML import
DescriptionCurrent HTML plugin has several shortcomings related to a limited support of HTML tags. Attached archive contains new implementation of HTML import plugins which adds various new features.

First of all the plugin adds a support for basic set of CSS rules. The second main advantage is that the plugin can be now very easily extended to support new HTML tags via the CSS definition. Also it is quite easy to modify the CSS definition of extended plugins.

A complete list of supported features is on the mailing list:
http://lists.scribus.net/pipermail/scribus-dev/2012-August/001669.html
TagsNo tags attached.
PatchYes

Activities

jzeleny

2012-09-04 19:51

reporter   ~0028902

Please note that I've already re-written the plugin and sent sources to the mailing list. The latest version at the moment is attached to this email (it was too large to be attached here):

http://lists.scribus.net/pipermail/scribus-dev/2012-September/001673.html

JLuc

2012-09-04 21:44

developer   ~0028903

AMOF the attached file's adress is
http://lists.scribus.net/pipermail/scribus-dev/attachments/20120904/f5ee31d3/attachment.obj

After downloading it, it should be renamed to htmlimporter.tar.bz2
That makes it readable.

It contains 5 complete source files.

ale

2012-09-07 20:04

manager  

htmlimporter.tar.bz2 (6,897 bytes)

jghali

2012-09-07 22:06

administrator   ~0028921

I started testing the new html importer :
- on the architecture side, it is a definite improvement
- currently the importer creates to much styles and basic styles (h1...h6, etc...) are not clearly defined after import in style manager

The second point is problematic as it would break some workflows and needs to be improved.

jzeleny

2012-09-26 21:01

reporter   ~0028981

Thank you for the review. I'm not entirely sure what do you mean by the clear definition in style manager. All styles are pre-created in the same way they were in the old plugin or am I wrong about that? Perhaps you can suggest a way to improve this? I'll be happy to implement the adjustment.

I understand your concern regarding creation of many styles but that's the point, isn't it? Every time a property of the HTML/CSS style changes, a new style has to be created for it. Otherwise we would need some kind of hashing of styles to find and reuse already created style. I'm not saying it can't be done but I simply don't think the effort to implement something like that would be worth the gain.

jghali

2012-09-26 21:51

administrator   ~0028982

>> I understand your concern regarding creation of many styles but that's the point, isn't it?

Basically no. For HTML styles should be created only for html elements (h1...h6, p, etc...). Properties found in CSS attributes should be applied as direct formatting using style for specific element as a base (ie the style for specific element must be set as parent of corresponding ParagraphStyle object). After import you should just have as many styles as different html element names, eg:

html_h1
html_h2
html_h3
html_h4
html_h5
html_h6
html_p
...

Having too many styles make styling imported text painful and tedious.

jzeleny

2012-09-26 22:06

reporter   ~0028983

Ah, ok. Thanks for the explanation. Now I understand it a little bit better. I'll look into it and send you the code once I'm done. Probably in couple days.

jzeleny

2012-09-28 23:11

reporter   ~0028994

Last edited: 2015-11-21 17:08

I have a prototype code with the requested functionality, however there is still one problem and that is application of different paragraph styles to one element type. Let me give an example:

  (p) left text (/p)
  (p) style="text-align: center">center text (/p)
  (p) style="text-align: right">right text (/p)

When I apply your approach, there is no way to set paragraph style attributes like alignment and margin. In the example code above this leads to left-aligned text in all three paragraphs. The alignment parameter is simply ignored, as it is not a part of the original style definition.

The solution would be to create variations of each style if necessary (say style_p_1, style_p_2, ...) but that would ultimately lead to the same state as there is now, except that all styles would be called nicely. Much more complex situations leading to the same issue can occur in the future versions if we want to support features like style = "display: block".

That leads me to conclusion it's probably not worth further effort. Also you can consider that the number of created styles is not that high in the end - one for each block of text. As I proved above, this number is necessary if we want to support paragraph-related attributes in CSS. What is your opinion on this issue? Do you have any suggestion how could I solve the issue and still implement your approach with all the functionality? If you want, I can send you the new code.

pygmee

2012-09-29 09:57

developer   ~0028995

my opinion would be : import styles defined with css rules only. only those should worth to become styles.
pb is to know if they need to be paragrah or char styles (usé à list id properties, detect context ?)

for css inline définition, rhétorique should be apply as direct formatting.

jzeleny

2012-09-29 12:35

reporter   ~0028997

Just making sure I understand correctly what do you mean: you would take only those styles defined within <style></style> in HTML head definition, is that right? I'm afraid the parser is not that complex, it supports only those inline definitions you mentioned.

Current state on my development machine does apply inline styles as direct formatting but there is a problem with that. Inline style definitions of block properties such as margin and alignment are ignored because they would need a new Scribus style, their direct application doesn't have any effect. I thought about this issue some more and the proposed solution (to create more variations of element styles, like style_p_1, ...) still seems like the best I can do but it doesn't bring much improvement over the code I originally submitted for review.

If I'm missing something and block attributes don't need a new style to be defined, please let me know, I'll be happy to give it another shot.

pygmee

2012-09-29 14:02

developer   ~0028998

I think that your script should keep, as possible, the idea of the source file. Where the HTML+CSS file uses classes, then use a style. Where it doesn't use inline formatting.
i have several reason for that :
- keep the document spirit
- don't use style for people who don't know them (if a user don't use classes in CSS may be he won't know about styles in Scribus)
- avoid to have too many styles with insignificant names or difference.

I'll try to have a look at your patch to see if i can see something to help.
Cheers

jzeleny

2012-09-29 14:30

reporter   ~0028999

Thanks in advance for taking a look at the code. However I have to state again that the current code can parse *only* inline style definitions, it knows nothing about ids, classes and similar stuff. Implementation of such functionality would make the code at least by an order of magnitude more complex.

To implement your approach would mean to completely disable block attributes like margin and alignment. Please know that I understand your reasoning, however with current plugin possibilities you basically want me to disable these attributes, as they can't be applied as inline formatting and no other way to parse them from HTML file exists.

To comment your three points:
ad 1) agreed, the spirit should be kept as much as possible
ad 2) considering that classes aren't parsed by the plugin and inline style definition is the only supported definition, this is probably irrelevant
ad 3) how about my proposal to have variants of element-based styles, e.g style_p for plain p elements and style_p_X for any p elements changed by an inline style definition? That would address your concern about having styles with insignificant names or difference, wouldn't it?

ale

2012-09-29 16:03

manager   ~0029000

i agree with pygmee, where no named styles are used in html (class), direct formatting should be used in scribus...

jzeleny, would you mind putting the patch in the git repository?

ciao
a.l.e

jzeleny

2012-09-29 21:18

reporter   ~0029002

I agree with both of you guys but as I outlined, that would mean no block-based styling is allowed (which basically means I would have to run my own branch of code the same way I do now, since I kinda have to support that feature).

Here is a repository with some of my WIP, you will be interested in the "public" branch:
git://fedorapeople.org/~jzeleny/scribus.git

ale

2012-09-30 09:04

manager   ~0029003

i see...
what about creating those styles only when you detect html-formatting that you want support but scribus does not support as local formatting?
imo, the final goal should be that all formatting supported by scribus should be supported as both local and style formatting.
clearly identifying those cases in the html importer will help removing the exceptions as soon as the feature is supported in scribus.
btw, afaik, cezary has already added most of the formatting to the PP.

and thanks for the link to your git repository!

jzeleny

2012-09-30 10:40

reporter   ~0029004

Yes, that's what I had in mind. Basically I would have base style for each element type (say H1) and when one of block-based style attributes needed to be applied, I'd create a derivate (say h1_1, which would have the h1 style as base and right text alignment applied on top of it).

Right now almost everything is in my repo except for the derivation I just described. I will post that probably later today.

Kunda

2015-06-17 21:42

updater   ~0035422

Found https://github.com/mattjudge/scribus_sla_to_html

JLuc

2015-06-18 06:19

developer   ~0035423

Is there any new step about HTML import that might possibly soon benefit scribus main repo ?

JLuc

2015-06-18 07:56

developer   ~0035424

I added these proposals in the alterscribus github account of zombie forks : https://github.com/AlterScribus/jzeleny_htmlimport/blob/master/README.md

pygmee

2015-06-18 12:02

developer   ~0035425

Great ! but i.m not sure direct formatting is the right approach for we try te avoid it as possible. I would do HTML tags -> paragraph styles, CSS classes -> caracter styles, CSS #id, direct formatting as for inline style definitions

JLuc

2015-06-18 14:35

developer   ~0035428

Last edited: 2015-06-18 15:39

Pygmee, regarding this subject, HTML has 2 types of tags : BLOCKS and INLINE

DIV is a typical BLOCK type tag, but there are lots of other.
block is generaly a bit similar to paragraph.

SPAN is a typical INLINE type.
inline is generaly similar to a a bunch of characters.

First basic approach would be
- block type tag -> paragraph style
- inline type tag -> character style

But CSS can alter these caracteristics with the display property.
- display: block;
- display : inline;
- and lots of other possible values...

I agree with "CSS #id" could go to inline style definition.
This should for sure also include HTML inline style definition : style="...css styling..."

ale

2015-06-18 15:47

manager   ~0035429

Last edited: 2015-06-18 15:48

the question is, if scribus should parse arbitrary html or well formed html only (leaving the transformation from arbitrary html to a well formed one to an external pre-processor).

since html (and css) tend to be heavily abused both on the web and in the export from office programs, i'd prefer that scribus limits itself to a specific set of sane (and well documented) rules and does not support every possible crazy layout directives people have come up with. (like an inline div! use it for your web design, if you can't do otherwise, but do not expect scribus to care about it.)

assuming that a paragraph style is applied through a p, li or h# is imo a sane rule.
assuming that span, b, i, sup, sub and a few other can apply a specific formatting rule (bold, italic, superscript, subscript, ...) and/or a character style is in my eyes ok.

in this case, i'd really prefer scribus to behave in a reproducible and predictable way, rather than having it trying to catch everything and sometimes fail.


btw, the lists above are not meant to be exhaustive, i only use them to better show in which direction my thoughts are going...

JLuc

2015-06-18 16:45

developer   ~0035433

Last edited: 2015-06-18 16:46

For more complete list see
- https://developer.mozilla.org/en-US/docs/Web/HTML/Block-level_elements
- https://developer.mozilla.org/en-US/docs/Web/HTML/Inline_elemente

I agree with not trying to follow every sophisticate rule, but the place where to draw the line should be further searched... and probably depend on the programmer's good will to elaborate the rules accordingly.

For example IMO : 'display: inline' is too frequent and normal a use, in the semantic web, for it to be neglected.

pygmee

2015-06-19 07:56

developer   ~0035448

i totally agree with you but i consider div and span as fake tags just used to apply css classes. But in fact it’s true there are important and specific inline tags. My purpose was mostly say that we should avoid as possible the use of inline formatting in scribus for it increases the typographical work on the scribus document afterwords and that may be even "Properties found in CSS attributes" should be considered as a style. On the web, people often use direct formatting where they should create classes just because they don’t it exist

JLuc

2015-11-21 17:17

developer   ~0037575

Last edited: 2015-11-21 17:19

I dont use html import, but IMO, an html importer should manage css classes.
Not managing the css classes makes an importer useless in most of the use cases.
AMOF, using classes and not using inline styles in HTML is probably even more important and recommended good practice than using named styles in scribus and not using inline styles...

Some foss projects are reading html and converting the css rules into inline styles. For example for email use because many email readers are stubborn and dont understand the stylesheets. These codes might be a good starting point for a stylesheets and html importer.

Issue History

Date Modified Username Field Change
2012-09-04 19:38 jzeleny New Issue
2012-09-04 19:51 jzeleny Note Added: 0028902
2012-09-04 21:44 JLuc Note Added: 0028903
2012-09-07 20:04 ale File Added: htmlimporter.tar.bz2
2012-09-07 22:06 jghali Note Added: 0028921
2012-09-26 21:01 jzeleny Note Added: 0028981
2012-09-26 21:51 jghali Note Added: 0028982
2012-09-26 22:06 jzeleny Note Added: 0028983
2012-09-28 23:11 jzeleny Note Added: 0028994
2012-09-29 09:57 pygmee Note Added: 0028995
2012-09-29 12:35 jzeleny Note Added: 0028997
2012-09-29 14:02 pygmee Note Added: 0028998
2012-09-29 14:30 jzeleny Note Added: 0028999
2012-09-29 16:03 ale Note Added: 0029000
2012-09-29 21:18 jzeleny Note Added: 0029002
2012-09-30 09:04 ale Note Added: 0029003
2012-09-30 10:40 jzeleny Note Added: 0029004
2015-06-17 21:42 Kunda Note Added: 0035422
2015-06-18 06:19 JLuc Note Added: 0035423
2015-06-18 06:19 JLuc Patch => Yes
2015-06-18 07:56 JLuc Note Added: 0035424
2015-06-18 12:02 pygmee Note Added: 0035425
2015-06-18 14:35 JLuc Note Added: 0035428
2015-06-18 15:39 JLuc Note Edited: 0035428
2015-06-18 15:47 ale Note Added: 0035429
2015-06-18 15:48 ale Note Edited: 0035429
2015-06-18 16:45 JLuc Note Added: 0035433
2015-06-18 16:46 JLuc Note Edited: 0035433
2015-06-19 07:56 pygmee Note Added: 0035448
2015-11-21 17:06 JLuc Note Edited: 0028994
2015-11-21 17:07 JLuc Note Edited: 0028994
2015-11-21 17:08 JLuc Note Edited: 0028994
2015-11-21 17:08 JLuc Note Edited: 0028994
2015-11-21 17:17 JLuc Note Added: 0037575
2015-11-21 17:18 JLuc Note Edited: 0037575
2015-11-21 17:18 JLuc Note Edited: 0037575
2015-11-21 17:19 JLuc Note Edited: 0037575