View Issue Details

IDProjectCategoryView StatusLast Update
0016142ScribusImport / Exportpublic2020-10-02 21:27
Reporteroliverthered Assigned To 
PrioritynormalSeverityminorReproducibilityalways
Status newResolutionopen 
PlatformallOSOS Version
Product Version1.5.6.svn 
Target VersionFixed in Version 
Summary0016142: review and merge pull request 0000135 to begin supporting text import from PDF files
DescriptionI have created the following pull request from my fork of the Scribus project:
https://github.com/scribusproject/scribus/pull/135

Ale and I have been working on getting it ready over the last couple of weeks and I'm fairly confident it's good for reviewing and integrating with the main branch.

It adds basic PDF text import functionality, just the text and basic layout functionality, no styles or fonts, right or centere3d justification, rotation, multipage etc... but it does provide enough of a base that most of the extra required functionality can be worked on as isolated features with low interdependencies.

I hope the code is to your standard and liking, and we can get it merged quickly, so I can get on writing the extra required features.
Steps To ReproduceImport a vector file in pdf format.
Note that the text is vectors and you are unable to edit it in and reasonable fashion.
Additional InformationA final review and sign off by ale may be required.
Tagsimport, PDF, pdf import, vectorised fonts
PatchYes

Relationships

has duplicate 0016129 closedjghali basic textframe and text rendering 
has duplicate 0016128 closedjghali implement some more on the basic pdf text import, framework, abstract addchar and ui for selecting import method 
has duplicate 0016126 closedjghali refactor getChar so that it can be used for importing text as well as drawing vector characters 
has duplicate 0016125 closedjghali Implement a framework for importing text from pdf documents 
has duplicate 0016127 closedjghali override updateTextPos in SlaOutputDev 
has duplicate 0016103 closedjghali Update PDF import with features matching Inkscapes PDF import 

Activities

jghali

2020-06-21 13:06

administrator   ~0047710

I had a quick look. The renaming of updateFont() method to updateFontForVector() looks wrong. The updateFont() method is indeed a virtual method of poppler OutputDev class which is called by poppler itself and needs to be reimplemented at least for the text as vectors case.

oliverthered

2020-06-21 13:13

reporter   ~0047711

ok, sounds like a plan. most of the functionality of SlaOutputDev needs to remain the same so I'll create a new outputdev class and inherit from SlaOutputDev overriding the methods needed for text import and just choose the correct outputdev class in bool PdfPlug::convert(const QString& fn)

jghali

2020-06-21 13:24

administrator   ~0047712

Hmm, I think this is a bit overkill. You will need that updateFont() method anyway to get proper font information in the end.

oliverthered

2020-06-21 13:53

reporter   ~0047713

oh, I've done it. I'll tidy the code up and see what you think.

oliverthered

2020-06-21 13:55

reporter   ~0047714

laOutputDev::updateFont loads the fonts from the pdf file I think, which I don't want I need to use scribus accessible fonts and will probably load all the fonts from the document and allow substitutions to be set in the pdf import UI. well, that's my current thoughts, font overrides need to happen some place and should be user configurable.

oliverthered

2020-06-21 14:11

reporter   ~0047715

I think implementing a new OutputDev based on SlaOutputDev gives the tidiest implementation and doesn't really have any overheads.

oliverthered

2020-06-21 14:11

reporter   ~0047716

I've pushed the changes now.

oliverthered

2020-06-21 15:18

reporter   ~0047717

the other consideration is that the implementation would be nice if it were easy to use in other projects without too much modification. so keeping it all encapsulated in it's own outputdev as opposed to being incorporated in SlaOutputDev achieves that goal too.

jghali

2020-06-21 15:35

administrator   ~0047718

TextOutputDev is already a poppler class name, you should better avoid it.

oliverthered

2020-06-21 15:41

reporter   ~0047719

ok,
I've had a thought and there are a few more things from SlaOiutputDev that need to be implemented differently for importing text as opposed to vectors so cresting a new class makes sense. I'll change the name.

oliverthered

2020-06-21 15:44

reporter   ~0047720

ok, the name from TextOutputDev to PdfTextOutputDev.
the PdfTextOutputDev naming matches the naming of PdfTextRecognition which is why I chose it.

oliverthered

2020-06-21 21:18

reporter   ~0047721

I've gone through and renames all the classes and member variables so they all start with 'PdfText' for consistency

oliverthered

2020-06-26 15:50

reporter  

pdf-text-import-phase1.rar (12,937 bytes)

oliverthered

2020-06-26 15:51

reporter   ~0047736

this patch should be good to go. there's a couple of bugs i fixed upstream but they should be in the next patch, the bugs don't stop functionanlity etc..

oliverthered

2020-06-26 16:11

reporter   ~0047737

this patch should be better. it makes minimal changes to slaoutputdev and fixes a bug

pdf-text-import-phase1-2.rar (12,564 bytes)

jghali

2020-06-26 21:44

administrator   ~0047738

Unfortunately I cannot apply this patch:
patch: **** malformed patch at line 1015: @@ -3359,8 +3341,9 @@ void SlaOutputDev::drawChar(GfxState *state, double x, double y, double dx, doub

jghali

2020-06-26 22:05

administrator   ~0047739

I finished the merge manually. I started doing some tests and found a bug : in most cases I do not see the dialog which would allow me to choose how text should be imported.

jghali

2020-06-26 22:14

administrator   ~0047741

There are lots of missing braces in PdfTextRegion::linearTest() too

jghali

2020-06-26 22:35

administrator   ~0047742

Hmmm, in most of my tests, when importing text as text the imported text is often not even imported at an approximately correct position.

jghali

2020-06-26 22:42

administrator   ~0047743

Try this file for example;
https://drive.google.com/file/d/1pm_0kgsV9Q1RosYK5AkIIxLhFdc22g7r/view?usp=sharing
When importing text as text, import fails pretty miserably :(

oliverthered

2020-07-02 06:48

reporter   ~0047751

my current working version that supports font and styles isn't too far off. I've not added support for displacement which is different from just the text position I'm currently using.

2020-07-02 (3).png (1,031,872 bytes)
2020-07-02 (3).png (1,031,872 bytes)

oliverthered

2020-07-02 06:58

reporter   ~0047752

most of the pdf files I have for testing at my end were all probably generated by latex. it would be good to get a broader selection of pdf files from different sources.

oliverthered

2020-07-02 07:00

reporter   ~0047753

What are the requirement for getting the code merged with scribus.

I can add some more braces. ale said that there should be no braces if the if or else following is only one line.

oliverthered

2020-07-02 07:09

reporter   ~0047754

and for some of the documents a second parse over the layout is required because the text is separated by lots of begin/end text regions that should be merged together as a continuous block of text.

oliverthered

2020-07-02 07:17

reporter   ~0047755

my aim with the first patch certainly isn't perfection but enough of a base that features can be added to it fairly cleanly.

oliverthered

2020-07-02 08:37

reporter   ~0047756

here's a screenshot of my work in progress using the pdf file i've been using for the bulk of testing. After I've fixed the colours so they are presented correctly there should only be smaller features and bug fixes to be implemented.
https://i.imgur.com/v6jXidp.png

oliverthered

2020-07-02 10:44

reporter   ~0047757

here's what the index page of the attached pdf look like in my current working version:
https://i.imgur.com/4fmMMxe.png

oliverthered

2020-07-02 10:50

reporter   ~0047758

oops, that's imported as vectors not text.

oliverthered

2020-07-02 10:59

reporter   ~0047759

this is what it actually looks like : https://i.imgur.com/bzTzv57.png
the baseline is correct it's just creating textframes that are too tall which makes it look more of a mess than it actually is.

jghali

2020-07-02 11:15

administrator   ~0047760

>> I can add some more braces. ale said that there should be no braces if the if or else following is only one line.

In PdfTextRegion::linearTest() you have code like this:

if (collinear(point.y(), lastXY.y()))
        if (collinear(point.x(), lastXY.x()))
            return LineType::FIRSTPOINT;
        else if (xInLimits)
            return LineType::SAMELINE;
#ifdef DEBUG_TEXT_IMPORT
        else
            qDebug() << ...;
#endif

It's clear that the block related to the if (collinear(point.y(), lastXY.y())) condition uses more than one line, so code should be inside braces:

if (collinear(point.y(), lastXY.y()))
{
        if (collinear(point.x(), lastXY.x()))
            return LineType::FIRSTPOINT;
        else if (xInLimits)
            return LineType::SAMELINE;
#ifdef DEBUG_TEXT_IMPORT
        else
            qDebug() << ...;
#endif
}

You have similar blocks in same function which should be enclosed in braces.

oliverthered

2020-07-02 11:38

reporter   ~0047762

here's what it looks like after a small tweek: https://i.imgur.com/0lkvtuw.png
to get it looking correct i'll have to lookup the linespacing of the first line from the font instead of making an assumption based on a multiple of dx.

oliverthered

2020-07-02 12:03

reporter   ~0047763

setting the linespacing based on the font height instead of a multiple of dx seems to have done the trick. but that's in the fonts and styles update not in the patch for this issue.

2020-07-02 (13).png (551,962 bytes)
2020-07-02 (13).png (551,962 bytes)
2020-07-02 (12).png (272,812 bytes)
2020-07-02 (12).png (272,812 bytes)

oliverthered

2020-07-02 14:54

reporter   ~0047764

I can backport some of the bug fixes that i did for fonts and styles into the initial patch if you want to have the first patch in better working order but the main fix for the attached pdf was replacing some of my guestimates with more intelligent ones based upon the font in use.

oliverthered

2020-07-02 15:14

reporter   ~0047765

ok, i'll do the braces in a few minutes. is there anything else?

oliverthered

2020-07-02 15:33

reporter   ~0047766

find patch attached. I've compiled and run this one so it should be ok this time. I checked and changes to support the pdf document attached to this thread require font support to get anything like a sensible import.

pdftext-phase1.rar (13,453 bytes)

oliverthered

2020-07-02 16:51

reporter   ~0047767

I've had a look at the attached pdf file calls beginTextObject and endTextObject for pretty much every letter which is why there not getting merged. my plan for this was to import all the textframes for a page and then parse over them merging adjacent textframes or textframes that make up new lines in multiline text.

oliverthered

2020-07-02 17:31

reporter   ~0047768

if I ignore begin/end text object and the cursor being reset to the origin I get the following result:

This is the result I'd expect after parsing over all the textframes and merging appropriately.

2020-07-02 (15).png (327,659 bytes)
2020-07-02 (15).png (327,659 bytes)

oliverthered

2020-07-02 21:22

reporter   ~0047770

I'm currently implementing the code to do the merging. that will mean the attached pdf file should import properly.

oliverthered

2020-07-03 05:24

reporter   ~0047771

I got the merge code working successfully. There are still some flow control issues e.g. no line breaks. But they're not implemented anywhere yet. See attached screenshot.

2020-07-03.png (401,107 bytes)
2020-07-03.png (401,107 bytes)

oliverthered

2020-07-03 16:05

reporter   ~0047774

I'm going to add some bespoke code to manage tables as they are quite a common feature and it would be nice to have them laid out properly.

jghali

2020-07-03 17:26

administrator   ~0047778

I see one last major issue with the last patch you uploaded: when you open (with file > open) a PDF containing multiple pages, after first page text frames are not created at correct position. In fact they seem to be placed before first page.

oliverthered

2020-07-03 17:34

reporter   ~0047780

yeah, multiple pages aren't supported at the moment. it's on my list as the next thing to support.

oliverthered

2020-07-03 17:36

reporter   ~0047781

my comment in the description for this bug was "It adds basic PDF text import functionality, just the text and basic layout functionality, no styles or fonts, right or centere3d justification, rotation, multipage etc... but it does provide enough of a base that most of the extra required functionality can be worked on as isolated features with low interdependencies."

oliverthered

2020-07-03 17:56

reporter   ~0047782

I've a few more patches all but ready too. one for supporting fonts and styles, one to support merging of erratically placed content into textframes, and one that adds spaces to the end of the lines and line breaks so the text flows properly.

oliverthered

2020-07-03 18:42

reporter   ~0047785

I've just been focused on getting the text rendered in textframes correctly and making sure the code that does it is robust and versatile enough to support all the features required.

oliverthered

2020-07-03 20:11

reporter   ~0047786

ok, multipage support is a quick fix. I'll provide an updated patch. PDF files seem to be read only when opened as files as opposed to being imported as vector files.

oliverthered

2020-07-03 20:25

reporter   ~0047787

find the attached path with multipage support. currently, y is offset by linespace when it should only be offset by the fonts ascending value which is slightly smaller.

pdftext-import-phase1.rar (13,469 bytes)

oliverthered

2020-07-06 17:35

reporter   ~0047791

this patch additionally fixes a problem with the grouping/z-order when importing multiple pages.

pdf-text-import-phase-1-with-multipage-support.rar (13,475 bytes)

oliverthered

2020-07-06 17:45

reporter   ~0047792

that patch breaks importing individual pages.

oliverthered

2020-07-06 18:54

reporter   ~0047793

this patch is correct

pdf-text-import-phase-1-with-multipage-support-2.rar (13,504 bytes)

oliverthered

2020-07-08 00:03

reporter   ~0047799

side by side with acrobat https://i.imgur.com/9zce4zh.png

oliverthered

2020-07-08 00:04

reporter  

2020-07-08.png (1,193,579 bytes)

oliverthered

2020-07-11 01:05

reporter   ~0047811

I'm going to stop working on PDF whilst I think about patent issues. I'm aggregating the text not placing it as PDF documents do so I doubt there will be any issues
 I'm going to take on a-l-e 's idea of placement due to this too and it was probably a good idea.
in theory you can always code around a patent, though it may not be elegant

oliverthered

2020-07-11 01:22

reporter   ~0047812

I also inadvertently breached copyright, but I think it's worth while giving the paper a little promotion as it's a good one.

oliverthered

2020-07-11 16:56

reporter   ~0047817

I'm going to put together a feature list. I'll use my commits to git to put it together and also provide a list of known outstanding features.

oliverthered

2020-07-11 16:59

reporter   ~0047818

i didn't breach copyright in the end.it came under uk fair use laws.

oliverthered

2020-07-11 17:35

reporter   ~0047819

******in this patch******
Merging free flow text into editable text frames
layout of text frames on the page matching the pdf document.
Correct scaling and width.
Support for multiple pages
******todo, including and initially features in other waiting patches******
UI for selecting text or vectors
automated font substitution
manual override font substitution
styles and fonts
colours
font scaling pixels to points
font scaling substitute font sizes to pdf font sizes
subscript text
superscript text
switching between subscript superscript and normal text
use mode average lineheight for the frame
bold and italic
removal of excess spaces from scanned documents
merging of text from scanned documents
flow-control: inserting of spaces at end of lines
flow-control: inserting of new paragraph markers
flow-control: inserting of carriage returns where needed at end of lines
flow-control: inserting of spaces where letter displacement is a significant amount greater than the letter width
getters and setters
use scfaces for fonts
font substitution including the font style
text overflows: extend textframes to match n*the lineheight
text overflows: nudge remaining overflowing textframes by one em width
iterate fonts at startup to preprepare font substitutions

******that's the list of completed features to merge.******

******this is the list of remaining features to add******

determine the kind of layout a text frame/textframes have.
support for grid-form layouts
support for pageheight when importing single pages
support for indented text to the right
support for indented text to the left
support for rotated text
removal of extra spaces added at the end of break characters.
better handling when text flows from one frame to the next with more than a line break inbetween
some font tidyups add some support for bold and italic that is missing
some font tidyups ensure scaling of text to match pdf text is working in all cases
recognize text as left aligned
recognize text as right aligned
recognise text as centred
support for justified text.
Recognize text that has an end of line less than the size of the first word on the next line as being a kind of list.

****** that's the end of the list ******

oliverthered

2020-07-11 17:48

reporter   ~0047820

matching columns and text continuation and adding scribus frame continuation links.

oliverthered

2020-07-11 22:11

reporter   ~0047822

can you wait a few days before patching please. minior legal issues again. but positive ones this time.

jghali

2020-07-12 00:02

administrator   ~0047823

Ok, I'll wait then.

jghali

2020-07-27 13:20

administrator   ~0047880

>> can you wait a few days before patching please. minor legal issues again.

Where are we now? Could you make the situation clear by explaining it here?

mara004

2020-10-02 17:58

reporter   ~0048123

I would be very interested in this feature (many thanks @oliverthered for implementing it). As far as I know, there currently is no other libre / open-source program which supports as much PDF editing as Scribus. The only thing which is missing to make it a fully-featured PDF editor are the text recognition capabilities.
I hope/think patching should be safe - a few days are definitely over and really I can't see any legal issues.
Kind regards and thanks for this useful software!

jghali

2020-10-02 21:27

administrator   ~0048124

>> The only thing which is missing to make it a fully-featured PDF editor are the text recognition capabilities.

Well, don't have too much expectation here : in lots of cases, importing text as text will not allow the original PDF visual aspect to be preserved unless you have the fonts embedded in the PDF installed on your system.

Issue History

Date Modified Username Field Change
2020-06-21 12:17 oliverthered New Issue
2020-06-21 12:17 oliverthered Tag Attached: import
2020-06-21 12:17 oliverthered Tag Attached: PDF
2020-06-21 12:17 oliverthered Tag Attached: pdf import
2020-06-21 12:17 oliverthered Tag Attached: vectorised fonts
2020-06-21 12:39 jghali Relationship added has duplicate 0016129
2020-06-21 12:39 jghali Relationship added has duplicate 0016128
2020-06-21 12:39 jghali Relationship added has duplicate 0016126
2020-06-21 12:40 jghali Relationship added has duplicate 0016125
2020-06-21 12:40 jghali Relationship added has duplicate 0016127
2020-06-21 12:56 jghali Relationship added has duplicate 0016103
2020-06-21 13:06 jghali Note Added: 0047710
2020-06-21 13:13 oliverthered Note Added: 0047711
2020-06-21 13:24 jghali Note Added: 0047712
2020-06-21 13:53 oliverthered Note Added: 0047713
2020-06-21 13:55 oliverthered Note Added: 0047714
2020-06-21 14:11 oliverthered Note Added: 0047715
2020-06-21 14:11 oliverthered Note Added: 0047716
2020-06-21 15:18 oliverthered Note Added: 0047717
2020-06-21 15:35 jghali Note Added: 0047718
2020-06-21 15:41 oliverthered Note Added: 0047719
2020-06-21 15:44 oliverthered Note Added: 0047720
2020-06-21 21:18 oliverthered Note Added: 0047721
2020-06-26 15:50 oliverthered File Added: pdf-text-import-phase1.rar
2020-06-26 15:51 oliverthered Note Added: 0047736
2020-06-26 16:11 oliverthered File Added: pdf-text-import-phase1-2.rar
2020-06-26 16:11 oliverthered Note Added: 0047737
2020-06-26 21:44 jghali Note Added: 0047738
2020-06-26 22:01 jghali Priority high => normal
2020-06-26 22:05 jghali Note Added: 0047739
2020-06-26 22:14 jghali Note Added: 0047741
2020-06-26 22:35 jghali Note Added: 0047742
2020-06-26 22:42 jghali Note Added: 0047743
2020-07-02 06:48 oliverthered File Added: 2020-07-02 (3).png
2020-07-02 06:48 oliverthered Note Added: 0047751
2020-07-02 06:58 oliverthered Note Added: 0047752
2020-07-02 07:00 oliverthered Note Added: 0047753
2020-07-02 07:09 oliverthered Note Added: 0047754
2020-07-02 07:17 oliverthered Note Added: 0047755
2020-07-02 08:37 oliverthered Note Added: 0047756
2020-07-02 10:44 oliverthered Note Added: 0047757
2020-07-02 10:50 oliverthered Note Added: 0047758
2020-07-02 10:59 oliverthered Note Added: 0047759
2020-07-02 11:15 jghali Note Added: 0047760
2020-07-02 11:38 oliverthered Note Added: 0047762
2020-07-02 12:03 oliverthered File Added: 2020-07-02 (13).png
2020-07-02 12:03 oliverthered File Added: 2020-07-02 (12).png
2020-07-02 12:03 oliverthered Note Added: 0047763
2020-07-02 14:54 oliverthered Note Added: 0047764
2020-07-02 15:14 oliverthered Note Added: 0047765
2020-07-02 15:33 oliverthered File Added: pdftext-phase1.rar
2020-07-02 15:33 oliverthered Note Added: 0047766
2020-07-02 16:51 oliverthered Note Added: 0047767
2020-07-02 17:31 oliverthered File Added: 2020-07-02 (15).png
2020-07-02 17:31 oliverthered Note Added: 0047768
2020-07-02 21:22 oliverthered Note Added: 0047770
2020-07-03 05:24 oliverthered File Added: 2020-07-03.png
2020-07-03 05:24 oliverthered Note Added: 0047771
2020-07-03 16:05 oliverthered Note Added: 0047774
2020-07-03 17:26 jghali Note Added: 0047778
2020-07-03 17:34 oliverthered Note Added: 0047780
2020-07-03 17:36 oliverthered Note Added: 0047781
2020-07-03 17:56 oliverthered Note Added: 0047782
2020-07-03 18:42 oliverthered Note Added: 0047785
2020-07-03 20:11 oliverthered Note Added: 0047786
2020-07-03 20:25 oliverthered File Added: pdftext-import-phase1.rar
2020-07-03 20:25 oliverthered Note Added: 0047787
2020-07-06 17:35 oliverthered File Added: pdf-text-import-phase-1-with-multipage-support.rar
2020-07-06 17:35 oliverthered Note Added: 0047791
2020-07-06 17:45 oliverthered Note Added: 0047792
2020-07-06 18:54 oliverthered File Added: pdf-text-import-phase-1-with-multipage-support-2.rar
2020-07-06 18:54 oliverthered Note Added: 0047793
2020-07-08 00:03 oliverthered Note Added: 0047799
2020-07-08 00:04 oliverthered File Added: 2020-07-08.png
2020-07-11 01:05 oliverthered Note Added: 0047811
2020-07-11 01:22 oliverthered Note Added: 0047812
2020-07-11 16:56 oliverthered Note Added: 0047817
2020-07-11 16:59 oliverthered Note Added: 0047818
2020-07-11 17:35 oliverthered Note Added: 0047819
2020-07-11 17:48 oliverthered Note Added: 0047820
2020-07-11 22:11 oliverthered Note Added: 0047822
2020-07-12 00:02 jghali Note Added: 0047823
2020-07-27 13:20 jghali Note Added: 0047880
2020-10-02 17:58 mara004 Note Added: 0048123
2020-10-02 21:27 jghali Note Added: 0048124