View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0016142 | Scribus | Import / Export | public | 2020-06-21 12:17 | 2021-03-23 06:25 |
Reporter | oliverthered | Assigned To | jghali | ||
Priority | normal | Severity | minor | Reproducibility | always |
Status | closed | Resolution | fixed | ||
Platform | all | ||||
Product Version | 1.5.6.svn | ||||
Fixed in Version | 1.5.7.svn | ||||
Summary | 0016142: Preliminary support for importing PDF text as text | ||||
Description | I have created the following pull request from my fork of the Scribus project: https://github.com/scribusproject/scribus/pull/135 Ale and I have been working on getting it ready over the last couple of weeks and I'm fairly confident it's good for reviewing and integrating with the main branch. It adds basic PDF text import functionality, just the text and basic layout functionality, no styles or fonts, right or centere3d justification, rotation, multipage etc... but it does provide enough of a base that most of the extra required functionality can be worked on as isolated features with low interdependencies. I hope the code is to your standard and liking, and we can get it merged quickly, so I can get on writing the extra required features. | ||||
Steps To Reproduce | Import a vector file in pdf format. Note that the text is vectors and you are unable to edit it in and reasonable fashion. | ||||
Additional Information | A final review and sign off by ale may be required. | ||||
Tags | import, PDF, pdf import, vectorised fonts | ||||
Patch | Yes | ||||
has duplicate | 0016129 | closed | jghali | basic textframe and text rendering |
has duplicate | 0016128 | closed | jghali | implement some more on the basic pdf text import, framework, abstract addchar and ui for selecting import method |
has duplicate | 0016126 | closed | jghali | refactor getChar so that it can be used for importing text as well as drawing vector characters |
has duplicate | 0016125 | closed | jghali | Implement a framework for importing text from pdf documents |
has duplicate | 0016127 | closed | jghali | override updateTextPos in SlaOutputDev |
has duplicate | 0016103 | closed | jghali | Update PDF import with features matching Inkscapes PDF import |
|
I had a quick look. The renaming of updateFont() method to updateFontForVector() looks wrong. The updateFont() method is indeed a virtual method of poppler OutputDev class which is called by poppler itself and needs to be reimplemented at least for the text as vectors case. |
|
ok, sounds like a plan. most of the functionality of SlaOutputDev needs to remain the same so I'll create a new outputdev class and inherit from SlaOutputDev overriding the methods needed for text import and just choose the correct outputdev class in bool PdfPlug::convert(const QString& fn) |
|
Hmm, I think this is a bit overkill. You will need that updateFont() method anyway to get proper font information in the end. |
|
oh, I've done it. I'll tidy the code up and see what you think. |
|
laOutputDev::updateFont loads the fonts from the pdf file I think, which I don't want I need to use scribus accessible fonts and will probably load all the fonts from the document and allow substitutions to be set in the pdf import UI. well, that's my current thoughts, font overrides need to happen some place and should be user configurable. |
|
I think implementing a new OutputDev based on SlaOutputDev gives the tidiest implementation and doesn't really have any overheads. |
|
I've pushed the changes now. |
|
the other consideration is that the implementation would be nice if it were easy to use in other projects without too much modification. so keeping it all encapsulated in it's own outputdev as opposed to being incorporated in SlaOutputDev achieves that goal too. |
|
TextOutputDev is already a poppler class name, you should better avoid it. |
|
ok, I've had a thought and there are a few more things from SlaOiutputDev that need to be implemented differently for importing text as opposed to vectors so cresting a new class makes sense. I'll change the name. |
|
ok, the name from TextOutputDev to PdfTextOutputDev. the PdfTextOutputDev naming matches the naming of PdfTextRecognition which is why I chose it. |
|
I've gone through and renames all the classes and member variables so they all start with 'PdfText' for consistency |
|
|
|
this patch should be good to go. there's a couple of bugs i fixed upstream but they should be in the next patch, the bugs don't stop functionanlity etc.. |
|
this patch should be better. it makes minimal changes to slaoutputdev and fixes a bug |
|
Unfortunately I cannot apply this patch: patch: **** malformed patch at line 1015: @@ -3359,8 +3341,9 @@ void SlaOutputDev::drawChar(GfxState *state, double x, double y, double dx, doub |
|
I finished the merge manually. I started doing some tests and found a bug : in most cases I do not see the dialog which would allow me to choose how text should be imported. |
|
There are lots of missing braces in PdfTextRegion::linearTest() too |
|
Hmmm, in most of my tests, when importing text as text the imported text is often not even imported at an approximately correct position. |
|
Try this file for example; https://drive.google.com/file/d/1pm_0kgsV9Q1RosYK5AkIIxLhFdc22g7r/view?usp=sharing When importing text as text, import fails pretty miserably :( |
|
my current working version that supports font and styles isn't too far off. I've not added support for displacement which is different from just the text position I'm currently using. |
|
most of the pdf files I have for testing at my end were all probably generated by latex. it would be good to get a broader selection of pdf files from different sources. |
|
What are the requirement for getting the code merged with scribus. I can add some more braces. ale said that there should be no braces if the if or else following is only one line. |
|
and for some of the documents a second parse over the layout is required because the text is separated by lots of begin/end text regions that should be merged together as a continuous block of text. |
|
my aim with the first patch certainly isn't perfection but enough of a base that features can be added to it fairly cleanly. |
|
here's a screenshot of my work in progress using the pdf file i've been using for the bulk of testing. After I've fixed the colours so they are presented correctly there should only be smaller features and bug fixes to be implemented. https://i.imgur.com/v6jXidp.png |
|
here's what the index page of the attached pdf look like in my current working version: https://i.imgur.com/4fmMMxe.png |
|
oops, that's imported as vectors not text. |
|
this is what it actually looks like : https://i.imgur.com/bzTzv57.png the baseline is correct it's just creating textframes that are too tall which makes it look more of a mess than it actually is. |
|
>> I can add some more braces. ale said that there should be no braces if the if or else following is only one line. In PdfTextRegion::linearTest() you have code like this: if (collinear(point.y(), lastXY.y())) if (collinear(point.x(), lastXY.x())) return LineType::FIRSTPOINT; else if (xInLimits) return LineType::SAMELINE; #ifdef DEBUG_TEXT_IMPORT else qDebug() << ...; #endif It's clear that the block related to the if (collinear(point.y(), lastXY.y())) condition uses more than one line, so code should be inside braces: if (collinear(point.y(), lastXY.y())) { if (collinear(point.x(), lastXY.x())) return LineType::FIRSTPOINT; else if (xInLimits) return LineType::SAMELINE; #ifdef DEBUG_TEXT_IMPORT else qDebug() << ...; #endif } You have similar blocks in same function which should be enclosed in braces. |
|
here's what it looks like after a small tweek: https://i.imgur.com/0lkvtuw.png to get it looking correct i'll have to lookup the linespacing of the first line from the font instead of making an assumption based on a multiple of dx. |
|
setting the linespacing based on the font height instead of a multiple of dx seems to have done the trick. but that's in the fonts and styles update not in the patch for this issue. |
|
I can backport some of the bug fixes that i did for fonts and styles into the initial patch if you want to have the first patch in better working order but the main fix for the attached pdf was replacing some of my guestimates with more intelligent ones based upon the font in use. |
|
ok, i'll do the braces in a few minutes. is there anything else? |
|
find patch attached. I've compiled and run this one so it should be ok this time. I checked and changes to support the pdf document attached to this thread require font support to get anything like a sensible import. |
|
I've had a look at the attached pdf file calls beginTextObject and endTextObject for pretty much every letter which is why there not getting merged. my plan for this was to import all the textframes for a page and then parse over them merging adjacent textframes or textframes that make up new lines in multiline text. |
|
if I ignore begin/end text object and the cursor being reset to the origin I get the following result: This is the result I'd expect after parsing over all the textframes and merging appropriately. |
|
I'm currently implementing the code to do the merging. that will mean the attached pdf file should import properly. |
|
I got the merge code working successfully. There are still some flow control issues e.g. no line breaks. But they're not implemented anywhere yet. See attached screenshot. |
|
I'm going to add some bespoke code to manage tables as they are quite a common feature and it would be nice to have them laid out properly. |
|
I see one last major issue with the last patch you uploaded: when you open (with file > open) a PDF containing multiple pages, after first page text frames are not created at correct position. In fact they seem to be placed before first page. |
|
yeah, multiple pages aren't supported at the moment. it's on my list as the next thing to support. |
|
my comment in the description for this bug was "It adds basic PDF text import functionality, just the text and basic layout functionality, no styles or fonts, right or centere3d justification, rotation, multipage etc... but it does provide enough of a base that most of the extra required functionality can be worked on as isolated features with low interdependencies." |
|
I've a few more patches all but ready too. one for supporting fonts and styles, one to support merging of erratically placed content into textframes, and one that adds spaces to the end of the lines and line breaks so the text flows properly. |
|
I've just been focused on getting the text rendered in textframes correctly and making sure the code that does it is robust and versatile enough to support all the features required. |
|
ok, multipage support is a quick fix. I'll provide an updated patch. PDF files seem to be read only when opened as files as opposed to being imported as vector files. |
|
find the attached path with multipage support. currently, y is offset by linespace when it should only be offset by the fonts ascending value which is slightly smaller. |
|
this patch additionally fixes a problem with the grouping/z-order when importing multiple pages. |
|
that patch breaks importing individual pages. |
|
this patch is correct |
|
side by side with acrobat https://i.imgur.com/9zce4zh.png |
|
|
|
I'm going to stop working on PDF whilst I think about patent issues. I'm aggregating the text not placing it as PDF documents do so I doubt there will be any issues I'm going to take on a-l-e 's idea of placement due to this too and it was probably a good idea. in theory you can always code around a patent, though it may not be elegant |
|
I also inadvertently breached copyright, but I think it's worth while giving the paper a little promotion as it's a good one. |
|
I'm going to put together a feature list. I'll use my commits to git to put it together and also provide a list of known outstanding features. |
|
i didn't breach copyright in the end.it came under uk fair use laws. |
|
******in this patch****** Merging free flow text into editable text frames layout of text frames on the page matching the pdf document. Correct scaling and width. Support for multiple pages ******todo, including and initially features in other waiting patches****** UI for selecting text or vectors automated font substitution manual override font substitution styles and fonts colours font scaling pixels to points font scaling substitute font sizes to pdf font sizes subscript text superscript text switching between subscript superscript and normal text use mode average lineheight for the frame bold and italic removal of excess spaces from scanned documents merging of text from scanned documents flow-control: inserting of spaces at end of lines flow-control: inserting of new paragraph markers flow-control: inserting of carriage returns where needed at end of lines flow-control: inserting of spaces where letter displacement is a significant amount greater than the letter width getters and setters use scfaces for fonts font substitution including the font style text overflows: extend textframes to match n*the lineheight text overflows: nudge remaining overflowing textframes by one em width iterate fonts at startup to preprepare font substitutions ******that's the list of completed features to merge.****** ******this is the list of remaining features to add****** determine the kind of layout a text frame/textframes have. support for grid-form layouts support for pageheight when importing single pages support for indented text to the right support for indented text to the left support for rotated text removal of extra spaces added at the end of break characters. better handling when text flows from one frame to the next with more than a line break inbetween some font tidyups add some support for bold and italic that is missing some font tidyups ensure scaling of text to match pdf text is working in all cases recognize text as left aligned recognize text as right aligned recognise text as centred support for justified text. Recognize text that has an end of line less than the size of the first word on the next line as being a kind of list. ****** that's the end of the list ****** |
|
matching columns and text continuation and adding scribus frame continuation links. |
|
can you wait a few days before patching please. minior legal issues again. but positive ones this time. |
|
Ok, I'll wait then. |
|
>> can you wait a few days before patching please. minor legal issues again. Where are we now? Could you make the situation clear by explaining it here? |
|
I would be very interested in this feature (many thanks @oliverthered for implementing it). As far as I know, there currently is no other libre / open-source program which supports as much PDF editing as Scribus. The only thing which is missing to make it a fully-featured PDF editor are the text recognition capabilities. I hope/think patching should be safe - a few days are definitely over and really I can't see any legal issues. Kind regards and thanks for this useful software! |
|
>> The only thing which is missing to make it a fully-featured PDF editor are the text recognition capabilities. Well, don't have too much expectation here : in lots of cases, importing text as text will not allow the original PDF visual aspect to be preserved unless you have the fonts embedded in the PDF installed on your system. |
|
<oliverthered> OK, I have a couple more bumps from that in my repository that need to be merged. One adds the rest of basic PDF support and the other de-noises OCDed documents. There's still some work on tabular and list layouts to be done, but I was hitting quite a backlog of stuff to be merged, so I stopped. I also have to change the code over to using C++ algorithms where possible. <oliverthered> The code I'm working on in Inkscape may well help with tabular layouts and lists. <MrB> ok, our biggest concern right now is that it is unclear about the current state with copyright, patents and licencing to include into Scribus <oliverthered> the only thing that may fall under patent is a noise harmonizing algorithm. But I'm only intending on enforcing any kind of licensing requirements on closed source software, it's free to open software with no string attached. I think it's real potential will come from hardware anyway.. That and tidying up the huge amount of OCRd documents held by institutions such as libraries. <oliverthered> I wrote all the code myself except for a little help from ale, so there's no copyright issues, it's <oliverthered> it's all under GPL. <oliverthered> it doesn't change the PDF file format, so there are no licensing issues with adobe. <oliverthered> The patent application I'm putting together only lists hardware implementations. I think that also covers software implementations of hardware. |
|
After dropping the additional patches for free flowing text and noisy text such as OCRd document, the last major thing to attack is Grid, tabular and list layouts that are a lot looser than free-flow text and currently fall into the category of 'things that should be kept separate from each other in the document'. This is good because it means they are kept separate, so the left overs, so to speak, should be quite easy to pick up.. My plan is to use a matrix that starts off fairly large and gets progressively smaller and count the number of squares that have something in them for each column and row. In theory the grid should never need to get smaller than one glyph in size but could be as large as say, two thirds of the page width/height with the layout fitting into one bounded area. Here's my example: https://imgur.com/a/XuTiETe |
|
This is what some grids in that imgur link look like when they are filled in. all that has to be done is a cell marked if there is something in it and then look for columns and rows in the white space. A little noise also has to be considered to do the job well. |
|
The code needs to be able to look for horizontal or vertical repeats with approximately the same spacing and a header and footer or margin elements that are aligned but where the spacing is out. For some tables the distribution won't be evenly spaced anyway, but where the distribution is evenly spaced you can say that there's a good chance that the text within the bounds of those areas belong together. |
|
tables that don't have even, say, with, spacing should however have identical column tops and bottoms. |
|
It's fairly simple just a little bit fiddly. For instance, squares with a little erratic content can be taken as empty if they disrupt what would otherwise be an empty line. It gets tricky drawing a good balance between saying something should be ignored or not ignored. Creating a higher resolution grid should solve that issue though because you'd expect at least one path to become clear when the resolution is increased. I may knock up a quick prototype, if I'm feeling brave I'll make it so it only increases the resolution for squares that have something in them. |
|
I implemented this in JavaScript but I don't think the bug tracker will let me post HTML or JavaScript. The JavaScript is here: https://pastebin.com/iVEGq6Ec The HTML is here: https://pastebin.com/fY1qT42S And the linked and only sensible image I have for testing is attached... The only things that need fixing in this version are a: the tolerances, b: if in tolerance do we then check to see if any non-empty cells were noise or something else c: bail out before all the grid matrix are processed if enough rows/cols are found matching empty criteria. |
|
@oliverthered: I think we should be careful about the various patches. Please confirm that pdf-text-import-phase-1-with-multipage-support-2.rar is still the correct first patch for review. Any others should be considered as additional on top, right? |
|
I now have applied the patch (with a few code style fixes). |
Date Modified | Username | Field | Change |
---|---|---|---|
2020-06-21 12:17 | oliverthered | New Issue | |
2020-06-21 12:17 | oliverthered | Tag Attached: import | |
2020-06-21 12:17 | oliverthered | Tag Attached: PDF | |
2020-06-21 12:17 | oliverthered | Tag Attached: pdf import | |
2020-06-21 12:17 | oliverthered | Tag Attached: vectorised fonts | |
2020-06-21 12:39 | jghali | Relationship added | has duplicate 0016129 |
2020-06-21 12:39 | jghali | Relationship added | has duplicate 0016128 |
2020-06-21 12:39 | jghali | Relationship added | has duplicate 0016126 |
2020-06-21 12:40 | jghali | Relationship added | has duplicate 0016125 |
2020-06-21 12:40 | jghali | Relationship added | has duplicate 0016127 |
2020-06-21 12:56 | jghali | Relationship added | has duplicate 0016103 |
2020-06-21 13:06 | jghali | Note Added: 0047710 | |
2020-06-21 13:13 | oliverthered | Note Added: 0047711 | |
2020-06-21 13:24 | jghali | Note Added: 0047712 | |
2020-06-21 13:53 | oliverthered | Note Added: 0047713 | |
2020-06-21 13:55 | oliverthered | Note Added: 0047714 | |
2020-06-21 14:11 | oliverthered | Note Added: 0047715 | |
2020-06-21 14:11 | oliverthered | Note Added: 0047716 | |
2020-06-21 15:18 | oliverthered | Note Added: 0047717 | |
2020-06-21 15:35 | jghali | Note Added: 0047718 | |
2020-06-21 15:41 | oliverthered | Note Added: 0047719 | |
2020-06-21 15:44 | oliverthered | Note Added: 0047720 | |
2020-06-21 21:18 | oliverthered | Note Added: 0047721 | |
2020-06-26 15:50 | oliverthered | File Added: pdf-text-import-phase1.rar | |
2020-06-26 15:51 | oliverthered | Note Added: 0047736 | |
2020-06-26 16:11 | oliverthered | File Added: pdf-text-import-phase1-2.rar | |
2020-06-26 16:11 | oliverthered | Note Added: 0047737 | |
2020-06-26 21:44 | jghali | Note Added: 0047738 | |
2020-06-26 22:01 | jghali | Priority | high => normal |
2020-06-26 22:05 | jghali | Note Added: 0047739 | |
2020-06-26 22:14 | jghali | Note Added: 0047741 | |
2020-06-26 22:35 | jghali | Note Added: 0047742 | |
2020-06-26 22:42 | jghali | Note Added: 0047743 | |
2020-07-02 06:48 | oliverthered | File Added: 2020-07-02 (3).png | |
2020-07-02 06:48 | oliverthered | Note Added: 0047751 | |
2020-07-02 06:58 | oliverthered | Note Added: 0047752 | |
2020-07-02 07:00 | oliverthered | Note Added: 0047753 | |
2020-07-02 07:09 | oliverthered | Note Added: 0047754 | |
2020-07-02 07:17 | oliverthered | Note Added: 0047755 | |
2020-07-02 08:37 | oliverthered | Note Added: 0047756 | |
2020-07-02 10:44 | oliverthered | Note Added: 0047757 | |
2020-07-02 10:50 | oliverthered | Note Added: 0047758 | |
2020-07-02 10:59 | oliverthered | Note Added: 0047759 | |
2020-07-02 11:15 | jghali | Note Added: 0047760 | |
2020-07-02 11:38 | oliverthered | Note Added: 0047762 | |
2020-07-02 12:03 | oliverthered | File Added: 2020-07-02 (13).png | |
2020-07-02 12:03 | oliverthered | File Added: 2020-07-02 (12).png | |
2020-07-02 12:03 | oliverthered | Note Added: 0047763 | |
2020-07-02 14:54 | oliverthered | Note Added: 0047764 | |
2020-07-02 15:14 | oliverthered | Note Added: 0047765 | |
2020-07-02 15:33 | oliverthered | File Added: pdftext-phase1.rar | |
2020-07-02 15:33 | oliverthered | Note Added: 0047766 | |
2020-07-02 16:51 | oliverthered | Note Added: 0047767 | |
2020-07-02 17:31 | oliverthered | File Added: 2020-07-02 (15).png | |
2020-07-02 17:31 | oliverthered | Note Added: 0047768 | |
2020-07-02 21:22 | oliverthered | Note Added: 0047770 | |
2020-07-03 05:24 | oliverthered | File Added: 2020-07-03.png | |
2020-07-03 05:24 | oliverthered | Note Added: 0047771 | |
2020-07-03 16:05 | oliverthered | Note Added: 0047774 | |
2020-07-03 17:26 | jghali | Note Added: 0047778 | |
2020-07-03 17:34 | oliverthered | Note Added: 0047780 | |
2020-07-03 17:36 | oliverthered | Note Added: 0047781 | |
2020-07-03 17:56 | oliverthered | Note Added: 0047782 | |
2020-07-03 18:42 | oliverthered | Note Added: 0047785 | |
2020-07-03 20:11 | oliverthered | Note Added: 0047786 | |
2020-07-03 20:25 | oliverthered | File Added: pdftext-import-phase1.rar | |
2020-07-03 20:25 | oliverthered | Note Added: 0047787 | |
2020-07-06 17:35 | oliverthered | File Added: pdf-text-import-phase-1-with-multipage-support.rar | |
2020-07-06 17:35 | oliverthered | Note Added: 0047791 | |
2020-07-06 17:45 | oliverthered | Note Added: 0047792 | |
2020-07-06 18:54 | oliverthered | File Added: pdf-text-import-phase-1-with-multipage-support-2.rar | |
2020-07-06 18:54 | oliverthered | Note Added: 0047793 | |
2020-07-08 00:03 | oliverthered | Note Added: 0047799 | |
2020-07-08 00:04 | oliverthered | File Added: 2020-07-08.png | |
2020-07-11 01:05 | oliverthered | Note Added: 0047811 | |
2020-07-11 01:22 | oliverthered | Note Added: 0047812 | |
2020-07-11 16:56 | oliverthered | Note Added: 0047817 | |
2020-07-11 16:59 | oliverthered | Note Added: 0047818 | |
2020-07-11 17:35 | oliverthered | Note Added: 0047819 | |
2020-07-11 17:48 | oliverthered | Note Added: 0047820 | |
2020-07-11 22:11 | oliverthered | Note Added: 0047822 | |
2020-07-12 00:02 | jghali | Note Added: 0047823 | |
2020-07-27 13:20 | jghali | Note Added: 0047880 | |
2020-10-02 17:58 | mara004 | Note Added: 0048123 | |
2020-10-02 21:27 | jghali | Note Added: 0048124 | |
2020-11-16 08:58 | oliverthered | Note Added: 0048427 | |
2020-11-16 09:06 | oliverthered | Note Added: 0048428 | |
2020-11-16 09:33 | oliverthered | File Added: table and liost detection - 3 - filled inwith guides.png | |
2020-11-16 09:33 | oliverthered | File Added: table and liost detection - 1 - filled.png | |
2020-11-16 09:33 | oliverthered | Note Added: 0048429 | |
2020-11-16 09:37 | oliverthered | Note Added: 0048430 | |
2020-11-16 09:42 | oliverthered | Note Added: 0048431 | |
2020-11-16 09:50 | oliverthered | Note Added: 0048432 | |
2020-11-16 13:30 | oliverthered | File Added: 2020-07-02 (12)-2.png | |
2020-11-16 13:30 | oliverthered | Note Added: 0048438 | |
2020-11-16 21:06 | cbradney | Note Added: 0048439 | |
2021-01-04 21:24 | jghali | Summary | review and merge pull request 0000135 to begin supporting text import from PDF files => Preliminary support for import PDF text as text |
2021-01-04 21:32 | jghali | Assigned To | => jghali |
2021-01-04 21:32 | jghali | Status | new => resolved |
2021-01-04 21:32 | jghali | Resolution | open => fixed |
2021-01-04 21:32 | jghali | Fixed in Version | => 1.5.7.svn |
2021-01-04 21:32 | jghali | Note Added: 0048629 | |
2021-01-04 23:10 | jghali | Summary | Preliminary support for import PDF text as text => Preliminary support for importing PDF text as text |
2021-03-23 06:25 | cbradney | Status | resolved => closed |