View Issue Details

IDProjectCategoryView StatusLast Update
0014960ScribusPDFpublic2017-08-25 19:30
Reporterchandra Assigned To 
PrioritynormalSeverityfeatureReproducibilityalways
Status newResolutionopen 
PlatformWindowsOS10OS Version1607
Product Version1.5.3 
Target VersionFixed in Version 
Summary0014960: Tagged PDF and accessibility
DescriptionAdobe AGLFN open type font glyph name standard helps to copy to the text from PDF exported in adobe indesign.

In Indesign, when a PDF is exported with PDF 1.5 or higher, Indesign has the ability to add content and visible layer. In the PDF, search engine can read the text and also users can copy the text from it.

1. PDF should contain "visible and printable layers" (content and presentation layer)

2. Also PDF export needs to add PDF version 1.6 and 1.7
Steps To ReproduceCreate a document with Indic unicode script using font that uses "adobe glyph name for new fonts" standard

Export as PDF

Copy text from PDF.

Check if the text copied from PDF matches the original text.
Additional Informationhttps://github.com/adobe-type-tools/agl-aglfn/issues/1#issuecomment-323838956

https://helpx.adobe.com/acrobat/using/pdf-layers.html

https://github.com/adobe-type-tools/agl-aglfn/
TagsNo tags attached.
PatchNo

Activities

chandra

2017-08-22 06:43

reporter  

jghali

2017-08-22 12:55

administrator   ~0044348

Last edited: 2017-08-22 13:26

View 3 revisions

There is some confusion here. AGLFN is to be used by font developers, PDF producers rarely need to use it. Also as the Postscript glyph name table provided by TTF fonts is notoriously unreliable, we have often to perform output in such a way we do not use glyph names at all for the font encoding (this is true for 1.5.x, 1.4.x relied more on glyph names availability). When possible, we use already font provided glyph names to provide a glyph => unicode mapping. This is the /ToUnicode key of PDF font dictionnary. But as you have already seen, this way of doing does not work really well with complex scripts.

These "content and presentation layers" are technically not layers in the Scribus sense or even in the Acrobat sense (Acrobat layers are called OCG, optional content groups). PDF ISO 32000 refers to this "content layer" feature as Tagged PDF (14.8) and Accessibility (14.9). The part which is more specifically interesting for copy/paste is 14.9.4 Replacement Text. And glyph names are not needed at all here...

chandra

2017-08-22 14:50

reporter   ~0044349

I am not technically competent to understand. Please follow the below link.

https://github.com/adobe-type-tools/agl-aglfn/issues/1#issuecomment-322659661

Nonetheless, you already know what needs to be done.

FYI: PDF generated by Indesign has issue copying/pasting the non-printing characters such as zero-width-space, zero-width-non-joiner and zero-width-joiner.

As you might be well aware these characters are part of Indic script. With and without these characters, meaning of the word changes.
Please ensure even these characters are taken care.

My mistake I was under the impression, text copying/pasting from PDF is related to glyph names.

chandra

2017-08-22 15:48

reporter   ~0044350

FYI: Second thought.

I was able to extract text with font formatting from PDF created in Indesign.
I used PDF 1.5 in InDesign same as in scribus.
with the PDF, generated from InDesign, I was able to extracted all the content.
But PDF generated from Scribus extracted only few characters.

Please refer to PDF 1.5 under section 5.9 "Extraction of Text Content"

http://wwwimages.adobe.com/content/dam/Adobe/en/devnet/pdf/pdfs/pdf_reference_archives/PDFReference15_v5.pdf

jghali

2017-08-22 16:55

administrator   ~0044351

Please upload a sample scribus document and give us the download link(s) to used font(s). That would help in any case as none of the core team developers has knowledge of Indic scripts.

jghali

2017-08-24 21:25

administrator   ~0044357

I noticed that we were not creating the PDF ToUnicode glyph => unicode mapping when creating Type 3 font subsets (this is what we use when subsetting lots of OpenType fonts for asian languages). This is now fixed. I created new 1.5.4.svn snapshots for Windows so you have a try. You can download them here:
https://sourceforge.net/projects/scribus/files/scribus-svn/1.5.4.svn/

This snapshot should improve your copy/paste problem. But as said, this way of doing has limitations for complex scripts and will probably not be perfect.

chandra

2017-08-24 23:14

reporter   ~0044358

@jghali

Thanks for the prompt reply.

Unfortunately, this latest 1.5.4 build has not improved the situation.

I have attached the PDF created from Adobe Indesign, Scribus and scribus document.

Please open both the PDF's in latest version of Adobe Reader DC.

Copy the text and paste it into any unicode supported text editor. You will find the difference in result.

Also, know both the source text and font used are same between indesign and scribus.

PDF compatibility type used was PDF 1.5.

Font: Hubballi
https://github.com/erinmclaughlin/Hubballi/blob/master/build/Hubballi-Regular.otf
ttf version is also available for testing, in the same folder.

I have used otf version.

Adobe_Indesign_Test1.pdf (33,414 bytes)
Scribus_test1.pdf (38,826 bytes)
Scribus_test1.sla (17,709 bytes)

jghali

2017-08-25 13:06

administrator   ~0044360

Given the font you used, you have indeed not seen any difference with the lastest snapshot as Hubballi is not a CID-keyed font and hence was not triggering the Type3 subsetting I had modified.

Thanks to the font information you provided, I could perform more tests and noticed Hubballi had lots of ligatures and that our code generating the PDF ToUnicode map was not properly handling those. I have modified the code and now things seems to work better, at least for Type 1 fonts and OpenType fonts with Postscript outlines. When glyph name is of the form uniXXXXYYYY (for eg.), I now extract the unicode hex string from glyph name and hence can generate a better ToUnicode map. There will still be issues with TTF based fonts where I can hardly use glyph names given the unreliability of glyph name table in those fonts.

I have generated new snapshots you can download at the usual address:
https://sourceforge.net/projects/scribus/files/scribus-svn/1.5.4.svn/

chandra

2017-08-25 17:12

reporter   ~0044361

@jghali

Great, this is lot better.

Still we few have issues.

1. PDF from Indesign picks up all gsub, and gpos features perfectly. I don't know what it is writing.
blwf passed in Scribus, and Indesign:
ಕನ್ನಡ ರಾಜ್ಯ
blwf with abvs failed in Scribus and passed in Indesign:
ಪಶ್ಚಿಮ ತೀರದಲ್ಲಿರುವ ದಕ್ಷಿಣ
Failed: uni200c and uni200d
reph failed in scribus: ನಟಸಾರ್ವಭೌಮ
reph is repositioned in scribus.

There are few more issues, with above symptoms, we should be able to zero in on the issue.

2. PDF from Indesign picks up the font name and size, scribus doesn't.

3. PDF from Indesign picks up the paragraph (continuous line). Scribus breaks the line at each line.

Everything that we can fixing here applies to all Indic fonts, which will be a great welcome.

jghali

2017-08-25 19:23

administrator   ~0044362

Last edited: 2017-08-25 19:30

View 2 revisions

1. At this point, I believe we'll need the Tagged PDF feature. This features allows to have a copy of the original text in the PDF. The only way PDF readers have to copy text from Scribus PDFs is currently the ToUnicode map created for each font. The PDF reader use this map to map each glyph to a unicode value. But this map can have some ambiguities, hence the need of tagged PDF.

2. Cannot confirm that. With current scribus output, if I copy the content from Adobe Acrobat and paste it into MS Word, fonts and sizes are picked up as well.

3. Same answer as 1.

Issue History

Date Modified Username Field Change
2017-08-22 06:43 chandra New Issue
2017-08-22 06:43 chandra File Added: Export PDF in InDesign.png
2017-08-22 12:55 jghali Note Added: 0044348
2017-08-22 12:57 jghali Note Edited: 0044348 View Revisions
2017-08-22 13:26 jghali Note Edited: 0044348 View Revisions
2017-08-22 13:28 jghali Summary AGLFN in content layer in PDF for unicode => Tagged PDF and accessibility
2017-08-22 14:50 chandra Note Added: 0044349
2017-08-22 15:48 chandra Note Added: 0044350
2017-08-22 16:55 jghali Note Added: 0044351
2017-08-24 21:25 jghali Note Added: 0044357
2017-08-24 23:14 chandra File Added: Adobe_Indesign_Test1.pdf
2017-08-24 23:14 chandra File Added: Scribus_test1.pdf
2017-08-24 23:14 chandra File Added: Scribus_test1.sla
2017-08-24 23:14 chandra Note Added: 0044358
2017-08-25 13:06 jghali Note Added: 0044360
2017-08-25 17:12 chandra Note Added: 0044361
2017-08-25 19:23 jghali Note Added: 0044362
2017-08-25 19:30 jghali Note Edited: 0044362 View Revisions