13 February 2013

338. Annotating PDFs in linux -- revisited. Still no obvious solution

Update 28/02/2014: I've had a look at I, Librarian and Master PDF Editor here: http://verahill.blogspot.com.au/2014/02/558-more-options-for-pdf-annotation.html

There are a a few main reasons why even platform agnostic people have trouble moving to linux. One of them is poor compatibility of open solutions with MS Office (incompatibility between MS Equations and libre/openoffice is my main gripe, and some may find the lack of native EndNote an issue), and one quite particular to academics is the lack of proper pdf annotation (traditionally you edit your galley proofs by making annotations on the pdf -- my latest paper from Wiley came back with a doc file though, which was a bit surprising, but promising)

I've gotten by in the past by using 'pdf x-change viewer', which is a windows program, under wine. However, there are a lot of things I don't like about that solution, and the search for a native solution has continued to the point where I'm willing to throw money at it.

Why the exercise is unfair
The problem is that our definition of what 'works' and what doesn't is based on what we expect it to look like -- and that's based on our experience. I have colleagues that consider libreoffice 'crap' because it behaves or looks different from MS Office. While I agree that it's not viable as a replacement to office when collaborating with Office users who refuse to use libreoffice, it works fine if libreoffice is all you use. Anyway, it's unfair. Same goes for pdf annotation -- we expect it to look and function like adobe acrobat, simply because that's what we're used to. Any deviation is akin to a bug.

So keep that in mind when I write off some of the alternatives that actually do work -- just not in the way I expect them to.

I've found a lot of people (online) swearing by e.g.  Xournal, so even if it doesn't work for me, you may decide differently.

Commercial software in academia

I don't normally like buying software which is critical to my work. There are many reasons for it, and cost is only a minor one (although I don't like spending tax payer money on overpriced software).

Instead, if the software is critical to the science that I'm doing, I prefer to write my own algorithms in octave or python, and if it's a bit more peripheral I'm still wary of becoming reliant on a piece of software that may one day disappear -- either because I'm forced to upgrade through planned obsolescence, or because the company goes bankrupt/discontinues the software without releasing it as open source.

I cannot recommend Mendeley. And that's for two reasons:
1. it installs a list file in /etc/apt/sources.list.d/ without asking. That's the way malware (and google...) behaves!
2. There's plenty of mention about how it's 'free', but the free version is very restrictive, and if you create a private group you're stuck with a nagging message saying that you "must" upgrade if you want to add more users/groups. A simple one-time message would suffice.

Maybe I'm overreacting, but I don't appreciate this behaviour at all. Beyond that the annotation function works fine, so you decide for yourself.

Anyway, here's a short list of programs I've considered:
(and FOSS= Free and Open Source Software)



Evince
Evince is FOSS and now (since when?) supports adding annotations. It's not working very well (sometimes doesn't save annotation, slow), and you can't delete annotations, so be careful what you're saving.
Evince


pdfedit
pdfedit is another standard linux package. It does highlighting well, but annotates by adding text on top of the document -- not as a sticky note. FOSS.
pdfedit


flpsed
Does annotation, but as text superimposed on the pdf document, not as a collapsible sticky note. FOSS..

Qoppa PDF Studio
This is java based, and runs on Linux. I must already now say that there's one aspect of it that I really don't like: it's available in a Standard and a Professional version. That's the kind of artificial crippling of software that Microsoft likes to engage in, but I though we were beyond that on Linux...the price, $89, is too steep for something that I'd only use for annotations. Note also that the trial version puts a big nasty watermark over everything -- but you can hardly fault them for that, since it's not free. Closed source. Commercial.
Other than that it works, although it's not as pretty as mendeley.

Qoppa PDF studio


Xournal
Xournal can export annotated PDFs, but it doesn't do annotation in the same way as the other programs i.e. using sticky notes. Instead you can simply add text on top of the pdf, and it doesn't really do it for me. FOSS..
Xournal -- the annotations are not easy to spot


Whyteboard
It draws on top of PDF using imagemagick. FOSS.

Mendeley
While it's meant mainly for reference management, this does proper PDF annotation as well, and is platform agnostic. However, it is closed source, requires you to log in (even if you're using the desktop client) every time you use it, and needs you to explicitly keep documents (at least their titles etc.) out of the shared web catalogue. Other than that the pdf annotation works beautifully. I get really annoyed by the requirement to log in even when working offline though. It's free in the sense of gratis though -- but only up to 100 Mb of shared document space, you can only have one private group, and it can only have two members (+you). If you want more you need to pay (see e.g. here).
In practical terms, it seems to use GMT to time stamp annotationsand I haven't found an obvious way
of changing that (without going online). Also, it installs a file into /etc/apt/sources.list.d/ without asking.
Mendeley


Misc
Other 'solutions' that pop up is Okular which work in a roundabout way -- i.e the annotations aren't stored as part of the pdf. Again, it looks pretty, but the annotations are not exported with the pdf.
UPDATE: Note that this doesn't seem to be an issue anymore -- see comments below and this post: https://groakat.wordpress.com/2013/08/27/annotating-pdf-with-okular/ -- note that the new version is NOT in either wheezy or jessie i.e. they won't work.

okular


I tried FoxIt reader as well which claims to do annotation but doesn't on Linux -- and the windows version is not functional under linux/wine.

PDF X-change viewer/wine pops up so often as a suggested solution that it's beginning to look like spam. It does work though:
PDF Xchange viewer


An online only option is http://www.pdfescape.com -- but the paranoid part of me doesn't like the idea of uploading documents that are meant to be private.

38 comments:

  1. Thanks so much for the breakdown! Now that Mendeley has been bought up by Elsevier, I'm looking for a better solution for both reference management and PDF annotation.

    It's looking like my solution will be something along the lines of JabRef for reference management, Evince or one of the other solutions you posted above for annotation, and Org-Mode tying the BibTeX file to the PDFs and offering tags, etc.

    Oy vey!

    ReplyDelete
    Replies
    1. You should give Master PDF editor a try. It's cross-platform and also comes packaged as deb files. Does pretty much everything highlights, notes, strikethrough, etc.

      Delete
    2. Forgot the link to Master PDF earlier http://code-industry.net/pdfeditor.php

      Delete
    3. I reviewed Master PDF editor a while ago. The link is here: http://verahill.blogspot.com.au/2014/02/558-more-options-for-pdf-annotation.html

      Delete
  2. @gabe -- maybe I misunderstand you, but you can use Jabref for both of what I think your last two points are. Using the gui, take the 3rd tab along (general), use the "file" box to point to the file and stick your tags in the "keywords" box. Now you can search for those tags, and a little icon will appear in the file column of the big list above that you can click on to open the file.

    So Jabref "replaces" Mendelay fine for all but annotation.

    It is so annoying that no linux software properly supports pdf comments, though, especially since there actually is android software that does! Ezpdf does the same format as Adobe Acrobat, interoperable with it.

    ... you can't run Android software under linux, easily, can you?

    ReplyDelete
    Replies
    1. Gabe, ELizabeth,
      thanks for sharing your workflow.

      Doing a bit of searching it seems that the only options for running closed source android apps on linux is via emulator or vm, which aren't real options since you might as well use wine instead.

      Delete
  3. Reportedly, recent versions of Okular support saving annotations in the PDF file: http://docs.kde.org/stable/en/kdegraphics/okular/annotations.html. You need to have Okular 0.15 and Poppler 0.20 or higher. I haven't tested this though.

    ReplyDelete
    Replies
    1. I'll look into it.However, the faq at http://okular.kde.org/faq.php still states

      "Short answer: Okular cannot change the annotations in PDF documents.

      Longer answer: the library used for reading PDF documents (Poppler) does not allow changing the annotations inside a PDF document. Also, due to the fact that Okular allows you to annotate in any kind of document it supports (even if that format does not support annotations), Okular saves the annotations internally in the local data directory for each user."

      In addition, a brief search didn't yield anything. If you have a link, please share it and I'll test it.

      Delete
    2. Ignore the comment about the link -- you did provide one already.

      Delete
    3. I had a look at Okular 0.16.2 + poppler 0.22-3 (on arch linux) and it seems that the rumours are partially true. At this point you can make inline notes, which are saved as images in the document. However, adding 'normal' post-it style notes doesn't work properly -- they can be added ok, but if you open the document in evince you can't expand the notes to actually read them.

      Delete
    4. I can confirm that annotations using sticky notes, inline highlighting, etc. work with Okular 0.16.2 and poppler 0.22.2. They can be saved in the PDF and are recognized correctly by adobe reader etc. At last I can stop using the x-change viewer for this :).

      Delete
    5. Cheers Martin. Did you compile it yourself or did it come with your distro (e.g. Arch)?

      Delete
    6. It works with Okular 0.17 and poppler 0.22. Maybe even earlier versions. The trick is to use "save as" after annotating to embed the annotations in a pdf. It is described in the Okular handbook. But I think the process should be easier and for now be placed more prominent on their site.

      I tested my annotations with the Adobe Reader on my Android. I can open and view the annotations and alter them as well. So I suppose its according to the standard.

      Delete
    7. Groakat,
      thanks for the information -- while it's been possible to annotate for a little while now using different techniques, the main remaining problem has been that the annotations haven't been editable. It seems from your comment that that's no longer the case, which is very promising indeed.

      Delete
    8. I know it's quite an old thread... but I checked with the normal okular in Ubuntu 13.10 and it works ok (this can be useful for future searches...).
      The real nice thing would be to mix Okular with Xournal --- and I would be happy.

      Delete
    9. I don't mind necrobumping -- some of the most popular pages on this blog are some pretty old ones, so all feedback and updates are welcome.

      Delete
    10. I agree that Xournal puts the comments in a different file. However, you can always print to PDF and you get a PDF with all the comments on it. That is the way I have been operating for a few years now. Although not as good as Acrobat, it is what has worked for me. I tried PDF Xchange / wine for a while as well but it collapsed a few times on me so I decided to drop it (as well as wine since that was the only Windows application I was using).

      Delete
  4. No native EndNote? Why use EndNote when you can use Zotero? Zotero is extremely well integrated into Firefox (or Chrome if that's your thing), syncs with a remote server (keeping your references available between your academic workplace, laptop and home), integrates well with LibreOffice or Word. It's also opensource. EndNote has got unversities convinced we all need there software, but, frankly, I'm not so sure.

    ReplyDelete
    Replies
    1. People are stubborn -- latex + bibtex is a better solution in the long run in technical and practical terms, but native versions of familiar programmes would probably ease adoption.

      I gave up on Endnote when moving from endnote 7 to 8 (or was it 6 -> 7?) screwed up a manuscript. Never trusted a GUI reference manager since.

      As for universities and software -- universities move extremely slow and are highly susceptible to the marketing by software companies. A more acceptable reason may also be the lack of training material and staff for FOSS -- mathworks and wolfram seem to be sending people to my uni on a regular basis to give short courses in matlab and mathematica. Good for their business, I suppose.

      I don't like it, and luckily, by using FOSS, I don't have to be part of it.

      Delete
  5. I use I, Librarian (http://bioinformatics.org/librarian). It is FOSS, cross-platform, and it annotates PDFs.

    ReplyDelete
    Replies
    1. Thanks for the suggestion. I'll check it out and add it to the list.

      Delete
  6. I agree with the post, it is really annoying that Linux PDF applications are not as good as the Adobe Acrobat one. There are simply no solutions, and given that this problem has been around for so long, I do not expect solutions soon.

    I think Okular is the best PDF reader (even though I do not use KDE otherwise, it has some great applications). It is "almost" perfect, but is nothing compared to the nice Acrobat annotations system (for which I now need to use windows).

    As an academic, I often need to copy lines from tables in PDF files (copy a line from a table from the PDF and paste into a text editor for later data analysis). Interestingly, only Adobe Acrobat lets you copy properly properly. This is also true for texts with, say, two columns and you want to copy a paragraph from the right column. Evince typically does not "know" that the text flows in columns and will copy from left to right (try it to see what I mean).

    ReplyDelete
    Replies
    1. Does acrobat reader do copying ok, or only the full version?

      In my field I luckily only have to deal with the annotation issue in two situations:
      * when making corrections to a galley proof (doesn't happen more than 5-10 times per year so it's alright)
      * when making corrections to a student report/thesis which is written in latex -- and given the age of tex/latex and that they are so closely associated with linux, it's a little bit surprising that there aren't any good .ps/.dvi editors -- which could later have been extended to include pdf.

      Anyway, things are improving, but surprisingly slowly given how simple this would seem to a lay person, and given the importance of pdf annotation to one of the bigger linux user groups: academics.

      Delete
  7. I can use Adobe Reader X 10.0.0 (35.09 MB) in wine-1.7.4 (32 bit executed in linux 64 bit)
    That version is not available from adobe.com, but I can get it
    from http://www.oldapps.com

    You will need that precise verison. 10.0.1 does not work in my wine. Either the adobe guys are just plain dumb or they are wicked evil and they don't want people to use Linux. I think it should also work in previous versions fo wine.

    HTH

    ReplyDelete
    Replies
    1. I wouldn't think that they are /actively/ evil -- they are probably just lazy and do not consider the linux market as being big enough. So I'll vote for 'dumb' rather than 'evil'.

      Anyway, it puzzles me why programs that support annotation on e.g. windows don't support it on linux -- adobe reader isn't the only example either, as e.g. foxit reader has the same issue.

      Might be time to have another look at the solutions for pdf annotation -- I very quickly fell out with Mendeley, which I used to annotate the thesis of one of my students.

      Delete
    2. Unfortunately it crashed with wine-1.4-0ubuntu4 on ubuntu 12.04 with the first pdf document.

      Delete
    3. "It' being reader X 10.0.0? The anon poster suggests using wine 1.7.4 which is significantly more capable than 1.4 -- the current stable release is 1.6. You might have more luck with the current testing release.

      Delete
  8. I use Master PDF Editor (http://code-industry.net/pdfeditor.php). It's proprietary, but the Linux version is free and fully-functional.

    FFIW, I'm using Linux Mint 13 64-bit with the MATE desktop.

    ReplyDelete
    Replies
    1. Well, hard to find (even harder to see that the linux version is free, if you don't know about it). But it's doing a nice job, Mint 16, Cinnamon, 64bit. Thanks :-)

      Delete
    2. I don't know why I missed your comments. Master PDF editor does look quite interesting, in particular since they provide a .deb. However, I can't get it to run on Debian Wheezy since it complain about missing glibc v 2.14:
      "master-pdf-editor: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.14' not found (required by master-pdf-editor)"

      It might just be my system though.

      Delete
    3. OK, libc on debian stable is 2.13. Backports might sort that one out though.

      Delete
  9. You will need that precise verison. 10.0.1 does not work in my wine. Either the adobe guys are just plain dumb or they are wicked evil and they don't want people to use Linux. I think it should also work in previous versions fo wine ...

    ReplyDelete
  10. Zotero is extremely well integrated into Firefox (or Chrome if that's your thing), syncs with a remote server (keeping your references available between your academic workplace, laptop and home)..!!!

    ReplyDelete
  11. correct me if I'm wrong, but Zotero does not do annotations at all...

    ReplyDelete
    Replies
    1. That's right -- Zotero was only mentioned in connection with Endnote.

      Seems like the best hope at the moment is that Evince keeps improving when it comes to annotations...
      MasterPDF editor is OK in terms of functionality, but everything else about it reminds me of what I don't like about the philosophy of windows software (e.g. there being a Pro version)

      Delete
  12. From Ja-Kid:

    The best I have seen is "Master PDF Editor": https://code-industry.net/free-pdf-editor/ or http://linuxconfig.org/master-pdf-editor-linux-installation

    ReplyDelete
  13. I found a way !
    Download Foxit Reader at https://www.foxitsoftware.com/products/pdf-reader/ -> Free Download
    It allows annotations, highlighting, everything !
    It saves the annotations without having to do Save as
    And it hasn't bothered me with paying a license or anything.

    One problem easily fixable is that it doesn't appear in the list of Open With... softwares, for pdfs, so you have to add %F in the Exec field of the .desktop file in .local/share/applications (procedure described here http://askubuntu.com/questions/162612/how-can-i-add-an-application-to-the-list-of-open-with-applications)
    Now, my workflow with Zotero is a lot better, I just have to right click on a pdf and do "open with external viewer" or something. An it opens in Foxit (if you defined it as the Default program).

    ReplyDelete
    Replies
    1. I forgot to mention that you can install Foxit without Wine, because thhe developer released a Linux version
      Instructions on how to install it can be found at http://askubuntu.com/questions/602861/install-foxitreader-without-using-wine/679213#679213

      Delete