diff options
Diffstat (limited to 'ChangeLog')
| -rw-r--r-- | ChangeLog | 544 |
1 files changed, 531 insertions, 13 deletions
@@ -1,3 +1,521 @@ +2019-02-01 Jay Berkenbilt <ejb@ql.org> + + * 8.4.0: release + +2019-01-31 Jay Berkenbilt <ejb@ql.org> + + * Bug fix: do better pre-checks on images before optimizing; + refuse to optimize images that can't be converted to JPEG because + of colorspace or depth. + + * Add new options --externalize-inline-images, which converts + inline images larger than a specified size to regular images, and + --ii-min-bytes, which tweaks that size. + + * When optimizing images, inline images are now included in the + optimization, first being converted to regular images. Use + --keep-inline-images to exclude them from optimization. Fixes #278. + + * Add method QPDFPageObjectHelper::externalizeInlineImages, which + converts inline images whose size is at least a specified amount + to regular images. + + * Remove traces of acroread, which hasn't been available in Linux + for a long time. + +2019-01-30 Jay Berkenbilt <ejb@ql.org> + + * Do not include space after ID operator in inline image data. The + token now correctly contains the image data, the EI operator, + and the delimiter that precedes the EI operator. + + * Improve locating of an inline image's EI operator to correctly + handle the case of EI appearing inside the image data. + + * Very low-level QPDFTokenizer API now includes an + expectInlineImage method that takes an input stream, enabling it + to locate an inline image's EI operator better. When this method + is called, the inline image token returned will not contain the EI + operator and will contain correct image data. This is called + automatically everywhere within the qpdf library. Most user code + will never have to use the low-level tokenizer API. If you use + Pl_QPDFTokenizer, this will be done automatically for you. If you + use the low-level API and call expectInlineImage, you should call + the new version. + +2019-01-29 Jay Berkenbilt <ejb@ql.org> + + * Bug fix: when returning an inline image token, the tokenizer no + longer includes the delimiter that follows EI. The + QPDFObjectHandle created from the token was correct. + + * Handle files with direct page objects, which is not allowed by + the PDF spec but has been seen in the wild. Fixes #164. + +2019-01-28 Jay Berkenbilt <ejb@ql.org> + + * Bug fix: when using --stream-data=compress, object streams and + xref streams were not compressed. They were compressed if no + --stream-data option was specified. Fixes #271. + + * When linearizing or getting the list of all pages in a file, + replace duplicated page objects with a shallow copy of the page + object. Linearization and all page manipulation APIs require page + objects to be unique. Pages that were originally duplicated will + still share contents and any other indirect resources. Fixes #268. + +2019-01-26 Jay Berkenbilt <ejb@ql.org> + + * Add --overlay and --underlay options. Fixes #207. + + * Create examples/pdf-overlay-page.cc to demonstrate use of + page/form XObject interaction + + * Add new methods QPDFPageObjectHelper::getFormXObjectForPage, + which creates a form XObject equivalent to a page, and + QPDFObjectHandle::placeFormXObject, which generates content stream + code to placing a form XObject on a page. + +2019-01-25 Jay Berkenbilt <ejb@ql.org> + + * Add new method QPDFObjectHandle::getUniqueResourceName() to + return an unused key available to be used in a resource + dictionary. + + * Add new method QPDFPageObjectHelper::getAttribute() that + properly handles inherited attributes and allows for creation of a + copy of shared attributes. This is very useful if you are getting + an attribute of a page dictionary with the intent to modify it + privately for that page. + + * Fix QPDFPageObjectHelper::getPageImages (and the legacy + QPDFObjectHandle::getPageImages()) to properly handle images in + inherited resources dictionaries. + +2019-01-20 Jay Berkenbilt <ejb@ql.org> + + * Tweak the content code generated for variable text fields to + better handle font sizes and multi-line text. + + * When generating appearance streams for variable text + annotations, properly handle the cases of there being no + appearance dictionary, no appearance stream, or an appearance + stream with no BMC..EMC marker. + + * When flattening annotations, remove annotations from the file + that don't have appearance streams. These were previously being + preserved, but since they are invisible, there is no reason to + preserve them when flattening annotations. + +2019-01-19 Jay Berkenbilt <ejb@ql.org> + + * NOTE: qpdf CLI: some non-compatible changes were made to how + qpdf interprets password arguments that contain Unicode characters + that fall outside of ASCII. On Windows, the non-compatibility was + unavoidable, as explained in the release notes. On all platforms, + it is possible to get the old behavior if desired, though the old + behavior would almost always result in files that other + applications were unable to open. As it stands, qpdf should now be + able to open passwords encrypted with a wide range of passwords + that some other viewers might not handle, though even now, qpdf's + Unicode password handling is not 100% complete. + + * Add --password-mode option, which allows fine-grained control of + how password arguments are treated. This is discussed fully in the + manual. Fixes #215. + + * Add option --suppress-password-recovery to disable the behavior + of searching for a correct password by re-encoding the provided + password. This option can be useful if you want to ensure you know + exactly what password is being used. + +2019-01-17 Jay Berkenbilt <ejb@ql.org> + + * When attempting to open an encrypted file with a password, if + the password doesn't work, try alternative passwords created by + re-interpreting the supplied password with different string + encodings. This makes qpdf able to recover passwords with + non-ASCII characters when either the decryption or encryption + operation was performed with an incorrectly encoded password. + + * Fix data loss bug: qpdf was discarding referenced resources in + the case in which a page's resource dictionary contained an + indirect reference for either /Font or /XObject that contained + fonts or XObjects not referenced on all pages that shared the + resource. This was a "typo" in the code. The comment explained the + correct behavior, and the code was clearly intended to handle this + issue, but the implementation had an error in it. This is fixed by + a single-line change, which can be found in git commit + 4bc434000c42a7191e705c8a38216ca6743ad9ff. That commit can be used + as a patch that applies cleanly against qpdf 8.1.0 and forward. + The bug was introduced in version 8.1.0. For the record, this is + the first bug in qpdf's history that could result in silent loss + of data when processing a correct input file. Fixes #276. + +2019-01-15 Jay Berkenbilt <ejb@ql.org> + + * Add QUtil::possible_repaired_encodings which, given a string, + generates other strings that represent re-interpretation of the + bytes in a different coding system. This is used to help recover + passwords if the password string was improperly encoded on a + different system due to user error or a software bug. + +2019-01-14 Jay Berkenbilt <ejb@ql.org> + + * Add new CLI flags to 128-bit and 256-bit encryption: --assemble, + --annotate, --form, and --modify-other to control encryption + permissions with more granularity than was allowed with the + --modify flag. Fixes #214. + + * Add new versions of + QPDFWriter::setR{3,4,5,6}EncryptionParameters that allow + individual setting of the various permission bits. The old + interfaces are retained for backward compatibility. In the "C" + API, add qpdf_set_r{3,4,5,6}_encryption_parameters2. The new + interfaces use separate booleans for various permissions instead + of the qpdf_r3_modify_e enumerated type, which set permission bits + in predefined groups. + + * Add versions of utf8 to single-byte character transcoders that + return a success code. + +2019-01-13 Jay Berkenbilt <ejb@ql.org> + + * Add several more string transcoding and analysis methods to + QUtil for bidirectional conversion between PDF Doc, Win Ansi, Mac + Roman, UTF-6, and UTF-16 along with detection of valid UTF-8 and + UTF-16. + +2019-01-12 Jay Berkenbilt <ejb@ql.org> + + * In the --pages option, allow the same page to be specified more + than once. You can now do "--pages A.pdf 1,1 --" or + "--pages A.pdf 1 A.pdf 1" instead of having to use two different + paths to specify A.pdf. Fixes #272. + + * Add QPDFPageObjectHelper::shallowCopyPage(). This method creates + a new page object that is a "shallow copy" of the given page as + described in the comments in QPDFPageObjectHelper. The resulting + object has not been added anywhere but is ready to be passed to + QPDFPageDocumentHelper::addPage of its own QPDF or another QPDF + object. + + * Add QPDF::getUniqueId() method to return an identifier that is + intended to be unique within the scope of all QPDF objects created + by the calling application in a single run. + + * In --pages, allow "." as a replacement for the current input + file, making it possible to say "qpdf A.pdf --pages . 1-3 --" + instead of having to repeat the input filename. + +2019-01-10 Jay Berkenbilt <ejb@ql.org> + + * Add new configure option --enable-avoid-windows-handle, which + causes the symbol AVOID_WINDOWS_HANDLE to be defined. If set, we + avoid using Windows I/O HANDLE, which is disallowed in some + versions of the Windows SDK, such as for Windows phones. + QUtil::same_file will always return false in this case. Only + applies to Windows builds. + + * Add new method QPDF::setImmediateCopyFrom. When called on a + source QPDF object, streams can be copied FROM that object to + other ones without having to keep the source QPDF or its input + source around. The cost is copying the streams into RAM. See + comments in QPDF.hh for setImmediateCopyFrom for a detailed + explanation. + +2019-01-07 Jay Berkenbilt <ejb@ql.org> + + * 8.3.0: release + + * Add sample completion files in completions. These can be used by + packagers to install on the system wherever bash and zsh keep + their vendor-supplied completions. + + * Add configure flag --enable-check-autofiles, which is on by + default. Packagers whose packaging systems automatically refresh + autoconf or libtool files should pass --disable-check-autofiles to + ./configure to suppress warnings about automatically generated + files being outdated. + +2019-01-06 Jay Berkenbilt <ejb@ql.org> + + * Remove the restriction in most cases that the source QPDF used + in a copyForeignObject call has to stick around until the + destination QPDF is written. The exceptional case is when the + source stream gets is data using a + QPDFObjectHandle::StreamDataProvider. For a more in-depth + discussion, see comments around copyForeignObject in QPDF.hh. + Fixes #219. + +2019-01-05 Jay Berkenbilt <ejb@ql.org> + + * When generating appearances, if the font uses one of the + standard, built-in encodings, restrict the character set to that + rather than just to ASCII. This will allow most appearances to + contain characters from the ISO-Latin-1 range plus a few + additional characters. + + * Add methods QUtil::utf8_to_win_ansi and + QUtil::utf8_to_mac_roman. + + * Add method QUtil::utf8_to_utf16. + +2019-01-04 Jay Berkenbilt <ejb@ql.org> + + * Add new option --optimize-images, which recompresses every image + using DCT (JPEG) compression as long as the image is not already + compressed with lossy compression and recompressing the image + reduces its size. The additional options --oi-min-width, + --oi-min-height, and --oi-min-area prevent recompression of images + whose width, height, or pixel area (width * height) are below a + specified threshold. + + * Add new option --collate. When specified, the semantics of + --pages change from concatenation to collation. See the manual for + a more detailed discussion. Fixes #259. + + * Add new method QPDFWriter::getFinalVersion, which returns the + PDF version that will ultimately be written to the final file. See + comments in QPDFWriter.hh for some restrictions on its use. Fixes + #266. + + * When unexpected errors are found while checking linearization + data, print an error message instead of calling assert, which + cause the program to crash. Fixes #209, #231. + + * Detect and recover from dangling references. If a PDF file + contained an indirect reference to a non-existent object (which is + valid), when adding a new object to the file, it was possible for + the new object to take the object ID of the dangling reference, + thereby causing the dangling reference to point to the new object. + This case is now prevented. Fixes #240. + +2019-01-03 Jay Berkenbilt <ejb@ql.org> + + * Add --generate-appearances flag to the qpdf command-line tool to + trigger generation of appearance streams. + + * Fix behavior of form field value setting to handle the following + cases: + - Strings are always written as UTF-16 + - Check boxes and radio buttons are handled properly with + synchronization of values and appearance states + + * Define constants in qpdf/Constants.h for interpretation of + annotation and form field flags + + * Add QPDFAnnotationObjectHelper::getFlags + + * Add many new methods to QPDFFormFieldObjectHelper for querying + flags and field types + + * Add new methods for appearance stream generation. See comments + in QPDFFormFieldObjectHelper.hh for generateAppearance() for a + description of limitations. + - QPDFAcroFormDocumentHelper::generateAppearancesIfNeeded + - QPDFFormFieldObjectHelper::generateAppearance + + * Bug fix: when writing form field values, always write string + values encoded as UTF-16. + + * Add method QUtil::utf8_to_ascii, which returns an ASCII string + for a UTF-8 string, replacing out-of-range characters with a + specified substitute. + +2019-01-02 Jay Berkenbilt <ejb@ql.org> + + * Add method QPDFObjectHandle::getResourceNames that returns a set + of strings representing all second-level keys in a dictionary + (i.e. all keys of all direct dictionary members). + +2018-12-31 Jay Berkenbilt <ejb@ql.org> + + * Add --flatten-annotations flag to the qpdf command-line tool for + annotation flattening. + + * Add methods for flattening form fields and annotations: + - QPDFPageDocumentHelper::flattenAnnotations - integrate + annotation appearance streams into page contents with special + handling for form fields: if appearance streams are up to date + (/NeedAppearances is false in /AcroForm), the /AcroForm key of + the document catalog is removed. Otherwise, a warning is + issued, and form fields are ignored. Non-form-field + annotations are always flattened if an appearance stream can + be found. + - QPDFAnnotationObjectHelper::getPageContentForAppearance - + generate the content stream fragment to render an appearance + stream in a page's content stream as a form xobject. Called by + flattenAnnotations. + + * Add method QPDFObjectHandle::mergeResources(), which merges + resource dictionaries. See detailed description in + QPDFObjectHandle.hh. + + * Add QPDFObjectHandle::Matrix, similar to + QPDFObjectHandle::Rectangle, as a convenience class for + six-element arrays that are used as matrices. + +2018-12-23 Jay Berkenbilt <ejb@ql.org> + + * When specifying @arg on the command line, if the file "arg" does + not exist, just treat this is a normal argument. This makes it + easier to deal with files whose names start with the @ character. + Fixes #265. + + * Tweak completion so it works with zsh as well using + bashcompinit. + +2018-12-22 Jay Berkenbilt <ejb@ql.org> + + * Add new options --json, --json-key, and --json-object to + generate a json representation of the PDF file. This is described + in more depth in the manual. You can also run qpdf --json-help to + get a description of the json format. + +2018-12-21 Jay Berkenbilt <ejb@ql.org> + + * Allow --show-object=trailer for showing the document trailer. + + * You can now use eval $(qpdf --completion-bash) to enable bash + completion for qpdf. It's not perfect, but it works pretty well. + +2018-12-19 Jay Berkenbilt <ejb@ql.org> + + * When splitting pages using --split-pages, the outlines + dictionary and some supporting metadata are copied into the split + files. The result is that all bookmarks from the original file + appear, and those that point to pages that are preserved work + while those that point to pages that are not preserved don't do + anything. This is an interim step toward proper support for + bookmark preservation in split files. + + * Add QPDFOutlineDocumentHelper and QPDFOutlineObjectHelper for + handling outlines (bookmarks) including bidirectionally mapping + between bookmarks and pages. Initially there is no support for + modifying the outlines hierarchy. + +2018-12-18 Jay Berkenbilt <ejb@ql.org> + + * New method QPDFObjectHandle::getJSON() returns a JSON object + with a partial representation of the object. See + QPDFObjectHandle.hh for a detailed description. + + * Add a simple JSON serializer. This is not a complete or + general-purpose JSON library. It allows assembly and serialization + of JSON structures with some restrictions, which are described in + the header file. + + * Add QPDFNameTreeObjectHelper class. This class provides useful + methods for dealing with name trees, which are discussed in + section 7.9.6 of the PDF spec (ISO-32000). + + * Preserve page labels when merging and splitting files. Prior + versions of qpdf simply preserved the page label information from + the first file, which usually wouldn't make any sense in the + merged file. Now any page that had a page number in any original + file will have the same page number after merging or splitting. + + * Add QPDFPageLabelDocumentHelper class. This is a document helper + class that provides useful methods for dealing with page labels. + It abstracts the fact that they are stored as number trees and + deals with interpolating intermediate values that are not in the + tree. It also has helper functions used by the qpdf command line + tool to preserve page labels when merging and splitting files. + + * Add QPDFNumberTreeObjectHelper class. This class provides useful + methods for dealing with number trees, which are discussed in + section 7.9.7 of the PDF spec (ISO-32000). Page label dictionaries + are represented as number trees. + + * New method QPDFObjectHandle::wrapInArray returns the object + itself if it is an array. Otherwise, it returns an array + containing the object. This is useful for dealing with PDF data + that is sometimes expressed as a single element and sometimes + expressed as an array, which is a somewhat common PDF idiom. + +2018-10-11 Jay Berkenbilt <ejb@ql.org> + + * Files generated by autogen.sh are now committed so that it is + possible to build on platforms without autoconf directly from a + clean checkout of the repository. The configure script detects if + the files are out of date when it also determines that the tools + are present to regenerate them. + + * Add build in Azure Pipelines, now that it is free for open + source projects. + +2018-08-18 Jay Berkenbilt <ejb@ql.org> + + * 8.2.1: release + + * Add new option --keep-files-open=[yn] to control whether qpdf + keeps files open when merging. Prior to version 8.1.0, qpdf always + kept all files open, but this meant that the number of files that + could be merged was limited by the operating system's open file + limit. Version 8.1.0 opened files as they were referenced, but + this caused a major performance impact. Version 8.2.0 optimized + the performance but did so in a way that, for local file systems, + there was a small but unavoidable performance hit, but for + networked file systems, the performance impact could be very high. + Starting with version 8.2.1, the default behavior is that files + are kept open if no more than 200 files are specified, but that + the behavior can be explicitly overridden with the + --keep-files-open flag. If you are merging more than 200 files but + less than the operating system's max open files limit, you may + want to use --keep-files-open=y. If you are using a local file + system where the overhead is low and you might sometimes merge + more than the OS limit's number of files, you may want to specify + --keep-files-open=n. Fixes #237. + +2018-08-16 Jay Berkenbilt <ejb@ql.org> + + * 8.2.0: release + +2018-08-14 Jay Berkenbilt <ejb@ql.org> + + * For the mingw builds, change the name of the DLL import library + from libqpdf.a to libqpdf.dll.a to avoid confusing it with a + static library. This potentially clears the way for supporting a + static library in the future, though presently, the qpdf Windows + build only builds the DLL and executables. Fixes #225. + +2018-08-13 Jay Berkenbilt <ejb@ql.org> + + * Add new class QPDFSystemError, derived from std::runtime_error, + which is now thrown by QUtil::throw_system_error. This enables the + triggering errno value to be retrieved. Fixes #221. + +2018-08-12 Jay Berkenbilt <ejb@ql.org> + + * qpdf command line: add --no-warn option to suppress issuing + warning messages. If there are any conditions that would have + caused warnings to be issued, the exit status is still 3. + + * Rewrite the internals of Pl_Buffer to be much more efficient in + use of memory at a very slight performance cost. The old + implementation could cause memory usage to go out of control for + files with large images compressed using the TIFF predictor. + Fixes #228. + +2018-08-05 Jay Berkenbilt <ejb@ql.org> + + * Bug fix: end of line characters were not properly handled inside + strings in some cases. Fixes #226. + + * Bug fix: infinite loop on progress reporting for very small + files. Fixes #230. + +2018-08-04 Jay Berkenbilt <ejb@ql.org> + + * Performance fix: optimize page merging operation to avoid + unnecessary open/close calls on files being merged. Fixes #217. + + * Add ClosedFileInputSource::stayOpen method, enabling a + ClosedFileInputSource to stay open during manually indicated + periods of high activity, thus reducing the overhead of frequent + open/close operations. + 2018-06-23 Jay Berkenbilt <ejb@ql.org> * 8.1.0: release @@ -133,7 +651,7 @@ * Ignore zlib data check errors while uncompressing streams. This is consistent with behaviors of other readers and enables handling - of some incorrectly written zlib strems. Fixes #191. + of some incorrectly written zlib streams. Fixes #191. 2018-02-25 Jay Berkenbilt <ejb@ql.org> @@ -151,7 +669,7 @@ * Major enhancements to handling of type errors within the qpdf library. This fix is intended to eliminate those annoying cases where qpdf would exit with a message like "operation for - dictionary object attemped on object of wrong type" without + dictionary object attempted on object of wrong type" without providing any context. Now qpdf keeps enough context to be able to issue a proper warning and to handle such conditions in a sensible way. This should greatly increase the number of bad files that @@ -267,7 +785,7 @@ counting in length for better readability. Ordinarily this makes no difference, but in the rare case of a page's contents being split in the middle of a token, the old behavior could cause the - extra newline to be interprted as part of the token. This bug + extra newline to be interpreted as part of the token. This bug could only be triggered in qdf mode, which is a mode intended for manual inspection of PDF files' contents, so it is very unlikely to have caused any actual problems for people using qpdf for @@ -346,7 +864,7 @@ QPDFObjectHandle::pipeStreamData, you don't need to worry about this at all. - * Provide heavily annoated examples/pdf-filter-tokens.cc example + * Provide heavily annotated examples/pdf-filter-tokens.cc example that illustrates use of some simple token filters. * When normalizing content streams, as in qdf mode, issue warning @@ -356,7 +874,7 @@ either there's a bug in qpdf's lexer, that the file is damaged, or that the page's contents are split in a weird way. In any of those cases, qpdf could potentially damage the stream's contents by - replacing carrige returns with newlines or otherwise messing with + replacing carriage returns with newlines or otherwise messing with spaces. The mostly likely case of this would be an inline image's compressed data being divided across two streams and having the compressed data in the second stream contain a carriage return as @@ -397,7 +915,7 @@ * 7.1.0: release - * Allow raw encryption key to be specified in libary and command + * Allow raw encryption key to be specified in library and command line with the QPDF::setPasswordIsHexKey method and --password-is-hex-key option. Allow encryption key to be displayed with --show-encryption-key option. Thanks to Didier Stevens @@ -775,7 +1293,7 @@ 2013-12-14 Jay Berkenbilt <ejb@ql.org> - * Allow anyspace rather than just newline to follow xref header. + * Allow any space rather than just newline to follow xref header. This allows qpdf to read a wider range of damaged files. 2013-11-30 Jay Berkenbilt <ejb@ql.org> @@ -934,7 +1452,7 @@ * Add QPDF::getCompressibleObjGens() and deprecate QPDF::getCompressibleObjects(), which had a flaw in its logic. - * Add new QPDFObjectHandle::getObjGen() method and indiciate in + * Add new QPDFObjectHandle::getObjGen() method and indicate in comments that its use is favored over getObjectID() and getGeneration() for most cases. @@ -981,7 +1499,7 @@ * Remove all calls to sprintf * New method QUtil::int_to_string_base to convert to octal or - hexademical (or decimal) strings without using sprintf + hexadecimal (or decimal) strings without using sprintf 2013-02-26 Jay Berkenbilt <ejb@ql.org> @@ -1110,7 +1628,7 @@ QPDF::compute_data_key to take the R and V values from the encryption dictionary. There is no reason for any application code to call this method since handling of encryption is done - automatically by the qpdf libary. It is used internally by + automatically by the qpdf library. It is used internally by QPDFWriter. * Support reading and decryption of files whose main text is not @@ -1425,7 +1943,7 @@ * Add new array mutation routines to QPDFObjectHandle. Implemented by Tobias Hoffmann. - * Rework APIs that use size_t, off_t, and primative integer types + * Rework APIs that use size_t, off_t, and primitive integer types so that size_t is used for sizes of memory and off_t is used for file offsets. Also set _FILE_OFFSET_BITS so that large files can be supported on 32-bit UNIX/Linux platforms. The code assumes in @@ -1528,7 +2046,7 @@ uses the invalid object number 0 as a regular object. * libqpdf/QPDF_linearization.cc (isLinearized): use -1 rather than - 0 as a sentintel for not having found the first object in the + 0 as a sentinel for not having found the first object in the file. Since -1 can never match the regular expression, this prevents an infinite loop when checking a file that starts with (erroneous) 0 0 obj. (Fixes qpdf-Bugs-3159950.) @@ -1751,7 +2269,7 @@ with the required padding as specified by the PDF specification. This is seldom useful to users. This function has been replaced by QPDF::getPaddedUserPassword. Call the new - QPDF::getTrimmedUserPassword to retreive the user password in a + QPDF::getTrimmedUserPassword to retrieve the user password in a human-readable format. * qpdf/qpdf.cc (main): qpdf --check now prints the PDF version |