diff options
| -rw-r--r-- | ChangeLog | 10 | ||||
| -rw-r--r-- | TODO | 12 | ||||
| -rw-r--r-- | cSpell.json | 1 | ||||
| -rw-r--r-- | job.sums | 2 | ||||
| -rw-r--r-- | manual/cli.rst | 5 | ||||
| -rw-r--r-- | manual/design.rst | 218 | ||||
| -rw-r--r-- | manual/qpdf-job.rst | 12 | ||||
| -rw-r--r-- | manual/release-notes.rst | 181 |
8 files changed, 424 insertions, 17 deletions
@@ -68,7 +68,7 @@ object. * A light C API around basic QPDFJob functionality is in - include/qpdf/qpdf-job-c.h.p + include/qpdf/qpdfjob-c.h.p * Add new functions version of QUtil::call_main_from_wmain that takes a constant argv array. @@ -79,10 +79,10 @@ description to --help and the manual. * The --json flag now takes a version number as an optional - parameter. The default will remain version 1 for compatibility. - This enables future code to use --json=latest to always get the - latest version or to use a specific version. At this time, there's - only version 1, but a version 2 may appear in a future qpdf. + parameter. The default will remain version 1 for compatibility + until the release of qpdf 11, after which it will become "latest". + At this time, there's only version 1, but a version 2 may appear + in a future qpdf. 2022-01-28 Jay Berkenbilt <ejb@ql.org> @@ -9,15 +9,6 @@ make qpdf more contributor-friendly. Look https://bestpractices.coreinfrastructure.org/en -* Remember for release notes: starting in qpdf 11, the default value - for the --json keyword will be "latest". If you are depending on - version 1, change your code to specify --json=1, which works - starting with 10.6.0. - -* Write up something about preparing for the PointerHolder to - shared_ptr migration. Clearly document the deprecations and how to - deal with them. - Output JSON v2 ============== @@ -332,6 +323,9 @@ Other notes: PointerHolder to std::shared_ptr ================================ +Remember to update the smart-pointers section of the manual in +design.rst. + Once all deprecation warnings are cleared up (changing getPointer() to get() and getRefcount() to use_count()), the only real issues are that implicit assignment of a pointer to a shared_ptr doesn't work while it diff --git a/cSpell.json b/cSpell.json index 1d78754c..322a8581 100644 --- a/cSpell.json +++ b/cSpell.json @@ -74,6 +74,7 @@ "cxxflags", "cygwin", "dctdecode", + "decltype", "decrypter", "deduplicating", "deps", @@ -14,4 +14,4 @@ libqpdf/qpdf/auto_job_json_decl.hh c5e3fd38a3b0c569eb0c6b4c60953a09cd6bc7d3361a3 libqpdf/qpdf/auto_job_json_init.hh b070350d304d137ba594c1ba40b373137e8459735f04b8ca0f8a2ffd1908c69e libqpdf/qpdf/auto_job_schema.hh 18a3780671d95224cb9a27dcac627c421cae509d59f33a63e6bda0ab53cce923 manual/_ext/qpdf.py e9ac9d6c70642a3d29281ee5ad92ae2422dee8be9306fb8a0bc9dba0ed5e28f3 -manual/cli.rst 3746df6c4f115387cca0d921f25619a6b8407fc10b0e4c9dcf40b0b1656c6f8a +manual/cli.rst 2dd5e5a9c0440aea65ed0a2bf6239aa6662afdb463224aafdc116a8a676dbc20 diff --git a/manual/cli.rst b/manual/cli.rst index 614be80d..147cd71e 100644 --- a/manual/cli.rst +++ b/manual/cli.rst @@ -3154,7 +3154,10 @@ Related Options supported value is ``1``, but it's possible that a new JSON output version will be added in a future version. You can also specify ``latest`` to use the latest JSON version. For backward - compatibility, the default value is ``1``. Use the + compatibility, the default value will remain ``1`` until qpdf + version 11, after which point it will become ``latest``. In all + case, you can tell what version of the JSON output you have from + the ``"version"`` key in the output. Use the :qpdf:ref:`--json-help` option to get a description of the JSON object. diff --git a/manual/design.rst b/manual/design.rst index 2c1e66d9..91200707 100644 --- a/manual/design.rst +++ b/manual/design.rst @@ -745,3 +745,221 @@ C API object handle methods returned error codes like the other methods and set return values in passed-in pointers, but this would complicate both the implementation and the use of the library for a case that is actually quite rare and largely avoidable. + +.. _smart-pointers: + +Smart Pointers +-------------- + +This section describes changes to the use of smart pointers in qpdf in +versions 10.6.0 and 11.0.0. + +Starting in qpdf 11, ``PointerHolder`` will be replaced with +``std::shared_ptr`` in qpdf's public API. A backward-compatible +``PointerHolder`` will be provided that should make it possible for +most code to remain unchanged. This new ``PointerHolder`` will be +marked deprecated but will provide a way to suppress the deprecation +warnings. Code that works with containers of ``PointerHolder`` may +have to be modified, though no qpdf interfaces do this. + +The remainder of this section describes how to prepare if you want to +eliminate ``PointerHolder`` from your code or what to do if you want +to stick with the old interfaces. + +Changes in 10.6.0 +~~~~~~~~~~~~~~~~~ + +In qpdf 10.6.0, two ``PointerHolder`` methods have been deprecated and +replaced with methods that are compatible with ``std::shared_ptr``: + +- ``getPointer()`` -- use ``get()`` instead + +- ``getRefcount()`` -- use ``use_count()`` instead + +If you build your code with deprecation warnings enabled and you want +to suppress these deprecation warnings for now, you can ``#define +NO_POINTERHOLDER_DEPRECATION`` before including any qpdf header files. +It may be possible to leave it this way long-term to facilitate +supporting older versions of qpdf without conditional compilation. + +``PointerHolder`` has had a long-standing bug: a ``const +PointerHolder<T>`` would only provide a ``T const*`` with its +``getPointer`` method. This is incorrect and is now how standard C++ +smart pointers or regular pointers behave. The correct semantics +would be that a ``const PointerHolder<T>`` would not accept a new +pointer after being created but would still allow you to modify the +item being pointed to. If you don't want to mutate the thing it points +to, use ``PointerHolder<T const>`` instead. The new ``get()`` method +behaves correctly. It is therefore not exactly the same as +``getPointer()``, but it does behave the way ``get()`` behaves with +``std::shared_ptr``. This shouldn't make any difference to any +correctly written code. + + +How to Prepare +~~~~~~~~~~~~~~ + +If you don't need to support versions of qpdf prior to 10.6, you can +just replace all occurrences of ``getPointer()`` with ``get()`` and +all occurrences of ``getRefcount()`` with ``use_count()``. That's +about all you will be able to do prior to qpdf 11. + +If you need to support older versions, you have two choices: + +- ``#define NO_POINTERHOLDER_DEPRECATION`` and leave everything the + way it was. You can just wait until qpdf 11. + +- Write code that uses ``get()`` but falls back to ``getPointer()`` if + ``QPDF_MAJOR_VERSION`` is not defined. The symbols + ``QPDF_MAJOR_VERSION``, ``QPDF_MINOR_VERSION``, and + ``QPDF_PATCH_VERSION`` were introduced with 10.6.0, so just checking + for whether ``QPDF_MAJOR_VERSION`` is defined is sufficient for + telling if you're running a version before 10.6.0. If you do this, + once qpdf 11 comes out, you will already know all the places that + have to be handled specially. + +If you are somehow relying on the fact that a ``const +PointerHolder<T>`` always gave back a ``T const*`` and are +dereferencing a ``const PointerHolder<T>`` to call methods that only +have ``const`` versions in ``T``, you may have to change from +``const PointerHolder<T>`` to ``PointerHolder<T const>``. This won't +be an issue for anything in the qpdf API, and if you are using qpdf +``PointerHolder`` objects for any other reason, you should just +replace them with ``std::shared_ptr``. + +What to Expect +~~~~~~~~~~~~~~ + +Note: if you are reading this in the 10.6 manual and 11 is out, you +should read it in the manual for qpdf 11 instead. Some early tests +have been done to try to ensure the accuracy of this information, but +it may change once the work is actually completed. + +When ``PointerHolder`` disappears from qpdf's API in qpdf 11, you will +have a few options: + +- Use the new ``PointerHolder``, which is derived from + ``std::shared_ptr`` and which has methods to make it + interchangeable. For things that use ``PointerHolder<T>`` directly, + this should "just work," though you will have to ``#define + NO_POINTERHOLDER_DEPRECATION`` if you don't want deprecation + warnings. + +- Replace all uses of ``PointerHolder<T>`` with ``std::shared_ptr<T>`` + and deal with the required changes, outlined below. This is the + recommended course of action. You will need conditional compilation + if you want to simultaneously support order code. Stay tuned for the + qpdf 11 documentation for specifics. + +While ``PointerHolder<T>`` and ``std::shared_ptr<T>`` will be mutually +assignable and convertible, this does not apply to containers of those +objects. The qpdf API doesn't have any containers of +``PointerHolder``, so this would have to be in your own code. You can +prepare yourself for the change by using ``auto`` and ``decltype`` +whenever possible so that a change to the underlying type of something +won't require source changes. + +Required Changes in qpdf 11 +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This section describes unavoidable changes when replacing +``PointerHolder`` with ``std::shared_ptr`` rather than continuing to +use the backward compatible API. Nothing here is needed (or can be +done) prior to qpdf 11, so consider this to be a preview. + +- Change ``getPointer`` to ``get`` and ``getRefcount`` to + ``use_count`` as above. If your starting point is no deprecation + warnings with qpdf 10.6, this will already be true. + +- Array allocations will have to be rewritten. + + To allocate a ``PointerHolder`` to an array: + + .. code-block:: c++ + + PointerHolder<X> p(true, new X[n]); + + To allocate a ``std::shared_ptr`` to an array: + + .. code-block:: c++ + + auto p = std::shared_ptr<X>(new X[n], std::default_delete<X[]>()); + + To allocate a ``std::unique_ptr`` to an array: + + .. code-block:: c++ + + auto p = std::make_unique<X[]>(n); + // or + auto p = std::unique_ptr<X[]>(new X[n]); + + The second form may be needed if ``X`` has a private constructor + from this context. + + C++-17 has a better way to allocate ``std::shared_ptr`` to an array, + but qpdf is still allowing C++-14 to be used. You can use whatever + method to handle shared arrays that is supported in your + environment. There are no shared arrays in qpdf's public API except + for some ``QUtil`` helper methods that are not essential for use of + qpdf features. + +- ``PointerHolder<T>`` can have plain pointers directly assigned to + it, while ``std::shared_ptr<T>`` cannot. This makes code like this + possible: + + .. code-block:: c++ + + PointerHolder<X> x_p; + X* x = new X(); + x_p = x; + + It also makes it possible to pass a plain pointer to a function + expecting a ``PointerHolder``, thereby transferring "ownership" of + the pointer into the function. + + Code like that is a risky because you can leak memory if an + exception is thrown between creation of the X and assignment of it + into the ``PointerHolder``. In any case, ``std::shared_ptr`` does + not allow that, so you need one of these instead: + + .. code-block:: c++ + + auto x_p = std::make_shared<X>(); + X* x = x_p.get(); + // or, less safe, but closer: + std::shared_ptr<X> x_p; + X* x = new X(); + x_p = std::shared_ptr<X>(x); + + Also, code like this: + + .. code-block:: c++ + + PointerHolder<Base> base_p; + Derived* derived = new Derived(); + base_p = derived; + + needs to be replaced with something like this instead: + + .. code-block:: c++ + + std::shared_ptr<Base> base_p; + Derived* derived = new Derived(); + base_p = std::shared_ptr<Base>(derived); + +Historical Background +~~~~~~~~~~~~~~~~~~~~~ + +Since its inception, the qpdf library used its own smart pointer +class, ``PointerHolder``. The ``PointerHolder`` class was originally +created long before ``std::shared_ptr`` existed, and qpdf itself +didn't start requiring a C++-11 compiler version 9.1.0 released in +late 2019. + +``PointerHolder`` is a reference-counted smart pointer with semantics +almost identical to ``std::shared_ptr`` except that it is not +thread-safe. It has a few interface differences that prevent +``std::shared_ptr`` from being a drop-in replacement. However, given +the value of using standard library smart pointers, qpdf is taking the +plunge for version 11 and switching over to standard library smart +pointers. diff --git a/manual/qpdf-job.rst b/manual/qpdf-job.rst index 5fd8b37b..d464ef64 100644 --- a/manual/qpdf-job.rst +++ b/manual/qpdf-job.rst @@ -14,7 +14,10 @@ executable is available from inside the C++ library using the - Use from the C++ API with ``QPDFJob::initializeFromArgv`` - - Use from the C API with ``qpdfjob_run_from_argv`` from :file:`qpdfjob-c.h` + - Use from the C API with ``qpdfjob_run_from_argv`` from + :file:`qpdfjob-c.h`. If you are calling from a Windows-style main + and have an argv array of ``wchar_t``, you can use + ``qpdfjob_run_from_wide_argv``. - The job JSON file format @@ -135,6 +138,13 @@ C++ code: return 0; } +Note the ``QPDFUsage`` exception above. This is thrown whenever a +configuration error occurs. These exactly correspond to usage messages +issued by the :command:`qpdf` CLI for things like omitting an output +file, specifying `--pages` multiple times, or other invalid +combinations of options. ``QPDFUsage`` is thrown by the argv and JSON +interfaces as well as the native ``QPDFJob`` interface. + It is also possible to mix and match command-line options and JSON from the CLI. For example, you could create a file called :file:`my-options.json` containing the following: diff --git a/manual/release-notes.rst b/manual/release-notes.rst index 6b5b85f4..44974c1d 100644 --- a/manual/release-notes.rst +++ b/manual/release-notes.rst @@ -6,6 +6,187 @@ Release Notes For a detailed list of changes, please see the file :file:`ChangeLog` in the source distribution. +10.6.0: XXX + - Deprecations/future replacement of ``PointerHolder`` + + The next major release of qpdf will replace ``PointerHolder`` with + ``std::shared_ptr`` across all of qpdf's public API. In + preparation for this change, the following ``PointerHolder`` + methods have been deprecated in favor of interfaces that more + closely match ``std::shared_ptr``: + + - ``getPointer()`` -- use ``get()`` instead; this also fixes + ``const`` semantics as discussed in + :file:`include/qpdf/PointerHolder.hh`. + + - ``getRefcount()`` -- use ``use_count()`` instead + + If you build your code with deprecation warnings enabled and you + want to suppress these deprecation warnings for now, you can + ``#define NO_POINTERHOLDER_DEPRECATION`` before including any qpdf + header files. Code that does this will *require no changes* prior + to qpdf 11 and may or may not require changes after qpdf 11. + + For a detailed discussion of this change and how to prepare for + it, see :ref:`smart-pointers`. + + - Preparation for a new JSON output version + + - The :qpdf:ref:`--json` option takes an optional parameter + indicating the version of the JSON output. At present, there is + only one JSON version (``1``), but there are plans for an + updated version in a coming release. Until the release of qpdf + 11, the default value of ``--json`` is ``1`` for compatibility. + Once qpdf 11 is out, the default version will be ``latest``. If + you are depending on the exact format of ``--json`` for code, + you should start using ``--json=1`` in preparation. + + - New QPDFJob API exposes CLI functionality + + Prior to qpdf 10.6, a lot of the functionality implemented by the + qpdf CLI executable was built into the executable itself and not + available from the library. qpdf 10.6 introduces a new object, + ``QPDFJob``, that exposes all of the command-line functionality. + This includes a native ``QPDFJob`` API with fluent interfaces that + mirror the command-line syntax, a JSON syntax for specifying the + equivalent of a command-line invocation, and the ability to run a + qpdf "job" by passing a null-terminated array of qpdf command-line + options. The command-line argument array and JSON methods of + invoking ``QPDFJob`` are also exposed to the C API. For details, + see :ref:`qpdf-job`. + + - Other Library Enhancements + + - New ``QPDFObjectHandle`` literal syntax using C++'s user-defined + literal syntax. You can use + + .. code-block:: c++ + + auto oh = "<</Some (valid) /PDF (object)>>"_qpdf; + + to create a QPDFObjectHandle. It is a shorthand for + ``QPDFObjectHandle::parse``. + + - Preprocessor symbols ``QPDF_MAJOR_VERSION``, + ``QPDF_MINOR_VERSION``, and ``QPDF_PATCH_VERSION`` are now + available and can be used to make it easier to write code that + supports multiple versions of qpdf. You don't have to include + any new header files to get these, which makes it possible to + write code like this: + + .. code-block:: c++ + + #if !defined(QPDF_MAJOR_VERSION) || QPDF_MAJOR_VERSION < 11 + // do something using qpdf 10 or older API + #else + // do something using qpdf 11 or newer API + #endif + + Since this was introduced only in qpdf version 10.6.0, testing + for an undefined value of ``QPDF_MAJOR_VERSION`` is equivalent + to detecting a version prior to 10.6.0. + + The symbol ``QPDF_VERSION`` is also defined as a string + containing the same version number that is returned by + ``QPDF::QPDFVersion``. Note that ``QPDF_VERSION`` may differ + from ``QPDF::QPDFVersion()`` if your header files and library + are out of sync with each other. + + - The method ``QPDF::QPDFVersion`` and corresponding C API call + ``qpdf_get_qpdf_version`` are now both guaranteed to return a + reference (or pointer) to a static string, so you don't have to + copy these if you are using them in your software. They have + always returned static values. Now the fact that they return + static values is part of the API contract and can be safely + relied upon. + + - New accessor methods for ``QPDFObjectHandle``. In addition to + the traditional ones, such as ``getIntValue``, ``getName``, + etc., there are a family of new accessors whose names are of the + form ``getValueAsX``. The difference in behavior is as follows: + + - The older accessor methods, which will continue to be + supported, return the value of the object if it is the + expected type. Otherwise, they return a fallback value and + issue a warning. + + - The newer accessor methods return a boolean indicating whether + or not the object is of the expected type. If it is, a + reference of the correct type is returned. + + In many cases, the new interfaces will enable more compact code + and will also never generate type warnings. Thanks to M. Holger + for contributing these accessors. Search for ``getValueAs`` in + :file:`include/qpdf/QPDFObjectHandle.hh` for a complete list. + + These are also exposed in the C API in functions whose names + start with ``qpdf_oh_get_value_as``. + + - New convenience methods in ``QPDFObjectHandle``: + ``isDictionaryOfType``, ``isStreamOfType``, and + ``isNameAndEquals`` allow more compact querying of dictionaries. + Also added to the C API: ``qpdf_oh_is_dictionary_of_type`` and + ``qpdf_oh_is_name_and_equals``. Thanks to M. Holger for the + contribution. + + - New functions added to ``QUtil``: ``make_shared_cstr`` and + ``make_unique_cstr`` copy ``std::string`` to + ``std::shared_ptr<char>`` and ``std::unique_ptr<char[]>``. These + are alternatives to the existing ``QUtil::copy_string`` function + which offer other ways to get a C string with safer memory + management. + + - New function ``QUtil::file_can_be_opened`` tests to see whether + a file can actually be opened by attempting to open it and close + it again. + + - There is a new version of ``QUtil::call_main_from_wmain`` that + takes a ``const`` argv array and calls a main that takes a + ``const`` argv array. + + - ``QPDF::emptyPDF`` has been exposed to the C API as + ``qpdf_empty_pdf``. This makes it possible to create PDF from + scratch with the C API. + + - New C API functions ``qpdf_oh_get_binary_utf8_value`` and + ``qpdf_oh_new_binary_unicode_string`` take length parameters, + which makes it possible to handle UTF-8-encoded C strings with + embedded NUL characters. Thanks to M. Holger for the + contribution. + + - The ``JSON`` object in the qpdf library has been enhanced to + include a parser and the ability to get values out of the + ``JSON`` object. Previously it was a write-only interface. Even + so, qpdf's ``JSON`` object is not intended to be a + general-purpose JSON implementation as discussed in + :file:`include/qpdf/JSON.hh`. + + - The ``JSON`` object's "schema" checking functionality now allows + for optional keys. Note that this "schema" functionality doesn't + conform to any type of standard. It's just there to help with + error reporting with qpdf's own JSON support. + + - Documentation Enhancements + + - Documentation for the command-line tool has been completely + rewritten. This includes a top-to-bottom rewrite of :ref:`using` + in the manual. Command-line arguments are now indexed, and + internal links can appear to them within the documentation. + + - The output of ``qpdf --help`` is generated from the manual and + is divided into help topics that parallel the sections of the + manual. When you run ``qpdf --help``, instead of getting a Great + Wall of Text, you are given basic usage information and a list + of help topics. It is possible to request help for any + individual topic or any specific command-line option, or you can + get a dump of all available help text. The manual continues to + contain a greater level of detail and more examples. + + - Bug Fixes + + - Some characters were not correctly translated from PDF doc + encoding to Unicode. + 10.5.0: December 21, 2021 - Packaging changes |