Make inline image token exactly contain the image data

Do not include the trailing EI, and handle cases where EI is not
preceded by a delimiter. Such cases have been seen in the wild.

1 files changed, 13 insertions, 9 deletions

diff --git a/include/qpdf/QPDFTokenizer.hh b/include/qpdf/QPDFTokenizer.hh
index b6adeb29..ed33e13c 100644
--- a/include/qpdf/QPDFTokenizer.hh
+++ b/include/qpdf/QPDFTokenizer.hh
@@ -176,19 +176,23 @@ class QPDFTokenizer
     // Calling this method puts the tokenizer in a state for reading
     // inline images. You should call this method after reading the
     // character following the ID operator. In that state, it will
-    // return all data up to and including the next EI token. After
-    // you call this method, the next call to readToken (or the token
+    // return all data up to BUT NOT INCLUDING the next EI token. This
+    // is a difference in behavior from the legacy version. After you
+    // call this method, the next call to readToken (or the token
     // created next time getToken returns true) will either be
     // tt_inline_image or tt_bad. This is the only way readToken
-    // returns a tt_inline_image token. The version of this method
-    // that takes a PointerHolder<InputSource> does a better job of
-    // locating the end of the inline image and should be used
-    // whenever the input source is available. It preserves both
-    // tell() and getLastOffset(). The version without the input
-    // source will always end the inline image the first time it sees
-    // something that looks like an EI operator.
+    // returns a tt_inline_image token. The older version of this
+    // method that takes does not take a PointerHolder<InputSource>
+    // will always end the inline image the first time it sees
+    // something that looks like an EI operator and will include the
+    // EI operator in the token. It is being maintained for backward
+    // compatibility only and will likely be removed in the future.
     QPDF_DLL
     void expectInlineImage(PointerHolder<InputSource> input);
+
+    // Legacy version. New code should not call this. The token
+    // returned will include the EI keyword. The recipient of the
+    // token will have to remove it.
     QPDF_DLL
     void expectInlineImage();