Improve stream length recovery

Eliminate PCRE and find endobj not preceded by endstream. Be more lax
about placement of endstream and endobj.

2 files changed, 56 insertions, 11 deletions

diff --git a/qpdf/qtest/qpdf/bad24-recover.out b/qpdf/qtest/qpdf/bad24-recover.out
index d1bbe736..8d7fd87c 100644
--- a/qpdf/qtest/qpdf/bad24-recover.out
+++ b/qpdf/qtest/qpdf/bad24-recover.out
@@ -1,10 +1,25 @@
 WARNING: bad24.pdf (object 4 0, file position 385): expected endstream
 WARNING: bad24.pdf (object 4 0, file position 341): attempting to recover stream length
-WARNING: bad24.pdf (object 4 0, file position 341): unable to recover stream data; treating stream as empty
-WARNING: bad24.pdf (object 4 0, file position 778): EOF while reading token
-/QTest is implicit
-/QTest is indirect and has type null (2)
-/QTest is null
+WARNING: bad24.pdf (object 4 0, file position 341): recovered stream length: 54
+/QTest is indirect and has type stream (10)
+/QTest is a stream.  Dictionary: << /Length 44 >>
+Raw stream data:
+BT
+  /F1 24 Tf
+  72 720 Td
+  (Potato) Tj
+ET
+enxstream
+
+Uncompressed stream data:
+BT
+  /F1 24 Tf
+  72 720 Td
+  (Potato) Tj
+ET
+enxstream
+
+End of stream data
 unparse: 4 0 R
-unparseResolved: null
+unparseResolved: 4 0 R
 test 1 done
diff --git a/qpdf/qtest/qpdf/issue-101.out b/qpdf/qtest/qpdf/issue-101.out
index 7010cdda..bd457c3a 100644
--- a/qpdf/qtest/qpdf/issue-101.out
+++ b/qpdf/qtest/qpdf/issue-101.out
@@ -5,10 +5,16 @@ WARNING: issue-101.pdf (file position 1242): expected dictionary key but found n
 WARNING: issue-101.pdf (file position 1242): dictionary ended prematurely; using null as value for last key
 WARNING: issue-101.pdf (object 5 0, file position 1438): /Length key in stream dictionary is not an integer
 WARNING: issue-101.pdf (object 5 0, file position 1509): attempting to recover stream length
-WARNING: issue-101.pdf (object 5 0, file position 1509): recovered stream length: 205
+WARNING: issue-101.pdf (object 5 0, file position 1509): recovered stream length: 8
+WARNING: issue-101.pdf (trailer, file position 1631): /Length key in stream dictionary is not an integer
+WARNING: issue-101.pdf (trailer, file position 1702): attempting to recover stream length
+WARNING: issue-101.pdf (trailer, file position 1702): recovered stream length: 12
 WARNING: issue-101.pdf (trailer, file position 2026): /Length key in stream dictionary is not an integer
 WARNING: issue-101.pdf (trailer, file position 2097): attempting to recover stream length
-WARNING: issue-101.pdf (trailer, file position 2097): recovered stream length: 709
+WARNING: issue-101.pdf (trailer, file position 2097): recovered stream length: 12
+WARNING: issue-101.pdf (trailer, file position 2613): /Length key in stream dictionary is not an integer
+WARNING: issue-101.pdf (trailer, file position 2684): attempting to recover stream length
+WARNING: issue-101.pdf (trailer, file position 2684): recovered stream length: 74
 WARNING: issue-101.pdf (trailer, file position 2928): unknown token while reading object; treating as string
 WARNING: issue-101.pdf (trailer, file position 2929): unknown token while reading object; treating as string
 WARNING: issue-101.pdf (trailer, file position 2928): expected dictionary key but found non-name object; inserting key /QPDFFake1
@@ -22,8 +28,32 @@ WARNING: issue-101.pdf (trailer, file position 3410): attempting to recover stre
 WARNING: issue-101.pdf (trailer, file position 3410): recovered stream length: 12
 WARNING: issue-101.pdf (trailer, file position 3560): /Length key in stream dictionary is not an integer
 WARNING: issue-101.pdf (trailer, file position 3631): attempting to recover stream length
-WARNING: issue-101.pdf (trailer, file position 3631): recovered stream length: 167
+WARNING: issue-101.pdf (trailer, file position 3631): recovered stream length: 8
 WARNING: issue-101.pdf (trailer, file position 4113): /Length key in stream dictionary is not an integer
 WARNING: issue-101.pdf (trailer, file position 4184): attempting to recover stream length
-WARNING: issue-101.pdf (trailer, file position 4184): unable to recover stream data; treating stream as empty
-issue-101.pdf: unable to find trailer dictionary while recovering damaged file
+WARNING: issue-101.pdf (trailer, file position 4184): recovered stream length: 8
+WARNING: issue-101.pdf (file position 591): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 625): treating unexpected brace token as null
+WARNING: issue-101.pdf (file position 626): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 637): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 639): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 644): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 647): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 687): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 691): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 696): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 698): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 701): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 711): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 742): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 745): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 747): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 777): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 790): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 800): treating unexpected brace token as null
+WARNING: issue-101.pdf (file position 801): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 811): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 819): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 832): unknown token while reading object; treating as string
+WARNING: issue-101.pdf (file position 856): unexpected >
+issue-101.pdf (file position 856): unable to find /Root dictionary