Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bugzilla DB migration completed. Please report issues to Infra team via email via infra@gentoo.org or IRC
Bug 606768 - app-admin/ulogd-2.0.5_p20161017[doc] - nsgmls:<OSFD>0:190:12:E: end tag for "DESCRIP" omitted, but its declaration does not permit this
Summary: app-admin/ulogd-2.0.5_p20161017[doc] - nsgmls:<OSFD>0:190:12:E: end tag for "...
Status: IN_PROGRESS
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: AMD64 Linux
: Normal normal (vote)
Assignee: No maintainer - Look at https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers if you want to take care of it
URL: https://gitlab.com/agmartin/linuxdoc-...
Whiteboard:
Keywords: UPSTREAM
: 599948 624488 625348 625360 (view as bug list)
Depends on: 667944
Blocks:
  Show dependency tree
 
Reported: 2017-01-22 05:28 UTC by deference
Modified: 2019-09-25 10:49 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (build-ulogd.txt,56.84 KB, text/plain)
2017-01-22 05:28 UTC, deference
Details
emerge --info (emergeinfo-ulogd.txt,30.21 KB, text/plain)
2017-01-22 05:28 UTC, deference
Details
out.3 (ulogd-out.3,8.94 KB, text/plain)
2017-07-03 20:48 UTC, deference
Details
strace.cat (ulogd-strace.cat,592.83 KB, text/plain)
2017-07-03 20:49 UTC, deference
Details
out strace cat (ulogd-out.strace.cat,8.94 KB, text/plain)
2017-07-03 20:50 UTC, deference
Details
cat guide.sgml | onsgmls > out.3 (out.3,8.67 KB, application/x-troff-man)
2017-07-11 09:25 UTC, Yarda
Details
Python 2 script for generating a test case for possible nsgmls bug (genBugTxt.py,963 bytes, text/x-python)
2018-06-30 08:19 UTC, Anon Emuss
Details
DTD file for output of genBugTxt.py (linuxdoc.dtd,16.14 KB, application/xml-dtd)
2018-06-30 08:20 UTC, Anon Emuss
Details
1000_LinuxDocTools.pm_nsgmls-no-pipe.diff (1000_LinuxDocTools.pm_nsgmls-no-pipe.diff,1.89 KB, patch)
2018-07-16 15:11 UTC, Coacher
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description deference 2017-01-22 05:28:11 UTC
[ebuild     U ] app-admin/ulogd-2.0.5_p20161017 [2.0.5_p20160205] USE="dbi doc* json mysql nfacct nfct nflog pcap postgres sqlite ulog" 

make: Entering directory '/var/tmp/portage/app-admin/ulogd-2.0.5_p20161017/work/ulogd-2.0.5_p20161017/doc'
sgml2txt ulogd.sgml
Processing file ulogd.sgml
nsgmls:<OSFD>0:190:12:E: end tag for "DESCRIP" omitted, but its declaration does not permit this
nsgmls:<OSFD>0:182:0: start tag was here
nsgmls:<OSFD>0:190:12:E: end tag for "ARTICLE" omitted, but its declaration does not permit this
nsgmls:<OSFD>0:3:0: start tag was here
LinuxdocTools::process_file: nsgmls failed with exit status 1. Aborting ...
Makefile:2: recipe for target 'all' failed
make: *** [all] Error 25
Comment 1 deference 2017-01-22 05:28:25 UTC
Created attachment 460874 [details]
build.log
Comment 2 deference 2017-01-22 05:28:32 UTC
Created attachment 460876 [details]
emerge --info
Comment 3 Coacher 2017-01-22 17:46:08 UTC
Does 

cat /var/tmp/portage/app-admin/ulogd-2.0.5_p20161017/work/ulogd-2.0.5_p20161017/doc/ulogd.sgml | sed 's/_/\\&lowbar;/g' | /usr/bin/sgmlpre output=txt | ( nsgmls -D /usr/share/sgml -D /usr/share/linuxdoc-tools -ifmttxt ) > /dev/null

command work?
Comment 4 Coacher 2017-04-15 18:57:00 UTC
No response. Can't reproduce.
Comment 5 deference 2017-04-19 00:21:35 UTC
I'm sorry. I meant to do this. I had the bug report on my computer for me to take care of. I completely forgot.
nsgmls:<OSFD>0:190:5:E: end tag for "DESCRIP" omitted, but its declaration does not permit this
nsgmls:<OSFD>0:182:0: start tag was here
nsgmls:<OSFD>0:190:5:E: end tag for "ARTICLE" omitted, but its declaration does not permit this
nsgmls:<OSFD>0:3:0: start tag was here
Comment 6 Coacher 2017-04-19 06:10:34 UTC
(In reply to deference from comment #5)
> I'm sorry. I meant to do this. I had the bug report on my computer for me to
> take care of. I completely forgot.
> nsgmls:<OSFD>0:190:5:E: end tag for "DESCRIP" omitted, but its declaration
> does not permit this
> nsgmls:<OSFD>0:182:0: start tag was here
> nsgmls:<OSFD>0:190:5:E: end tag for "ARTICLE" omitted, but its declaration
> does not permit this
> nsgmls:<OSFD>0:3:0: start tag was here
This is the error when running command from comment #3, right?
Comment 7 deference 2017-04-19 22:25:25 UTC
Yes.
Comment 8 Coacher 2017-04-20 08:33:51 UTC
(In reply to deference from comment #7)
> Yes.
Ok. Thanks. Now let's find where this pipeline fails.

Please execute the following commands and paste links into a comment here:

cat /var/tmp/portage/app-admin/ulogd-2.0.5_p20161017/work/ulogd-2.0.5_p20161017/doc/ulogd.sgml | wgetpaste

cat /var/tmp/portage/app-admin/ulogd-2.0.5_p20161017/work/ulogd-2.0.5_p20161017/doc/ulogd.sgml | sed 's/_/\\&lowbar;/g' | wgetpaste

cat /var/tmp/portage/app-admin/ulogd-2.0.5_p20161017/work/ulogd-2.0.5_p20161017/doc/ulogd.sgml | sed 's/_/\\&lowbar;/g' | /usr/bin/sgmlpre output=txt | wgetpaste
Comment 10 Coacher 2017-05-01 18:06:29 UTC
(In reply to deference from comment #9)
> Cmd 1: http://bpaste.net/raw/0d50e2d86f93
> Cmd 2: http://bpaste.net/raw/cd0fd08f161a
> Cmd 3: http://bpaste.net/raw/9bc9839cf458
> Thanks
For the last couple of days I've been trying to look at your links, but bpaste.net seem to be malfunctioning:

~ $ wget --tries=3 http://bpaste.net/raw/9bc9839cf458
URL transformed to HTTPS due to an HSTS policy
--2017-05-01 20:54:19--  https://bpaste.net/raw/9bc9839cf458
Resolving bpaste.net (bpaste.net)... 149.202.95.76
Connecting to bpaste.net (bpaste.net)|149.202.95.76|:443... failed: Connection timed out.
Retrying.

--2017-05-01 20:56:32--  (try: 2)  https://bpaste.net/raw/9bc9839cf458
Connecting to bpaste.net (bpaste.net)|149.202.95.76|:443... failed: Connection timed out.
Retrying.

--2017-05-01 20:58:44--  (try: 3)  https://bpaste.net/raw/9bc9839cf458
Connecting to bpaste.net (bpaste.net)|149.202.95.76|:443... failed: Connection timed out.
Giving up.

~ $ wget --tries=3 https://gentoo.org
--2017-05-01 21:05:01--  https://gentoo.org/
Resolving gentoo.org (gentoo.org)... 89.16.167.134
Connecting to gentoo.org (gentoo.org)|89.16.167.134|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 23996 (23K) [text/html]
Saving to: ‘index.html’

index.html             100%[=========================>]  23,43K  --.-KB/s    in 0,004s

2017-05-01 21:05:01 (5,13 MB/s) - ‘index.html’ saved [23996/23996]


Could you please upload those to a more reliable pastebin?
Comment 11 deference 2017-05-03 22:13:36 UTC
I verified these links are good after pasting:
http://dpaste.com/3JD6NJH
http://dpaste.com/3KWD054
http://dpaste.com/3JD6NJH
Thanks again.
Comment 12 Coacher 2017-05-03 22:17:43 UTC
(In reply to deference from comment #11)
> I verified these links are good after pasting:
> http://dpaste.com/3JD6NJH
> http://dpaste.com/3KWD054
> http://dpaste.com/3JD6NJH
> Thanks again.
Something is likely wrong. Two links are the same.
Comment 13 Coacher 2017-05-06 01:59:56 UTC
(In reply to Coacher from comment #12)
> Something is likely wrong. Two links are the same.
To clarify: I expect 3 different links, but there are only 2 different links.
Comment 14 deference 2017-05-07 03:35:27 UTC
Sorry, clipboard error.
1. http://dpaste.com/2111XZ2
2. http://dpaste.com/2HFRQ74
3. http://dpaste.com/2VRNWXG
Comment 15 Coacher 2017-05-09 18:07:47 UTC
Your files are correct and nsgmls successfully processes the last one. There's no problem with file generation.

It seems your issue occurs because nsgmls at the end of the pipeline from comment #3 receives only the first 8192 bytes of the output from previous commands instead of the whole file.
Do you have any ideas why this could be happening? Do you have some resource limits set up?
Comment 16 Coacher 2017-05-20 22:11:19 UTC
*** Bug 599948 has been marked as a duplicate of this bug. ***
Comment 17 deference 2017-05-24 16:52:38 UTC
I don't want you to think that I'm dispersing on you again.
I don't remember ever setting any limits on anything except sudo, but I needed to rebuild my system because of a gcc upgrade, so I created a new partition, chrooted and am rebuilding. If this persists then it is because of something I did not do, but I've not yet gotten to the point where I install ulogd.
Comment 18 Coacher 2017-05-24 17:24:16 UTC
(In reply to deference from comment #17)
> I don't want you to think that I'm dispersing on you again.
> I don't remember ever setting any limits on anything except sudo, but I
> needed to rebuild my system because of a gcc upgrade, so I created a new
> partition, chrooted and am rebuilding. If this persists then it is because
> of something I did not do, but I've not yet gotten to the point where I
> install ulogd.
I don't think so, don't worry :)

Please see upstream bugreport in ${URL}.
It would be great if you could provide info requested in these comments:
https://gitlab.com/agmartin/linuxdoc-tools/issues/10#note_30112403
https://gitlab.com/agmartin/linuxdoc-tools/issues/10#note_30302841

Instead of BrlAPI.sgml you can use ulogd.sgml. If you don't want to register on gitlab feel free to send the results to me. I'll attach them there.
Comment 19 deference 2017-07-03 20:48:14 UTC
Created attachment 480596 [details]
out.3

OK, I rebuild my system. I'm using the same locale, timezone, hosts, /etc/portage/FOO, and login.defs files.
I am still getting the same error.
As requested attached is out.3.
I'll add the others in subsequent comments.
Comment 20 deference 2017-07-03 20:49:02 UTC
Created attachment 480598 [details]
strace.cat
Comment 21 deference 2017-07-03 20:50:04 UTC
Created attachment 480602 [details]
out strace cat
Comment 22 Coacher 2017-07-06 09:12:31 UTC
(In reply to deference from comment #19)
> Created attachment 480596 [details]
> out.3
> 
> OK, I rebuild my system. I'm using the same locale, timezone, hosts,
> /etc/portage/FOO, and login.defs files.
> I am still getting the same error.
> As requested attached is out.3.
> I'll add the others in subsequent comments.
Thank you. Looks like onsgmls on your system truncates input.
This means that the problem is likely in app-text/opensp package.
Comment 23 deference 2017-07-07 13:20:46 UTC
Should I file a new bug for that package, and if so, how do I make this bug depend on that one?
Comment 24 Coacher 2017-07-07 13:37:32 UTC
(In reply to deference from comment #23)
> Should I file a new bug for that package, and if so, how do I make this bug
> depend on that one?
You should open a new bug only if the same truncation as seen in comment #19 happens with other sgml files. I'll take care of the bug dependencies then.
Comment 25 Coacher 2017-07-10 20:28:51 UTC
*** Bug 624488 has been marked as a duplicate of this bug. ***
Comment 26 Coacher 2017-07-10 20:31:33 UTC
Yarda, thank you for the report.

Please attach out.3 from
`cat /var/tmp/portage/app-text/linuxdoc-tools-0.9.72/work/linuxdoc-tools-0.9.72/doc/guide.sgml | onsgmls > out.3`
Comment 27 Yarda 2017-07-11 09:25:49 UTC
Created attachment 482990 [details]
cat guide.sgml | onsgmls > out.3
Comment 28 Yarda 2017-07-11 09:27:19 UTC
(In reply to Coacher from comment #26)
> Yarda, thank you for the report.
> 
> Please attach out.3 from
> `cat
> /var/tmp/portage/app-text/linuxdoc-tools-0.9.72/work/linuxdoc-tools-0.9.72/
> doc/guide.sgml | onsgmls > out.3`

Thanks for prompt response, output attached. I also tried to rebuild app-text/opensp-1.5.2-r3 and also update to app-test/opensp-1.5.2-r4, but it didn't help.
Comment 29 Yarda 2017-07-11 12:37:59 UTC
I am not sure whether my problem is the same, because after some debugging it seems the tool  doesn't truncate the pipe input.

For me the problem seems to be in app-text/docbook-sgml-dtd. I fixed it by uninstallation of the old DTDs:

# emerge -C app-text/docbook-sgml-dtd:3.0 app-text/docbook-sgml-dtd:3.1 app-text/docbook-sgml-dtd:4.0 app-text/docbook-sgml-dtd:4.1

And installation of the latest version:

# emerge app-text/docbook-sgml-dtd

which installed app-text/docbook-sgml-dtd-4.4 which wasn't previously there (emerge -uDN world didn't updated it).

Now it correctly emerges app-text/linuxdoc-tools with the USE=doc.
Comment 30 Yarda 2017-07-11 12:40:37 UTC
(In reply to Yarda from comment #29)
> And installation of the latest version:
> 
> # emerge app-text/docbook-sgml-dtd
> 
> which installed app-text/docbook-sgml-dtd-4.4 which wasn't previously there
> (emerge -uDN world didn't updated it).

Just manual upgrade to app-text/docbook-sgml-dtd-4.4 is not enough, from the strace it seems the tool is still using the old DTDs, I had to remove the old DTDs for onsgmls to start working.
Comment 31 Yarda 2017-07-11 13:00:28 UTC
(In reply to Yarda from comment #30)
> (In reply to Yarda from comment #29)
> > And installation of the latest version:
> > 
> > # emerge app-text/docbook-sgml-dtd
> > 
> > which installed app-text/docbook-sgml-dtd-4.4 which wasn't previously there
> > (emerge -uDN world didn't updated it).
> 
> Just manual upgrade to app-text/docbook-sgml-dtd-4.4 is not enough, from the
> strace it seems the tool is still using the old DTDs, I had to remove the
> old DTDs for onsgmls to start working.

I am not sure whether it's real solution or just sideeffect that it compiled because all the removed DTDs seems to be needed on my system:

# emerge -uDNpv --tree world
[nomerge       ] app-text/docbook-sgml-utils-0.6.14-r1::gentoo  USE="jadetex"
[ebuild  NS    ]  app-text/docbook-sgml-dtd-4.0-r3:4.0::gentoo [4.4:4.4::gentoo] 0 KiB
[ebuild  NS    ]  app-text/docbook-sgml-dtd-4.1-r3:4.1::gentoo [4.4:4.4::gentoo] 0 KiB
[ebuild  NS    ]  app-text/docbook-sgml-dtd-3.0-r3:3.0::gentoo [4.4:4.4::gentoo] 0 KiB
[ebuild  NS    ]  app-text/docbook-sgml-dtd-3.1-r3:3.1::gentoo [4.4:4.4::gentoo] 0 KiB

When I emerged them app-text/linuxdoc-tools with the USE=doc again stop compiling.
Comment 32 Coacher 2017-07-11 18:54:02 UTC
(In reply to Yarda from comment #31)
> (In reply to Yarda from comment #30)
> > (In reply to Yarda from comment #29)
On my system merging docbook-sgml-dtd:4.4 or any other version alone results in a broken linuxdoc-tools build. But having 4.3 and 4.4 merged together works fine.

deference@null.net, can you confirm that with all docbook-sgml-dtd versions removed linuxdoc-tools builds fine?
Comment 33 Coacher 2017-07-11 19:04:19 UTC
(In reply to Coacher from comment #32)
> (In reply to Yarda from comment #31)
> > (In reply to Yarda from comment #30)
> > > (In reply to Yarda from comment #29)
> On my system merging docbook-sgml-dtd:4.4 or any other version alone results
> in a broken linuxdoc-tools build. But having 4.3 and 4.4 merged together
> works fine.
> 
> deference@null.net, can you confirm that with all docbook-sgml-dtd versions
> removed linuxdoc-tools builds fine?
Or ulogd[doc], it shouldn't matter.
Comment 34 Coacher 2017-07-17 11:56:19 UTC
*** Bug 625348 has been marked as a duplicate of this bug. ***
Comment 35 deference 2017-07-19 02:43:03 UTC
I unmerged all versions of docbook-sgml-dtd and ulogd[doc] does not build. The same applies to linuxdoc-tools[doc]. I have had to force the removal of several of the docbook-sgml-dtd, that it to say, they would not depclean and be gone, I used --unmerge.
I tested ulogd[doc] after depclean while having the following versions of docbook-sgml-dtd installed without success: 3.0-r3, 4.0-r3, 4.1-r3, 4.4 and 3.1-r3.
I then emerged only 4.3-r2 and then tried both packages again without success.
I don't know where to go from here.
Comment 36 deference 2017-07-19 02:53:26 UTC
Let's see now I'll try merging 4.4 and see if that works...
No, both packages still will not merge.
Comment 37 deference 2018-03-09 22:46:14 UTC
I am most sorry, my desktop system failed and my laptop also did about 1 mouth ago. I can no longer contribute to Gentoo Linux until I get a new system and my life back in order. I am currently working off an old i386 system and borrowed laptop so I don't have the compute power to spare for running Gentoo.
I'll be back, but in the meantime please take what action you will.

Thanks for all the penguins and God bless!
Comment 38 Coacher 2018-06-24 14:25:50 UTC
Cannot reproduce with ulogd-2.0.7 and current amd64 system.

If someone can, please comment, otherwise I'm going to close this bug once ulogd-2.0.7 goes stable.
Comment 39 Anon Emuss 2018-06-30 08:16:52 UTC
I have the same problem on my system.  Thanks to the command in comment 3, I was able to track the problem down to what I think is a bug in the nsgmls program.  I wrote a little demo program to demonstrate the problem:
  $ ./genBugTxt.py demo.txt
  Detected block size: 4096
  Wrote demo.txt (size: 4114 bytes).

  $ nsgmls -D dtd <demo.txt >/dev/null
  nsgmls:<OSFD>0:170:19:E: end tag for "ARTICLE" omitted, but its declaration does not permit this
  nsgmls:<OSFD>0:3:0: start tag was here

  $ nsgmls -D dtd demo.txt >/dev/null
The last command succeeds without error.  I will try attaching the script and DTD I used.  Here is the version of nsgmls I used:
  $ equery b /usr/bin/nsgmls
   * Searching for /usr/bin/nsgmls ...
  app-text/openjade-1.3.2-r7 (/usr/bin/nsgmls -> onsgmls)
  app-text/opensp-1.5.2-r3 (/usr/bin/onsgmls)
I do not think the exact compilation environment matters, as I was able to reproduce the problem with a manual compile in a debug environment as well.

I dove into the source code for nsgmls, and quickly reached a point where I think one of the developers for the package would make better progress.  The issue appears to be that nsgmls creates multiple PosixStorageObject instances on the main input (demo.txt in the first example, and stdin in the second).  Each time, it seems to check a character, which causes it to read a block in to a buffer, presumably after attempting to reset the file position to 0.  This appears to work when reading demo.txt, but seeking does not work on stdin, so it does not try.  Each PosixStorageObject gets a different block from stdin in the demo case, until eventually reading stdin returns EOF.  I think it is the first PosixStorageObject that eventually gets passed to the parsing code, because the parsing code only receives the first block.  When that PosixStorageObject attempts to read more from stdin, it gets an EOF, and the parsing terminates there.

nsgmls appears to get a block size from a system stat() call on the file (or stdin).  On my system, it appears to return 4096 (I do not know what limits that -- is it set by the filesystem?).  The output passed to nsgmls via stdin in the command in comment 3 is over 25000 bytes, but the parsing code in nsgmls only receives the first 4096 bytes (the first block), because the rest was read by other PosixStorageObject instances during initialization.  This appears to be a bug, but I do not know enough about why those PosixStorageObject instances are created to know what to do about it.

On my system, the equivalent of the command in comment 3 fails with the missing end tag warnings, but if I save the output of sgmlpre to a file and call nsgmls on the file instead of stdin, it succeeds, just like my example.

In my example, the genBugTxt.py script creates a simple demo.txt file, uses a stat() call to determine the block size, and padds demo.txt until it is larger than that.  Passing demo.txt to nsgmls via stdin fails, as it does not get the full file, and passing demo.txt as a filename on the command line succeeds.  Please let me know if it helps in reproducing the problem (it only appears to work with python2 [2.7] and not python3).
Comment 40 Anon Emuss 2018-06-30 08:19:20 UTC
Created attachment 537848 [details]
Python 2 script for generating a test case for possible nsgmls bug

Run:
  ./genBugTxt.py demo.txt
This will hopefully create a demo.txt file of the appropriate size to demonstrate the bug.
Comment 41 Anon Emuss 2018-06-30 08:20:58 UTC
Created attachment 537850 [details]
DTD file for output of genBugTxt.py

This is file in the dtd directory passed via the -D option to nsgmls when parsing the output of genBugTxt.py:
  nsgmls -D dtd <demo.txt >/dev/null
vs.
  nsgmls -D dtd demo.txt >/dev/null
Comment 42 Anon Emuss 2018-06-30 08:52:10 UTC
(In reply to Anon Emuss from comment #41)
> Created attachment 537850 [details]
> DTD file for output of genBugTxt.py
> 
> This is file in the dtd directory passed via the -D option to nsgmls when
> parsing the output of genBugTxt.py:
>   nsgmls -D dtd <demo.txt >/dev/null
> vs.
>   nsgmls -D dtd demo.txt >/dev/null

This was taken from linuxdoc-tools-0.9.72 (/usr/share/linuxdoc-tools/dtd/linuxdoc.dtd -> /usr/share/linuxdoc-tools/dtd/linuxdoc96.dtd) in the hopes that this demo would work without having linuxdoc-tools installed, but that does not appear to be the case.  The general trick of inflating any input file that works to be larger than the block size will hopefully still trigger the problem.
Comment 43 Coacher 2018-06-30 13:05:36 UTC
Anon Emuss, thank you for the extensive report.

I too suspected nsgmls, but didn't have any sound evidence.
It's interesting that with the same versions of installed software some people
trigger this and some people don't.

Since you have a reliable way to reproduce this, could you please test one of my guesses: recompile app-text/opensp-1.5.2-r3 with all optimizations disabled (-O0 in CFLAGS/CXXFLAGS) and see if you can still trigger this bug.
Comment 44 Anon Emuss 2018-07-01 09:16:49 UTC
(In reply to Coacher from comment #43)
> Anon Emuss, thank you for the extensive report.

You are welcome.  I hope it turns out to be helpful.

> I too suspected nsgmls, but didn't have any sound evidence.
> It's interesting that with the same versions of installed software some
> people trigger this and some people don't.

Other than weird ideas like strange compiler malfunctions (I have gcc-7.3.0-r3 and glibc-2.26-r7, if that is useful), I was guessing maybe it was the block size.  It is 4096 for me.  Comment 15 makes it sound like it was 8192 for deference.  The data used when installing ulogd[doc], for me, is under 26000 bytes, so anybody with a block size of at least 32k should not have a problem.

It sounds like the problem does not occur for you.  I am curious what your block size is.  genBugTxt.py reports it, or a Perl one-liner can do the trick, too:
  $ perl -e 'my @s=stat("/var/tmp/portage");print $s[11].$/;'
  4096

> Since you have a reliable way to reproduce this, could you please test one
> of my guesses: recompile app-text/opensp-1.5.2-r3 with all optimizations
> disabled (-O0 in CFLAGS/CXXFLAGS) and see if you can still trigger this
> bug.

I tried two ways.  One way was to use emerge:
  $ CFLAGS=-O0 CXXFLAGS=-O0 emerge -a1 opensp
This installed app-text/opensp-1.5.2-r3.  With that, I ran (using demo.txt generated from genBugTxt.py):
  $ nsgmls -ifmttxt <demo.txt >/dev/null
  nsgmls:<OSFD>0:170:19:E: end tag for "ARTICLE" omitted, but its declaration does not permit this
  nsgmls:<OSFD>0:3:0: start tag was here
  $ nsgmls -ifmttxt demo.txt >/dev/null
The last command ran without error.  This appears to reproduce the problem.

In case the CFLAGS/CXXFLAGS did not work as I expected, or emerge overrode something, I also tried a more manual approach.  I unpacked the source with:
  $ ebuild /usr/portage/app-text/opensp/opensp-1.5.2-r3.ebuild unpack 
Then, I copied the source to another location, where I ran (I did not configure system catalog locations, so I supplied them on the command line):
  $ CFLAGS=-O0 CXXFLAGS=-O0 ./configure
  <cut>
  $ make
  <cut>
  $ nsgmls/onsgmls -c /etc/sgml/catalog -ifmttxt </tmp/demo.txt >/dev/null
  nsgmls/.libs/lt-onsgmls:<OSFD>0:170:19:E: end tag for "ARTICLE" omitted, but its declaration does not permit this
  nsgmls/.libs/lt-onsgmls:<OSFD>0:3:0: start tag was here
  $ nsgmls/onsgmls -c /etc/sgml/catalog -ifmttxt /tmp/demo.txt >/dev/null
The last command ran without error.  This also appears to reproduce the
problem.  The environment at both configure and compilation time did not have any compiler variables set (save for the ones on the command line), so I think that should have the most minimal flags.
Comment 45 Coacher 2018-07-01 13:09:56 UTC
Yes. Problem for some reason doesn't occur here.

```
coacher@Photon ~/Downloads $ python2 genBugTxt.py demo.txt
Detected block size: 4096
Wrote demo.txt (size: 4114 bytes).
coacher@Photon ~/Downloads $ nsgmls -ifmttxt <demo.txt >/dev/null
coacher@Photon ~/Downloads $ nsgmls -ifmttxt demo.txt >/dev/null
```

All commands work fine here.

Reasonable compiler flags:
CFLAGS="-O2 -march=native -mfpmath=sse -fomit-frame-pointer -pipe -frecord-gcc-switches -Wimplicit-function-declaration"
CXXFLAGS="-O2 -march=native -mfpmath=sse -fomit-frame-pointer -pipe -frecord-gcc-switches"

app-text/opensp-1.5.2-r3  USE="nls -doc -static-libs {-test}"
app-text/openjade-1.3.2-r7  USE="-static-libs"
Comment 46 Coacher 2018-07-01 13:13:05 UTC
Also I remember that installed dtds affected this some time in the past.
I have:
```
coacher@Photon ~ $ cat /etc/sgml/catalog
CATALOG "/etc/sgml/sgml-docbook.cat"
CATALOG "/etc/sgml/sgml-ent.cat"
CATALOG "/etc/sgml/xml-docbook-4.2.cat"
CATALOG "/etc/sgml/xml-docbook-4.1.2.cat"
CATALOG "/etc/sgml/xml-docbook-4.5.cat"
CATALOG "/etc/sgml/xml-docbook-4.3.cat"
CATALOG "/etc/sgml/xml-docbook-4.4.cat"
CATALOG "/etc/sgml/openjade-1.3.2.cat"
CATALOG "/etc/sgml/linuxdoc.cat"
```

What do you have, Anon Emuss?
Comment 47 Anon Emuss 2018-07-02 06:18:26 UTC
(In reply to Coacher from comment #46)
> Also I remember that installed dtds affected this some time in the past.
>...

Coacher:
That is a good point.  I should have played with that earlier.

My /etc/sgml/catalog has more entries, so I tried cutting it down.  I found a 2-line version that allows both of the commands I have to work.
  $ cat /etc/sgml/catalog
  CATALOG "/etc/sgml/sgml-ent.cat"
  CATALOG "/etc/sgml/linuxdoc.cat"
  $ nsgmls -ifmttxt <demo.txt >/dev/null
  $ nsgmls -ifmttxt demo.txt >/dev/null
With that 2-line /etc/sgml/catalog, both of the two methods of parsing demo.txt appear to work.  Next, I tried adding back various lines of the original /etc/sgml/catalog, and when I found a line that triggered the problem, I descended into it.  This was the file that appeared to trigger the problem:
  /etc/sgml/dsssl-docbook-stylesheets.cat
I notice it does NOT appear in your /etc/sgml/catalog.  On my system, it is also referenced in a few of the files that DO appear in your /etc/sgml/catalog (e.g. /etc/sgml/sgml-docbook.cat), so it is unlikely that just trying to get our topmost catalog files to match will work.  I am hoping that will not be necessary, because I think I found the important lines.  On my system, /etc/sgml/dsssl-docbook-stylesheets.cat contains a single line which essentially includes this file:
  /usr/share/sgml/docbook/dsssl-stylesheets-1.79/catalog
which is owned by app-text/docbook-dsssl-stylesheets-1.79-r2.  Playing around with the lines of that file, I found a single line (split into two lines, technically) that seemed to cause the problem:
  DTDDECL "-//Norman Walsh//DTD DocBook HTML 1.0//EN"
    dtds/html/dbhtml.dcl
As near as I can tell, dbhtml.dcl does not exist on my system (locate does not find it, and find on that directory tree does not find anything with that name).  It appears that any DTDDCL line causes the problem, whether or not the file exists.  It does not appear to be required that it be included directly through /etc/sgml/catalog, so I am hoping that this will reproduce the problem on your system:
  $ cat /tmp/dtddcl.cat
  DTDDECL "FooBar" /foo/bar
  $ nsgmls -ifmttxt <demo.txt >/dev/null
  $ nsgmls -c /tmp/dtddcl.cat -ifmttxt <demo.txt >/dev/null
  nsgmls:<OSFD>0:170:19:E: end tag for "ARTICLE" omitted, but its declaration does not permit this
  nsgmls:<OSFD>0:3:0: start tag was here
I can either include that dummy DTDDECL line directly in /etc/sgml/catalog, or via another catalog using the -c option as above, and it triggers the problem.

There is a little more to it than that, as I was able to put together a different txt file that is over 4k that DOES work.  It uses the declarations in /usr/share/sgml/sgml-dtd-4.1/ instead of something from /usr/share/linuxdoc-tools/, so there is apparently something about the linuxdoc-tools DTDs that do not work with a DTDDECL line.

Would you be willing to try running the same commands, with the same demo.txt, with the dummy DTDDECL line either added to your /etc/sgml/catalog or added via a -c switch on the command line?  I am curious whether that will trigger the problem on your system.  I am hoping it will trigger the problem without needing to modify your /etc/sgml/catalog.

Thanks for your help with this.
Comment 48 Coacher 2018-07-02 11:37:00 UTC
Success! Now I can reproduce this:
```
coacher@Photon ~/Downloads $ cat dtddcl.cat
DTDDECL "FooBar" /foo/bar
coacher@Photon ~/Downloads $ python2 genBugTxt.py demo.txt
Detected block size: 4096
Wrote demo.txt (size: 4114 bytes).
coacher@Photon ~/Downloads $ nsgmls -ifmttxt demo.txt >/dev/null
coacher@Photon ~/Downloads $ nsgmls -c dtddcl.cat -ifmttxt demo.txt >/dev/null
coacher@Photon ~/Downloads $ nsgmls -ifmttxt <demo.txt >/dev/null
coacher@Photon ~/Downloads $ nsgmls -c dtddcl.cat -ifmttxt <demo.txt >/dev/null
nsgmls:<OSFD>0:170:19:E: end tag for "ARTICLE" omitted, but its declaration does not permit this
nsgmls:<OSFD>0:3:0: start tag was here
```

Re dtds/html/dbhtml.dcl, please check if the file /usr/share/sgml/docbook/dsssl-stylesheets-1.79/html/dbhtml.dsl exists. Here docbook-dsssl-stylesheets-1.79-r2.ebuild installs it.

Seems paths are a bit broken in usr/share/sgml/docbook/dsssl-stylesheets-1.79/catalog. The files are there, but in different locations. Should be easy to fix.


Could you please manually do the following as root:
cd /usr/share/sgml/docbook/dsssl-stylesheets-1.79/dtds/
ln -s ../html html

And then re-run your tests with all your regular system-wide dtds.
See if this solves the issue.

Afterwards you can remove the created link via (notice no trailing slash):
rm /usr/share/sgml/docbook/dsssl-stylesheets-1.79/dtds/html
Comment 49 Anon Emuss 2018-07-03 04:39:49 UTC
(In reply to Coacher from comment #48)
> Success! Now I can reproduce this:
That sounds like progress.  I am frequently amused by the amount of work that goes into causing a problem before even being able to fix it or work around it.

>...
> Re dtds/html/dbhtml.dcl, please check if the file
> /usr/share/sgml/docbook/dsssl-stylesheets-1.79/html/dbhtml.dsl exists. Here
> docbook-dsssl-stylesheets-1.79-r2.ebuild installs it.
It exists, and appears to be installed by the same ebuild.

> Seems paths are a bit broken in
> usr/share/sgml/docbook/dsssl-stylesheets-1.79/catalog. The files are there,
> but in different locations. Should be easy to fix.
I am not so sure the files are there.  The extensions are different, and the contents look different.  See comments below.

> Could you please manually do the following as root:
> cd /usr/share/sgml/docbook/dsssl-stylesheets-1.79/dtds/
> ln -s ../html html
>...
All relative paths below are relative to /usr/share/sgml/docbook/dsssl-stylesheets-1.79/ (except demo.txt, which is the usual file).  Adding the symlink dtds/html -> ../html did not affect the problem; demo.txt still failed to parse when passed through stdin to nsgmls.  catalog points to dtds/html/dbhtml.dcl.  Adding the symlink would allow it to find html/dbhtml.dcl, but that does not exist -- the file in that directory is html/dbhtml.dsl (slightly different extension).  I tried modifying catalog to point at html/dbhtml.dsl, and that failed to fix the problem.  I also tried modifying it to point at an existing .dcl file (/usr/share/sgml/docbook/sgml-dtd-3.0/docbook.dcl), and that also failed to fix the problem.  Looking at the contents of /usr/share/sgml/xml.dcl and /usr/share/sgml/docbook/sgml-dtd-3.0/docbook.dcl, the contents look pretty different from html/dbhtml.dsl; I suspect the latter is a different type of file.

As far as missing files go, /usr/share/sgml/docbook/dsssl-stylesheets-1.79/ has several.  Here are some others (along with some guesses at which files in /usr/share/sgml/ might be similar):
--------
  dtds/dbdsssl/dbdsssl.dtd
    There are plenty of .dtd files in /usr/share/sgml/, but no matches for dbdsssl.*.
  dtds/html/ISOlat1.gml
    There are no .gml files in /usr/share/sgml/.  There are several versions of ISOlat1.ent and ISOlat1.sgm.
  docsrc/{docbook,olinktarget,olinksemantics}.sgm
    The only docsrc/ directory is /usr/share/sgml/xml-stylesheets/docsrc/.  It does not include any .sgm files, or anything with docbook or olink in its name.  The only .sgm files are:
      /usr/share/sgml/openjade-1.3.2/pubtext/ISOlat1.sgm
      /usr/share/sgml/opensp-1.5.2/OpenSP/ISOlat1.sgm
      /usr/share/sgml/opensp-1.5.2/OpenSP/demo.sgm
    which do not seem like a match.
--------

So far, it appears that nsgmls only parses the first block of stdin if using the  linuxdoc-tools docbook style, and the catalogs include a DTDDCL line, whether or not the file referenced in that line exists.  I suppose a next step would be to try taking apart the linuxdoc-tools files and see if any particular line seems to trigger the problem.  That, or I could dive into the nsgmls source code again and see if I could find some lines that seem to trigger the issue, but perhaps the OpenSP developers would be more efficient at that.  As I mentioned before, I got a file to work that uses a different document type, which suggests there is something specific about the linuxdoc-tools files that does not work well with the DTDDECL.

I will probably play with the linuxdoc-tools files, but I am open to suggestions.
Comment 50 Coacher 2018-07-04 19:11:08 UTC
The files are there in the tarball (only docsrc is missing from the tarball). They just don't get installed because of a poor Gentoo Makefile. I am working on fixing it.

In the meantime I'd appreciate if you could try to track down the cause of this problem in linuxdoc-tools. Thanks!
Comment 51 Coacher 2018-07-04 20:41:15 UTC
Anon Emuss, please give a try to the ebuild in the linked PR, see 'See Also'.
Comment 52 Anon Emuss 2018-07-06 06:40:17 UTC
(In reply to Coacher from comment #50)
> The files are there in the tarball (only docsrc is missing from the
> tarball). They just don't get installed because of a poor Gentoo Makefile. I
> am working on fixing it.
> 
> In the meantime I'd appreciate if you could try to track down the cause of
> this problem in linuxdoc-tools. Thanks!

Sorry.  I was away for a bit.  I'll look into it, including your updated ebuild.
Comment 53 Anon Emuss 2018-07-10 07:34:52 UTC
(In reply to Coacher from comment #51)
> Anon Emuss, please give a try to the ebuild in the linked PR, see
> 'See Also'.
OK.

On my system, with docbook-dsssl-stylesheets-1.79-r2 installed (before using your ebuild), the file referenced in the DTDDECL line of /usr/share/sgml/docbook/dsssl-stylesheets-1.79/catalog (dtds/html/dbhtml.dcl) does not exist (as mentioned earlier).  Using a clean compile of onsgmls from opensp-1.5.2, here are the commands I have been using as test cases:
  --------
  $ nsgmls/onsgmls -c /etc/sgml/catalog -ifmttxt /tmp/demo.txt >/dev/null
  $ nsgmls/onsgmls -c /etc/sgml/catalog -ifmttxt </tmp/demo.txt >/dev/null
  nsgmls/.libs/lt-onsgmls:<OSFD>0:170:19:E: end tag for "ARTICLE" omitted, but its declaration does not permit this
  nsgmls/.libs/lt-onsgmls:<OSFD>0:3:0: start tag was here
  --------
After I update from docbook-dsssl-stylesheets-1.79-r2 to docbook-dsssl-stylesheets-1.79-r3 using the pull requests mentioned above, the file referended in the DTDDECL line of /usr/share/sgml/docbook/dsssl-stylesheets-1.79/catalog is still dtds/html/dbhtml.dcl, but now it DOES exist (thanks!).  However, this does not affect the error.  It still happens:
  --------
  $ nsgmls/onsgmls -c /etc/sgml/catalog -ifmttxt /tmp/demo.txt >/dev/null
  $ nsgmls/onsgmls -c /etc/sgml/catalog -ifmttxt </tmp/demo.txt >/dev/null
  nsgmls/.libs/lt-onsgmls:<OSFD>0:170:19:E: end tag for "ARTICLE" omitted, but its declaration does not permit this
  nsgmls/.libs/lt-onsgmls:<OSFD>0:3:0: start tag was here
  --------
I suspect any DTDDECL line, whether or not the file exists, will cause a problem like this one.  I will try to summarize my findings on that in a separate comment.
Comment 54 Anon Emuss 2018-07-10 08:03:04 UTC
I dug deeper into the opensp-1.5.2 source code to try to figure out what about linuxdoc-tools was causing the problem, and found that I was heading down the wrong track.  This appears to be an opensp-1.5.2 bug, not a linuxdoc-tools bug.  Most of what I describe below I found directly, but there are a few inferences which seem to fit quite well with the evidence so far.

I found a brief description of what DTDDECL does in /usr/share/doc/opensp-1.5.2-r3/doc/catalog.htm:
  --------
  DTDDECL <pubid> <sysid>
  This specifies that if the document does not contain an SGML
  declaration and uses a doctype declaration with public identifier
  <pubid>, the SGML declaration in <sysid> should be implied.
  --------
I used a debug build of opensp-1.5.2 to track down the differences between reading from standard input and a file given on the command-line, and it appears that onsgmls handles DTDDECL lines by repeating the initialization for each DTDDECL line it finds, presumably with the appropriate SGML declaration inserted.  It does not appear to need to read each input file in its entirety each time it initializes, but it does attempt to read the beginning of each input file (including standard input, if using that), presumably to figure out declarations, character sets, line endings, and/or other similar things.  Only after all of this does it return to the original open handle and use that to parse the entire file.  When reading from a file on the command line, it can rewind and start reading from the beginning each time, and so the entire file gets parsed.  That is not possible with a pipe such as the standard input, however, and so each new handle opened by a repeated initialization reads a block from standard input that is then lost to the original.  The program uses a storage object class to wrap each handle, and that keeps a buffer, so the initial block read by the original handle is still available when it comes time for the final parse, but any blocks read by other handles wrapped in storage objects are lost.

In comment 47, I remarked that I had a different input file that used /usr/share/sgml/sgml-dtd-4.1/ and nothing from linuxdoc-tools, and that it worked.  I took this to mean that the problem was somehow associated with linuxdoc-tools.  I looked into that again and found I was wrong.  That other input file was being truncated just like the demo.txt being used here.  The docbook declaration used in /usr/share/sgml/sgml-dtd-4.1/ does not require the ending tags to be specified, and so when they were truncated off, onsgmls implicitly assumed them and did not report an error.  Checking the output more carefully, I see from its output that it was truncating the input file.  This happens any time there is a DTDDECL line, if the file is read from standard input.  If it would be useful, I can post all of the files required for that case as an example of the bug that does not use any linuxdoc-tools files.

I checked the documentation for the opensp-1.5.2 package, and the closest I could find of any mention of this being a problem was in a list of possible ideas for improving the program, in /usr/share/doc/opensp-1.5.2-r3/doc/ideas.htm:
  --------
  Entity Manager
  ...
  Try harder to rewind in StdioStorageObject.
  --------

I tried looking for more recent versions of the opensp package to test, and it does not look like there is much development happening.  A lot of the links on the homepage do not seem to work any more.  I saw nothing newer than opensp-1.5.2.

Interestingly, I did find an old bug (for opensp-1.5) that appears to point out the same problem:
  https://sourceforge.net/p/openjade/bugs/64/
I tried using the files posted there, but they referenced a lot of other files which were not included and I do not have on my system, so it was unclear whether or not I could reproduce the problem with those files.  The last comment in the bug report asks the submitter to try it again with opensp-1.5.2, but there is no response.  That was in early 2006, but maybe pinging that with one or two sets of files that demonstrate the problem with opensp-1.5.2 would generate a response.

As a result of this, I am pretty sure this is a bug in opensp-1.5.2 which apparently was introduced somewhere between sp-1.3.4 and opensp-1.5, according to the bug report, and not a problem with linuxdoc-tools.

I have two ideas as to where to go from here, and am hoping for suggestions as to which ones are worth pursuing (if any).  I am also open to any other ideas on how to proceed.

My first idea is to fix opensp.  My first choice would be to get the attention of one of the developers, either through direct contact, opening a new bug report, or posting to the existing one.  If that fails or seems likely to fail, and nobody else reading this feels particularly qualified, I could have a go at it myself.  I do not really understand what the program does, but I have gotten an understanding of the data structures used to deal with reading inputs, and there is a decent chance I can get it to reuse the same storage object each time it rereads the file instead of trying to rewind each time.  I am not confident that I could test it particularly well seeing as how I do not understand the main program or what it is supposed to do.  However, I think I could do this in a way that it only touches some of the low-level storage object wrappers.  That would make my changes pretty much transparent to the entire program, and, if I could do that properly, it would be unlikely to introduce bugs.  Any such patch could be included with an updated ebuild for opensp-1.5.2, and/or passed to the developers to be included in a later release.

My second idea is to provide a workaround in linuxdoc-tools.  The bug only manifests when reading from a pipe such as standard input, which is how some of the Perl scripts in linuxdoc-tools are running nsgmls (called onsgmls in the opensp package).  Those scripts already use some temporary files.  I see no reason why there could not be one more intermediate temporary file to hold the input of nsgmls, and have nsgmls read from that intermediate file directly instead of being piped in through standard input.  That would avoid the bug, and should fix any package affected by it (including ulogd[doc]).  As above, if nobody else more qualified chimes in, I am willing to have a go at it.  I am much more confident in my ability to handle this than patching opensp, and should be able to get a workaround patch for linuxdoc-tools working pretty shortly (but would be sad that if the bug in opensp were never fixed).  Again, that patch could be included with an updated ebuild for linuxdoc-tools-0.9.72, and/or passed to the developers to be included in a later release.

Thanks for the help so far, Coacher.  Any more advice for me?
Comment 55 Coacher 2018-07-13 07:13:32 UTC
Anon Emuss, thanks again for the detailed info.

It looks like the opensp bug you've linked describes the problem we have here.
AFAIU, opensp upstream is dead and in Gentoo it's on life support.
CC'ing opensp maintainers to see if they're willing to tackle opensp.
If not, I'd say we workaround it in linuxdoc-tools instead.

@heroxbd, please take a look at comment 47 and below.
Comment 56 Coacher 2018-07-16 14:47:55 UTC
@heroxbd, steps in comment 48 reportedly don't trigger this bug on Debian.
Debian has a huge patch for opensp. I was unable to find a possible fix in it, but you are probably more familiar with opensp codebase so maybe you will be able to spot it.
Comment 57 Coacher 2018-07-16 15:11:24 UTC
Created attachment 539722 [details, diff]
1000_LinuxDocTools.pm_nsgmls-no-pipe.diff

Anon Emuss, could you please test the attached patch in the meantime?
Comment 58 Anon Emuss 2018-07-19 07:48:20 UTC
(In reply to Coacher from comment #57)
> Created attachment 539722 [details, diff] [details, diff]
> 1000_LinuxDocTools.pm_nsgmls-no-pipe.diff
> 
> Anon Emuss, could you please test the attached patch in the meantime?

Apologies for the delay.  I have been unavailable for the past few days.

I tried the patch, and it works.  Without the patch, ulogd[doc] fails with the message about missing SGML tags.  If I then re-install linuxdoc-tools with your patch, ulogd[doc] installs fine.

Thanks for putting that together.
Comment 59 Anon Emuss 2018-07-19 07:55:10 UTC
(In reply to Coacher from comment #56)
> @heroxbd, steps in comment 48 reportedly don't trigger this bug on Debian.
> Debian has a huge patch for opensp. I was unable to find a possible fix in
> it, but you are probably more familiar with opensp codebase so maybe you
> will be able to spot it.
I tracked down the problem in the opensp source and started trying to patch it before I had to take a pause to work on other things.  If Debian has fixed the issue, they probably have tested it much better than I could, but I think my work would help me figure out whether or not they have a possible fix in there.  If you could point me towards this huge patch for opensp, I could take a look.
Comment 60 Coacher 2018-07-19 22:17:44 UTC
(In reply to Anon Emuss from comment #59)
> I tracked down the problem in the opensp source and started trying to patch
> it before I had to take a pause to work on other things.  If Debian has
> fixed the issue, they probably have tested it much better than I could, but
> I think my work would help me figure out whether or not they have a possible
> fix in there.  If you could point me towards this huge patch for opensp, I
> could take a look.

http://http.debian.net/debian/pool/main/o/opensp/opensp_1.5.2-13.diff.gz
Comment 61 Coacher 2018-07-22 20:19:45 UTC
(In reply to Coacher from comment #60)
> (In reply to Anon Emuss from comment #59)
> > I tracked down the problem in the opensp source and started trying to patch
> > it before I had to take a pause to work on other things.  If Debian has
> > fixed the issue, they probably have tested it much better than I could, but
> > I think my work would help me figure out whether or not they have a possible
> > fix in there.  If you could point me towards this huge patch for opensp, I
> > could take a look.
> 
> http://http.debian.net/debian/pool/main/o/opensp/opensp_1.5.2-13.diff.gz
Also Debian apparently builds opensp with `--disable-dtddecl'.
This might be the key.
Comment 62 Anon Emuss 2018-07-23 07:34:59 UTC
(In reply to Coacher from comment #61)
>...
> Also Debian apparently builds opensp with `--disable-dtddecl'.
> This might be the key.
You beat me to it.  I agree.  I do not see a fix in the patch.  I checked the whole patch, and there are very few code modifications, and they seem to mostly deal with patching a small memory leak.  I even was able to reproduce the problem with the patch installed.

Instead of a patch, they have the --disable-dtddecl workaround.  In debian/rules (created by the patch), configure is called with the --disable-dtddecl option, which disables the DTDDECL handling, circumventing the bug.  In debian/changelog (also created by the patch), this is described as a fix for Debian bugs #138924 and #208042, described here:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=138924
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=208042
It seems that in addition to causing problems reading from standard inputs, the DTDDECL handling can also cause quite a slowdown, which is why Debian developed it.  This supports my theory from comment 54 that onsgml performs its entire initialization once for every DTDDECL it finds on the first initialization.

I can confirm that configuring with --disable-dtdecl works around the problem even without the Debian patch.

I could continue trying to patch opensl to fix the standard input issue, but that would not fix the slowdown issue.  Trying to handle that would be much more of an investment than I think I would be up for.  I am not sure what the DTDDECL really does, except that it looks like some sort of default description if none is provided in the input document.  If Debian thought it was worth disabling, maybe that would work for Gentoo as well?
Comment 63 Coacher 2018-07-23 19:00:42 UTC
linuxdoc-tools upstream pushed a workaround for this into master.
I'll create a new revision with this patch in a couple of days.

As for opensp, I think the situation deserves a separate bug,
which I am planning to open sometime later if I don't forget.
Comment 64 Larry the Git Cow gentoo-dev 2018-07-26 08:35:52 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f3a064e1a7d2f6f526ebdff215a981387a7d652d

commit f3a064e1a7d2f6f526ebdff215a981387a7d652d
Author:     Ilya Tumaykin <itumaykin@gmail.com>
AuthorDate: 2018-07-04 19:36:20 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2018-07-26 08:35:09 +0000

    app-text/docbook-dsssl-stylesheets: install missing files, cleanup
    
    Due to Gentoo-originated poorly written Makefile not all required
    files were installed, which caused tools looking for them to fail,
    e.g. Gentoo bug 606768. Also one file that belong to another package
    was installed: dtds/decls/docbook.dcl is a part of docbook-sgml-dtd.
    Replace custom Makefile with a series of doins commands.
    
    Also remove dirty, ugly symlink hack to workaround Gentoo bug 15026.
    Current stable libgcrypt doesn't use docbook-dssl-stylesheets for docs.
    
    Bug: https://bugs.gentoo.org/606768
    Bug: https://bugs.gentoo.org/15026
    Package-Manager: Portage-2.3.41, Repoman-2.3.9

 .../docbook-dsssl-stylesheets-1.79-r3.ebuild       | 65 ++++++++++++++++++++++
 1 file changed, 65 insertions(+)
Comment 65 Larry the Git Cow gentoo-dev 2018-10-05 21:28:29 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6725a37b5c4eceb45fa398ab087b98f1107faec5

commit 6725a37b5c4eceb45fa398ab087b98f1107faec5
Author:     Ilya Tumaykin <itumaykin@gmail.com>
AuthorDate: 2018-07-25 19:26:11 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2018-10-05 21:28:19 +0000

    app-text/linuxdoc-tools: verbump to 0.9.73
    
    Among other things uses nsgmls without a pipe, which fixes bug 606768.
    
    Bug: https://bugs.gentoo.org/606768
    Signed-off-by: Ilya Tumaykin <itumaykin@gmail.com>
    Package-Manager: Portage-2.3.50, Repoman-2.3.11
    Signed-off-by: Michał Górny <mgorny@gentoo.org>
    Closes: https://github.com/gentoo/gentoo/pull/9350

 app-text/linuxdoc-tools/Manifest                   |  1 +
 .../linuxdoc-tools/linuxdoc-tools-0.9.73.ebuild    | 74 ++++++++++++++++++++++
 2 files changed, 75 insertions(+)
Comment 66 Coacher 2018-10-07 10:09:56 UTC
*** Bug 625360 has been marked as a duplicate of this bug. ***