Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 706252 - dev-db/mongodb-4.2.2 with dev-cpp/yaml-cpp-0.6.3-r1 ignores --config file passed
Summary: dev-db/mongodb-4.2.2 with dev-cpp/yaml-cpp-0.6.3-r1 ignores --config file passed
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Ultrabug
URL: https://jira.mongodb.org/browse/SERVE...
Whiteboard:
Keywords:
: 728688 (view as bug list)
Depends on:
Blocks: 713464
  Show dependency tree
 
Reported: 2020-01-24 17:01 UTC by Ben Kohler
Modified: 2020-06-29 04:56 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ben Kohler gentoo-dev 2020-01-24 17:01:41 UTC
I have found that mongodb-4.2.2 running with yaml-cpp-0.6.3-r1 seems unable or unwilling to use config options from a YAML config passed on command-line with --config/-f.

This showed up on my ~amd64 home server but I've been able to reproduce it easily on a fresh amd64 chroot, allowing ~amd64 for mongodb, mongo-tools, and yaml-cpp.  On this chroot test case, if I downgrade yaml-cpp to 0.6.2, the problem goes away.

To reproduce, here is a mongodb YAML config file, test.conf:

storage:
   dbPath: "/root/db-test"
   journal:
      enabled: true
   wiredTiger:
      engineConfig:
         cacheSizeGB: 1

systemLog:
   destination: file
   logAppend: true
   path: /root/log-test/mongod.log

net:
   port: 7441
   bindIp: 127.0.0.1


And here is the result when run with yaml-cpp0-.6.3:
# mongod -f test.conf
2020-01-24T11:00:01.216-0600 I  CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2020-01-24T11:00:01.217-0600 I  CONTROL  [initandlisten] MongoDB starting : pid=51 port=27017 dbpath=/data/db 64-bit host=64bit-stable
2020-01-24T11:00:01.217-0600 I  CONTROL  [initandlisten] db version v4.2.2
2020-01-24T11:00:01.217-0600 I  CONTROL  [initandlisten] git version: a0bbbff6ada159e19298d37946ac8dc4b497eadf
2020-01-24T11:00:01.217-0600 I  CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.1.1d  10 Sep 2019
2020-01-24T11:00:01.217-0600 I  CONTROL  [initandlisten] allocator: tcmalloc
2020-01-24T11:00:01.217-0600 I  CONTROL  [initandlisten] modules: none
2020-01-24T11:00:01.217-0600 I  CONTROL  [initandlisten] build environment:
2020-01-24T11:00:01.217-0600 I  CONTROL  [initandlisten]     distarch: x86_64
2020-01-24T11:00:01.217-0600 I  CONTROL  [initandlisten]     target_arch: x86_64
2020-01-24T11:00:01.217-0600 I  CONTROL  [initandlisten] options: { config: "test.conf" }
2020-01-24T11:00:01.218-0600 I  STORAGE  [initandlisten] exception in initAndListen: NonExistentPath: Data directory /data/db not found., terminating
2020-01-24T11:00:01.218-0600 I  NETWORK  [initandlisten] shutdown: going to close listening sockets...
2020-01-24T11:00:01.218-0600 I  NETWORK  [initandlisten] removing socket file: /tmp/mongodb-27017.sock
2020-01-24T11:00:01.218-0600 I  -        [initandlisten] Stopping further Flow Control ticket acquisitions.
2020-01-24T11:00:01.218-0600 I  CONTROL  [initandlisten] now exiting
2020-01-24T11:00:01.218-0600 I  CONTROL  [initandlisten] shutting down with code:100
#
Comment 1 Ben Kohler gentoo-dev 2020-01-24 17:04:31 UTC
In the output you can see it does know about the config:
2020-01-24T11:00:01.217-0600 I  CONTROL  [initandlisten] options: { config: "test.conf" }

But obviously it's not using the config's db path, port, bind_ip and more.

If I downgrade yaml-cpp to 0.6.2, it works as expected.

If I pass the individual options explicitly on cmdline (--port 7441 --bind_ip 127.0.0.1 etc) they do work, so it seems to just be an issue reading the on-disk YAML config file.
Comment 2 Ben Kohler gentoo-dev 2020-01-24 17:25:18 UTC
Possibly related upstream report: https://jira.mongodb.org/browse/SERVER-43980
Comment 3 Ultrabug gentoo-dev 2020-05-20 13:00:32 UTC
Hi Ben

Do you have this problem with yaml-cpp-0.6.3-r2 ?

+PATCHES=(
+	"${FILESDIR}/${P}-abi-breakage.patch"
+	"${FILESDIR}/${P}-CVE-2017-11692.patch"
+)

Since those two patches got added, I wonder...
Comment 4 Ben Kohler gentoo-dev 2020-05-29 11:58:45 UTC
This does seem to be working better now! I haven't looked into the patches in detail but my problem is gone
Comment 5 Sam James archtester gentoo-dev Security 2020-06-12 00:35:34 UTC
Reopen if it's a problem?
Comment 6 Sam James archtester gentoo-dev Security 2020-06-18 19:46:26 UTC
*** Bug 728688 has been marked as a duplicate of this bug. ***
Comment 7 Sam James archtester gentoo-dev Security 2020-06-18 19:46:50 UTC
I think this is still an issue: https://bugs.gentoo.org/728688#c0.
Comment 8 Ralph Seichter 2020-06-18 19:50:29 UTC
Yeah, this bug caused quite a headache for me today, with MongoDB 4.2.6. My current workaround is moving all settings found in /etc/mongodb.conf into an array "rs_opt" in /etc/conf.d/mongodb, and then pass them on in /etc/init.d/mongodb:

    # Original:
    #command_args="--config ${config_file}"

    # Workaround:
    command_args="${rs_opt[@]}"

It works, but given the number of required options, it is a major PITA.
Comment 9 Ralph Seichter 2020-06-27 07:37:19 UTC
This is still broken with dev-db/mongodb-4.2.8. I suggest updating the bug description accordingly. Also, this warrants "importance: high", because the recommended (by Upstream) way of configuring MongoDB is clearly broken for Gentoo at this time.
Comment 10 Ben Kohler gentoo-dev 2020-06-27 12:14:56 UTC
I'm not able to reproduce this bug anymore though.
---------------------
dev-cpp/yaml-cpp-0.6.3-r2
dev-db/mongodb-4.2.6
---------------------
test.conf:
---------------------
storage:
   dbPath: "/root/db-test"
   journal:
      enabled: true
   wiredTiger:
      engineConfig:
         cacheSizeGB: 1

systemLog:
   destination: file
   logAppend: true
   path: /root/mongod.log

net:
   port: 1234
   bindIp: 127.0.0.1
---------------------
# mongod -f test.conf
---------------------
mongodb.log:
---------------------
2020-06-27T07:12:43.966-0500 I  CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2020-06-27T07:12:43.979-0500 W  ASIO     [main] No TransportLayer configured during NetworkInterface startup
2020-06-27T07:12:43.991-0500 I  CONTROL  [initandlisten] MongoDB starting : pid=463837 port=1234 dbpath=/root/db-test 64-bit host=fserv
2020-06-27T07:12:43.991-0500 I  CONTROL  [initandlisten] db version v4.2.6
2020-06-27T07:12:43.991-0500 I  CONTROL  [initandlisten] git version: 20364840b8f1af16917e4c23c1b5f5efd8b352f8
2020-06-27T07:12:43.991-0500 I  CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.1.1g  21 Apr 2020
2020-06-27T07:12:43.991-0500 I  CONTROL  [initandlisten] allocator: tcmalloc
2020-06-27T07:12:43.992-0500 I  CONTROL  [initandlisten] modules: none
2020-06-27T07:12:43.992-0500 I  CONTROL  [initandlisten] build environment:
2020-06-27T07:12:43.992-0500 I  CONTROL  [initandlisten]     distarch: x86_64
2020-06-27T07:12:43.992-0500 I  CONTROL  [initandlisten]     target_arch: x86_64
2020-06-27T07:12:43.992-0500 I  CONTROL  [initandlisten] options: { config: "test.conf", net: { bindIp: "127.0.0.1", port: 1234 }, storage: { dbPath: "/root/db-test", journal: { enabled: true }, wiredTiger: { engineConfig: { cacheSizeGB: 1.0 } } }, systemLog: { destination: "file", logAppend: true, path: "/root/mongod.log" } }
2020-06-27T07:12:44.013-0500 I  STORAGE  [initandlisten] Detected data files in /root/db-test created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2020-06-27T07:12:44.013-0500 I  STORAGE  [initandlisten]
2020-06-27T07:12:44.013-0500 I  STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2020-06-27T07:12:44.013-0500 I  STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2020-06-27T07:12:44.013-0500 I  STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1024M,cache_overflow=(file_max=0M),session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress],
2020-06-27T07:12:45.498-0500 I  STORAGE  [initandlisten] WiredTiger message [1593259965:498705][463837:0x7f0567edab00], txn-recover: Recovering log 1 through 2
2020-06-27T07:12:45.722-0500 I  STORAGE  [initandlisten] WiredTiger message [1593259965:722921][463837:0x7f0567edab00], txn-recover: Recovering log 2 through 2
2020-06-27T07:12:45.990-0500 I  STORAGE  [initandlisten] WiredTiger message [1593259965:990131][463837:0x7f0567edab00], txn-recover: Main recovery loop: starting at 1/24576 to 2/256
2020-06-27T07:12:46.116-0500 I  STORAGE  [initandlisten] WiredTiger message [1593259966:116973][463837:0x7f0567edab00], txn-recover: Recovering log 1 through 2
2020-06-27T07:12:46.196-0500 I  STORAGE  [initandlisten] WiredTiger message [1593259966:196467][463837:0x7f0567edab00], txn-recover: Recovering log 2 through 2
2020-06-27T07:12:46.268-0500 I  STORAGE  [initandlisten] WiredTiger message [1593259966:268806][463837:0x7f0567edab00], txn-recover: Set global recovery timestamp: (0, 0)
2020-06-27T07:12:46.306-0500 I  RECOVERY [initandlisten] WiredTiger recoveryTimestamp. Ts: Timestamp(0, 0)
2020-06-27T07:12:46.324-0500 I  STORAGE  [initandlisten] Timestamp monitor starting
2020-06-27T07:12:46.332-0500 I  CONTROL  [initandlisten]
2020-06-27T07:12:46.333-0500 I  CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2020-06-27T07:12:46.333-0500 I  CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2020-06-27T07:12:46.333-0500 I  CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2020-06-27T07:12:46.333-0500 I  CONTROL  [initandlisten]
2020-06-27T07:12:46.359-0500 I  SHARDING [initandlisten] Marking collection local.system.replset as collection version: <unsharded>
2020-06-27T07:12:46.362-0500 I  STORAGE  [initandlisten] Flow Control is enabled on this deployment.
2020-06-27T07:12:46.362-0500 I  SHARDING [initandlisten] Marking collection admin.system.roles as collection version: <unsharded>
2020-06-27T07:12:46.362-0500 I  SHARDING [initandlisten] Marking collection admin.system.version as collection version: <unsharded>
2020-06-27T07:12:46.364-0500 I  SHARDING [initandlisten] Marking collection local.startup_log as collection version: <unsharded>
2020-06-27T07:12:46.365-0500 I  FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/root/db-test/diagnostic.data'
2020-06-27T07:12:46.419-0500 I  SHARDING [LogicalSessionCacheRefresh] Marking collection config.system.sessions as collection version: <unsharded>
2020-06-27T07:12:46.446-0500 I  SHARDING [LogicalSessionCacheReap] Marking collection config.transactions as collection version: <unsharded>
2020-06-27T07:12:46.465-0500 I  NETWORK  [listener] Listening on /tmp/mongodb-1234.sock
2020-06-27T07:12:46.465-0500 I  NETWORK  [listener] Listening on 127.0.0.1
2020-06-27T07:12:46.465-0500 I  NETWORK  [listener] waiting for connections on port 1234
2020-06-27T07:12:47.000-0500 I  SHARDING [ftdc] Marking collection local.oplog.rs as collection version: <unsharded>
Comment 11 Ben Kohler gentoo-dev 2020-06-27 12:26:54 UTC
Previously it completely ignored my config settings:

2020-01-24T11:00:01.217-0600 I  CONTROL  [initandlisten] MongoDB starting : pid=51 port=27017 dbpath=/data/db 64-bit host=64bit-stable

Now it does not:

2020-06-27T07:12:43.991-0500 I  CONTROL  [initandlisten] MongoDB starting : pid=463837 port=1234 dbpath=/root/db-test 64-bit host=fserv
Comment 12 Ralph Seichter 2020-06-27 12:34:41 UTC
This is weird. I have updated a second DB server to 4.2.8, and this one reads the YAML file.

    # equery l dev-db/mongodb
     * Searching for mongodb in dev-db ...
    [IP-] [  ] dev-db/mongodb-4.2.8:0

    # equery l dev-cpp/yaml-cpp
     * Searching for yaml-cpp in dev-cpp ...
    [IP-] [  ] dev-cpp/yaml-cpp-0.6.3-r2:0/0.6
Comment 13 Ben Kohler gentoo-dev 2020-06-27 12:49:29 UTC
The bug is in yaml-cpp, it would affect all mongodb versions.  What version of yaml-cpp is installed on the server where it does not work?
Comment 14 Ralph Seichter 2020-06-27 14:47:47 UTC
I have since gone through @world updates on all three members of a replica set, and now each of the servers loads the configuration file as expected. I am glad it worked out this way, although I don't know what caused the first server to fail after the upgrade earlier today.
Comment 15 Tomáš Mózes 2020-06-29 04:56:40 UTC
Rebuilding yaml-cpp and then mongodb made it working on my machines. Just upgrading yaml-cpp resulted in mongodb segfaulting :(