Using Stream Doctor

Stream doctor is a tool that performs checks on your Stream installation as well as its dependencies to ensure that everything is configured properly. Note that the tool requires root permissions to run.

Checks performed

At the moment, Stream doctor checks for :

OS checks

  • Checks for installed Stream version, MongoDB version, Java version, Nginx Version and OS version.

    • If the OS is a RedHat distribution, checks for RedHat subscription

    • If Mongo is not installed locally, it notices it as an information log

  • Checks for SELinux's configuration (throws a warning if SELinux is enabled)

  • Checks for the status of the necessary services: mongod, nginx and stream.

  • Checks how long the stream service has been running for.

  • Checks if there is an NTP service active on the machine and checks if the system clock is synchronized with the NTP service.

Config checks

  • Checks for existence and permissions of the configuration file: the permissions are expected to be at least 640 and the file is supposed to belong to stream:stream.

  • Checks for existence and permissions of the licence file: the permissions are expected to be at least 640 and the file is supposed to belong to stream:stream.

  • Checks for existence and permissions of the keyset file: the permissions are expected to be exactly 600 and the file is supposed to belong to stream:stream.

  • Checks for existence and permissions of the Stream directory (default : /opt/stream) : the permission is expected to be at least 755

  • Checks for the existence of the symbolic link for nginx configuration and runs an nginx -t test.

  • Retrieves the Java heap size parameters that were set for Stream and informs the user if the default ones are used (min = 2048 and max = 3072).

  • Retrieves the Stream DNS hostname and raises an error if it has not been set.

  • Retrieves the MongoDB URI (throws a warning if MongoDB is running on localhost; throws an error if MongoDB is running on an external instance but the authSource=admin parameter is missing from the URI).

  • Parses the licence file to retrieve its expiration date.

Network checks

  • Runs a MongoDB ping on the URI, then checks for the database used in the URI (throws a warning if the database used is not called stream; throws an error if no database is specified in the URI).

  • Checks for AKKA High Availability settings: if no node hostname is set up, skips the remaining HA checks. If 2 nodes are set up, retrieves which node is running the doctor and checks for the other node. If 3 nodes are set up, retrieves which node is running the doctor and checks for the other 2 nodes. The check runs as:

    • if curl is installed, runs a curl request on the Node hostname at alive on the management port (default is 8558), and if alive runs another curl request on the Node hostname at /ready on the management port. Both requests should return HTTP/200 if ok, 000 otherwise.

    • if curl is not installed, uses the built-in Linux TCP socket to run TCP SYN checks on both the HA communication port (default is 25520) and the management port (default is 8558) on the Node hostname.

  • Checks for firewall configuration. Currently only supports firewalld (RHEL) and a netstat test.

    • The netstat part will run a netstat command to check if the JVM listening socket is active (listening on port 9000). If netstat is not installed, it will skip this test.

    • The firewalld part will check if the HTTP and HTTPS services are opened in the firewall and if it detected a HA configuration, it will check if the HA ports (both of them) are allowed through the firewalld. If firewalld is not installed or not active, it will skip this test.

  • Checks if IPv6 is active on each network interface and raises a warning if it is the case (with the interface name).

TLS checks

  • Checks for existence and permissions of the Stream server certificate file: the permissions are expected to be at least 640 and the file is supposed to belong to the nginx group.

  • Parses the Stream server certificate file: it should be constituted of the actual TLS server certificate first, then of every certificate of the trust chain (order being leaf to root). It throws a warning if the certificate is self-signed or raises an error if the trust chain has not been imported. It otherwise tries to reconstitute the certificate trust chain via the openssl verify command, and throws an error if it cannot.

  • Parses the Stream server certificate file and checks if the Stream hostname is present in the SAN DNS names of the certificate, throws an error if it is not there.

Log packing option

If the Stream doctor is launched with the -l option, it will pack the logs of the last 7 days (in /opt/stream/var/log) as well as the startup logs (the /var/log/stream/stream.log file) and create a tar archive.

The -l option accepts an optional parameter that should be an integer (1-99) and will pack the logs of the last n days instead, as well as the startup logs.

Note that the Stream doctor will still perform all of its check; the log packing is done at the very end of the program.

Example of call to pack the logs of the last 7 days :

# stream-doctor -l

Example of call to pack the logs of the last 30 days :

# stream-doctor -l 30

Saving the doctor’s output

If the Stream doctor is launched with the -o option, it will perform all of its checks and save the output in the specified file instead of displaying it into the stdout (default is the commandline interface).

If you use the option, you must provide a filepath in a writable directory.

Example of call to save the output in a file named stream-doctor.out instead of the stdout :

# stream-doctor -o stream-doctor.out

Direct fixes

The Stream doctor is able to fix the following issues directly by itself if you use the --fix flag with the script:

  • If the application secrets (play secret and event seal secret) have not been changed, the doctor will generate random application secrets and provide them to Stream directly (requires you to manually restart Stream afterwards);

  • If firewalld is not allowing HTTP and HTTPS traffic, the doctor will change the firewall settings to allow both protocols and then restart the firewall by itself;

  • If some permissions for the configuration file, the license file or the keyset file are not what they should be, the doctor will change these permissions (file owner and rwx permissions) to be what they should.

Help menu

To display Stream doctor’s help menu, use the -h option.