In a previous post I described how to install Logstash (v. 1.3 and previous) on FreeBSD and in this post I will describe how to install Logstash v. 1.4 and greater.
Until version 1.3 included, Logstash was distributed as a single JAR file, and when version 1.4 was released a new packaging style was introduced. As a consequence, new instructions are required to properly setup Logstash in FreeBSD and registering it as a service. Further information about the new distribution layout can be found in the Logstash release notes.
As seen in the previous post, a Logstash FreeBSD port exists, but it is currently outdated since it bundles Logstash v. 1.2. But while this could be used as a starting point for JAR-based Logstash installations (as we have seen, the update process only required updating the Logstash JAR), this is not possible with the new Logstash distribution because the included rc script will fail to work.
Skimming through the original post is recommended because it provides general information about Logstash and FreeBSD which is required to properly plan and execute a Logstash setup.
Prerequisites
The essential prerequisites required to execute Logstash are:
- A working Java runtime environment.
- An ElasticSearch instance.
The former is required because LogStash is a JRuby application while the latter, although not technically a requirement, is the recommended output for Logstash.
Installing Java
To install OpenJDK on FreeBSD you can use pkg to install a ready-to-use binary package:
1 | # pkg install openjdk |
Currently, this command will install an instance of OpenJDK v. 7 in both FreeBSD 9 and 10. If you’d rather install a different version, you can search the available packages and pick the one you prefer (command output has been filtered for brevity):
1 2 3 4 5 | # pkg search openjdkopenjdk-7.60.19,1openjdk6-b31_3,1openjdk8-8.5.13_7# pkg install openjdk8-8.5.13_7 |
Installing ElasticSearch
Logstash includes an embedded ElasticSearch instance you can use for standalone installations (see my previous post for an introductory view on Logstash operation modes). The required configuration to bootstrap the embedded ElasticSearch instance and to have Logstash use it as its outputs is described in the following sections.
Although simpler from the standpoint of the configuration, Logstash installations using separate ElasticSearch instances are out of the scope of this post.
Installing Logstash
Logstash installation procedure is fairly simple since it is distributed as a tarball:
- Download Logstash from the official website.
- Extract the tarball in the designated installation directory (my personal suggestion is to avoid /usr/local because it is used by ports and to use /opt instead).
Creating an rc.d Script
An rc.d script is required in a BSD system to register a service, define its configuration and have the rc framework manage its lifetime. The following script can be used as is or as a starting point to customise your own. If used as is, be aware that the script uses the following default values:
- Installation directory: ${logstash14_home=”/opt/logstash-1.4.1″}
- Configuration file path: ${logstash14_config=”/usr/local/etc/${name}/${name}.conf”}
- ElasticSearch data directory: ${logstash14_elastic_datadir=”/var/db/logstash14″}
- Java home: ${logstash14_java_home=”/usr/local/openjdk6″}
001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 | #!/bin/sh# Configuration settings for Logstash in /etc/rc.conf:## logstash14_enable (bool):# Default value: "NO"# Flag that determines whether Logstash is enabled.## logstash14_home (string):# Default value: "/opt/logstash-1.4.1"# Logstash installation directory.## logstash14_config (string):# Default value: /usr/local/etc/${name}/${name}.conf# Logstash configuration file path.## logstash14_mode (string):# Default value: "standalone"# Valid options:# "standalone": agent, web & elasticsearch# "web": Starts logstash as a web ui# "agent": Justs works as a log shipper ## logstash14_port (int):# Default value: 9292# Port of the Kibana web interface.# # logstash14_log (bool):# Set to "NO" by default.# Set it to "YES" to enable logstash logging to file# Default output to /var/log/logstash.log## logstash14_log_file (string):# Default value: "${logdir}/${name}.log"# Log file path.## logstash14_java_home (string):# Default value: "/usr/local/openjdk6"# Root directory of the desired Java SDK.# The JAVA_HOME environment variable is set with the contents of this# variable.## logstash14_java_opts (string):# Default value: ""# Options to pass to the Java Virtual Machine.# The JAVA_OPTS environment variable is set with the contents of this# variable.## logstash14_elastic_datadir (string):# Default value: "/var/db/logstash14".# Data directory of the embedded ElasticSearch instance.#. /etc/rc.subrname=logstash14rcvar=logstash14_enableload_rc_config ${name}logdir="/var/log": ${logstash14_enable="NO"}: ${logstash14_home="/opt/logstash-1.4.1"}: ${logstash14_config="/usr/local/etc/${name}/${name}.conf"}: ${logstash14_log="NO"}: ${logstash14_mode="standalone"}: ${logstash14_port="9292"}: ${logstash14_log_file="${logdir}/${name}.log"}: ${logstash14_elastic_datadir="/var/db/logstash14"}: ${logstash14_java_home="/usr/local/openjdk6"}: ${logstash14_java_opts=""}piddir=/var/run/${name}pidfile=${piddir}/${name}.pidif [ -d $piddir ]; then mkdir -p $piddirfilogstash14_cmd="${logstash14_home}/bin/logstash"procname="${logstash14_java_home}/bin/java"logstash14_chdir=${logstash14_home}logstash14_log_options=""logstash14_elastic_options=""if checkyesno logstash14_log; then logstash14_log_options=" --log ${logstash14_log_file}"fiif [ ${logstash14_mode} = "standalone" ]; then logstash14_args="agent -f ${logstash14_config} ${logstash14_log_options} -- web --port ${logstash14_port}" logstash14_elastic_options="-Des.path.data=${logstash14_elastic_datadir}"elif [ ${logstash14_mode} = "agent" ]; then logstash14_args="agent -f ${logstash14_config} ${logstash14_log_options}"elif [ ${logstash14_mode} = "web" ]; then logstash14_args="web --port ${logstash14_port} ${logstash14_log_options}"fiJAVA_OPTS="${logstash14_java_opts} ${logstash14_elastic_options}"JAVA_HOME="${logstash14_java_home}"export JAVA_OPTSexport JAVA_HOMEcommand="/usr/sbin/daemon"command_args="-f -p ${pidfile} ${logstash14_cmd} ${logstash14_args}"required_files="${logstash14_home} ${logstash14_java_home} ${logstash14_cmd} ${logstash14_config}"run_rc_command "$1" |
You can override any of the supported configuration values in the /etc/rc.conf file. If, for example, you want to use an alternate Java home path, just add the following line to /etc/rc.conf setting the desired value:
1 | logstash14_java_home="/usr/local/openjdk7" |
Testing the Service
To test the Logstash service, the following command can be used:
1 | # service logstash14 onestart |
To stop it, use:
1 | # service logstash14 onestop |
To help troubleshooting any problem you might find you can enable the Logstash log, setting the logstash14_log variable to YES in the /etc/rc.conf file:
1 | logstash14_log="YES" |
The log file location is specified by the logstash14_log_file variable, whose default value is set by the service rc file (only the relevant lines are shown):
1 2 3 | name=logstash14logdir="/var/log": ${logstash14_log_file="${logdir}/${name}.log"} |
The log file location can be overridden setting the logstash14_log_file variable in the /etc/rc.conf file.
Enabling the Service
Note that the rc script described above does not enable the Logstash service:
1 | : ${logstash14_enable="NO"} |
If everything works, you can enable the Logstash service just adding the following line to /etc/rc.conf:
1 | logstash14_enable="YES" |
| Reference: | Installing Logstash v 1.4 (and Greater) on FreeBSD from our JCG partner Enrico Crisostomo at the The Grey Blog blog. |
