Home / How To / Install and configure Graylog Monitoring Server Ubuntu 20.04

Install and configure Graylog Monitoring Server Ubuntu 20.04



Graylog is a free open source log management tool that can be used to monitor network system logs from the central server. It uses Elasticsearch to store log data and provide search features and MongoDB to store metadata. It helps you monitor, search and analyze a large amount of data in an easy-to-read format.

In this tutorial we will show you how to install Graylog on the Ubuntu 20.04 server.

Conditions

  • A server running Ubuntu 20.04. With at least 4 GB of RAM
  • A root password is configured on.

Getting Started

First you need to update your system packages to the latest version. You can update them all with the following command:

apt-get update -y

After updating all packages, you will also need to install some dependencies on your server. You can install them all with the following command:

apt-get install apt-transport-https gnupg2 uuid-runtime pwgen curl dirmngr -y

Once all the necessary dependencies are installed, you can proceed to the next step.

Install Java

Graylog requires Java to be installed on your server. If it is not installed, you can install it with the following command:

apt-get install openjdk-11-jre-headless -y

Once Java is installed, you can verify the installed version of Java by running the following command:

java -version

You should get the following output:

openjdk version "11.0.8" 2020-07-14
OpenJDK Runtime Environment (build 11.0.8+10-post-Ubuntu-0ubuntu120.04)
OpenJDK 64-Bit Server VM (build 11.0.8+10-post-Ubuntu-0ubuntu120.04, mixed mode, sharing)

When you are done, you can move on to the next step.

Install and configure Elasticsearch

Graylog uses Elasticsearch to store logs from the external resource. So you need to install Elasticsearch in your system.

By default, the latest version of Elasticsearch is not available in the standard Ubuntu repository. So you need to add Elasticsearch archives to your system.

Download first and add the Elasticsearch GPG key with the following command:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -

Then add the Elasticsearch archive with the following command:

echo "deb https://artifacts.elastic.co/packages/oss-6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list

Then update the repository and install Elasticsearch with the following command:

apt-get update -y
apt-get install elasticsearch-oss -y

After installing Elasticsearch, you will need to edit the Elasticsearch configuration file and define the cluster name. You can do this with the following command:

nano /etc/elasticsearch/elasticsearch.yml

Define your cluster name for the gray log and add another line as below:

cluster.name: graylog
action.auto_create_index: false

Save and close the file when you are done. Then start the Elasticsearch service and activate it at startup with the following command:

systemctl daemon-reload
systemctl start elasticsearch
systemctl enable elasticsearch

You can also verify the status of the Elasticsearch service with the following command:

systemctl status elasticsearch

You should get the following output:

? elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; disabled; vendor preset: enabled)
     Active: active (running) since Sat 2020-09-05 08:41:18 UTC; 9s ago
       Docs: http://www.elastic.co
   Main PID: 7085 (java)
      Tasks: 17 (limit: 2353)
     Memory: 1.1G
     CGroup: /system.slice/elasticsearch.service
             ??7085 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly ->

Sep 05 08:41:18 ubuntu2004 systemd[1]: Started Elasticsearch.

Now verify the Elasticcsearch response with the following command:

curl -X GET http://localhost:9200

You should get the following output:

{
  "name" : "vzg8H4j",
  "cluster_name" : "graylog",
  "cluster_uuid" : "6R9SlXxNSUGe6aclcJa9VQ",
  "version" : {
    "number" : "6.8.12",
    "build_flavor" : "oss",
    "build_type" : "deb",
    "build_hash" : "7a15d2a",
    "build_date" : "2020-08-12T07:27:20.804867Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.3",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

Install MongoDB Server

Graylog uses MongoDB as a database. So you need to install the MongoDB database on your server. You can install it with the following command:

apt-get install mongodb-server -y

Once MongoDB has been installed, start the MongoDB service and allow it to restart at system startup with the following command:

systemctl start mongodb
systemctl enable mongodb

When you are done, you can move on to the next step.

Install and configure Graylog

By default, the Graylog package is not available in the standard Ubuntu repository. So you need to install the Graylog archive on your server.Advertisement

Advertisement

You can download the Graylog database package with the following command:

wget https://packages.graylog2.org/repo/packages/graylog-3.3-repository_latest.deb

When the download is complete, install the downloaded package with the following command:

dpkg -i graylog-3.3-repository_latest.deb

Then update the repository and install the Graylog server with the following command:

apt-get update -y
apt-get install graylog-server -y

After installing the Graylog server, you must create a secret to secure the user passwords. You can create it with the following command:

pwgen -N 1 -s 96

You should see the following output:

Wv4VQWCAA9sRbL7pxPeY7tb9lSo50esEWgNXxXHypx0Og3CezMmQLdF2QzQdRSIXmNXKINjRvZpPTrvZv4k4NlJrFYTfOc3c

Next, you must also create a secure password for the Graylog Administrator user. You need this password to log in to Graylog’s web interface. You can create it with the following command:

echo -n password | sha256sum

You should see the following output:

5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8  -

Now edit Graylog’s main configuration file and define both passwords:

nano /etc/graylog/server/server.conf

Paste both passwords that you have generated above as below:

password_secret = Wv4VQWCAA9sRbL7pxPeY7tb9lSo50esEWgNXxXHypx0Og3CezMmQLdF2QzQdRSIXmNXKINjRvZpPTrvZv4k4NlJrFYTfOc3c
root_password_sha2 = 5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8   

Next, you must also define a binding address for your server as shown below:Advertisement

http_bind_address = 127.0.0.1:9000

Save and close the file when you are done, then start the Graylog service and allow it to restart at system startup with the following command:

systemctl daemon-reload
systemctl start graylog-server
systemctl enable graylog-server

You can then verify the status of the Graylog server using the following command:

systemctl status graylog-server

You should see the following output:

? graylog-server.service - Graylog server
     Loaded: loaded (/lib/systemd/system/graylog-server.service; disabled; vendor preset: enabled)
     Active: active (running) since Sat 2020-09-05 08:50:16 UTC; 15min ago
       Docs: http://docs.graylog.org/
   Main PID: 8693 (graylog-server)
      Tasks: 156 (limit: 2353)
     Memory: 865.0M
     CGroup: /system.slice/graylog-server.service
             ??8693 /bin/sh /usr/share/graylog-server/bin/graylog-server
             ??8726 /usr/bin/java -Xms1g -Xmx1g -XX:NewRatio=1 -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX>

Sep 05 08:50:16 ubuntu2004 systemd[1]: Started Graylog server.

You can also verify the Graylog server log with the following command:

tail -f /var/log/graylog-server/server.log

Once the Graylog server has started successfully, you should get the following output:

2020-09-05T08:51:36.473Z INFO  [ServerBootstrap] Services started, startup times in ms: {InputSetupService [RUNNING]=59, JobSchedulerService [RUNNING]=105, GracefulShutdownService [RUNNING]=106, OutputSetupService [RUNNING]=110, BufferSynchronizerService [RUNNING]=111, UrlWhitelistService [RUNNING]=153, JournalReader [RUNNING]=166, KafkaJournal [RUNNING]=222, MongoDBProcessingStatusRecorderService [RUNNING]=240, ConfigurationEtagService [RUNNING]=259, EtagService [RUNNING]=302, StreamCacheService [RUNNING]=306, LookupTableService [RUNNING]=376, PeriodicalsService [RUNNING]=655, JerseyService [RUNNING]=58701}
2020-09-05T08:51:36.477Z INFO  [ServerBootstrap] Graylog server up and running.

At this time, the Graylog server starts and listens to port 9000.

Configure Nginx as a reverse proxy for Graylog

Next, you need to install and configure Nginx as a reverse proxy to access the Graylog server.

First install the Nginx server with the following command:

apt-get install nginx -y

After installing the Nginx server, create a new Nginx virtual host configuration file with the following command:

nano /etc/nginx/sites-available/graylog.conf

Add the following lines:

server {
    listen 80;
    server_name graylog.example.org;

    location / {
      proxy_set_header Host $http_host;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Server $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Graylog-Server-URL http://$server_name/;
      proxy_pass       http://127.0.0.1:9000;
    }
}

Save and close the file when you are done. Then check Nginx for syntax errors with the following command:

nginx -t

You should get the following output:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Then enable Nginx virtual host configuration file with the following command:

ln -s /etc/nginx/sites-available/graylog.conf /etc/nginx/sites-enabled/

Finally, restart the Nginx service to apply the changes:

systemctl restart nginx

Then check the status of Graylog with the following command:

systemctl status nginx

You should get the following output:

? nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2020-09-05 09:07:50 UTC; 20s ago
       Docs: man:nginx(8)
    Process: 9408 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
    Process: 9419 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
   Main PID: 9423 (nginx)
      Tasks: 3 (limit: 2353)
     Memory: 10.2M
     CGroup: /system.slice/nginx.service
             ??9423 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
             ??9424 nginx: worker process
             ??9425 nginx: worker process

Sep 05 09:07:50 ubuntu2004 systemd[1]: Starting A high performance web server and a reverse proxy server...
Sep 05 09:07:50 ubuntu2004 systemd[1]: Started A high performance web server and a reverse proxy server.

Go to the Graylog web interface

Now open your browser and enter the URL http://graylog.example.com. You will be redirected to the Graylog login page as follows:

Login for Graylog

Enter your admin username, password and click Sign in button. You should see the Graylog Dashboard on the following page:

Get started with Graylog

Now click on System >> Overview. You should see the status of the Graylog server on the following page:

GraylogAdvertisement

Conclusion

Congratulations! you have successfully installed and configured the Graylog server with Nginx as a reverse proxy on Ubuntu 20.04. You can now explore Graylog and create an entry to receive Rsyslog logs from external sources. Feel free to ask me if you have any questions.


Source link