Filebeat is a lightweight shipper for forwarding and centralizing log data. Filebeat can be installed as an agent on your servers. Filebeat monitors the log files or locations that you specify collect log events and forwards them to the Event Log Manager (ELM) module for indexing.
When you start Filebeat, it starts one or more inputs that look in the locations you’ve specified for log data. For each log that Filebeat locates, Filebeat starts a harvester. Each harvester reads a single log for new content and sends the new log data to the Event Log Manager (ELM) module.
Install Filebeat on all the servers you want to monitor custom log files.
To download and install Filebeat, use the commands that work with your system:
Download the Filebeat Windows zip file from the downloads page.
Extract the contents of the zip file into C:\Program Files.
Rename the filebeat--windows directory to Filebeat.
Open a PowerShell prompt as an Administrator (right-click the PowerShell icon and select Run As Administrator).
From the PowerShell prompt, run the following commands to install Filebeat as a Windows service:
PS > cd 'C:\Program Files\Filebeat'
PS C:\Program Files\Filebeat> .\install-service-filebeat.ps1
If script execution is disabled on your system, you need to set the execution policy for the current session to allow the script to run. For example: PowerShell.exe -ExecutionPolicy UnRestricted -File .\install-service-filebeat.ps1
Run the following command
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.10.2-amd64.deb
sudo dpkg -i filebeat-7.10.2-amd64.deb
Run the following command
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.10.2-x86_64.rpm
sudo rpm -vi filebeat-7.10.2-x86_64.rpm
Run the following command
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.10.2-darwin-x86_64.tar.gz
tar xzvf filebeat-7.10.2-darwin-x86_64.tar.gz
Run the following command
brew tap elastic/tap
brew install elastic/tap/filebeat-full
Run the following command
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.10.2-linux-x86_64.tar.gz
tar xzvf filebeat-7.10.2-linux-x86_64.tar.gz
Connections to blësk ELM are required to set up Filebeat.
Set the connection information in filebeat.yml. To locate this configuration file, see Directory layout.
This configuration is required only once and will be applied to similar platforms. Such as Windowns, Linux, MacOS etc.
Set the host and port where Filebeat can find the blësk ELM installation. Comment the line related to output.elasticsearch and uncomment the line related to output.logstash in the filebeat.yml file. For example:
# output.elasticsearch:
# Array of hosts to connect to.
# hosts: ["localhost:9200"]
output.logstash:
hosts: ["http://blesk.ip.addr:5045"]
There are several ways to collect log data with Filebeat.
Define the path information in filebeat.yml where the log files you want to analyze data are located.
In the following example, all files with extention .log under the /var/log/ folder will have their content crawled and send to blesk ELM. Change the line that read /var/log/*.log to the path where your log file is located. For example:
Change -> /var/log/*.log for C:\Program Files\mysoftware\mylogfolder\myfile.log
If you have multiple log files located in different locations, add each additional path, one line after the other.
To test your filebeat configuration (syntax), you can do:
PS C:\Program Files\filebeat> .\filebeat.exe test config
Config OK
To test the output block, you can do:
PS C:\Program Files\filebeat> .\filebeat.exe test output
parse url... OK
connection...
parse host... OK
dns lookup... OK
Start the Filebeat service, with the following command:
PS C:\Program Files\filebeat> Start-Service filebeat
By default, Windows log files are stored in C:\ProgramData\filebeat\Logs.
sudo service filebeat start
sudo service filebeat start
sudo chown root filebeat.yml
sudo chown root modules.d/system.yml
sudo ./filebeat -e
To have launchd start elastic/tap/filebeat and then restart it at login, run:
brew services start elastic/tap/filebeat-full
To run Filebeat in the foreground instead of running it as a background service, run:
sudo chown root /usr/local/etc/filebeat/filebeat.yml
sudo chown root /usr/local/etc/filebeat/modules.d/system.yml
sudo filebeat -e
sudo chown root filebeat.yml
sudo chown root modules.d/nginx.yml
sudo ./filebeat -e