Enable Logpush to New Relic
Cloudflare Logpush supports pushing logs directly to New Relic via the Cloudflare dashboard or via API.
Manage via the Cloudflare dashboard
To enable a Logpush service to New Relic via the dashboard:
-
Log in to the Cloudflare dashboard.
-
Select the Enterprise account or domain you want to use with Logpush.
-
Go to Analytics & Logs > Logs.
-
Select Add Logpush job.
-
In Select data set, choose the dataset to push to a storage service, and select Next.
-
In Select data fields:
- Select the data fields to include in your logs. Add or remove fields later by modifying your settings in Logs > Logpush.
- In Advanced Settings, you can change the Timestamp format (
RFC3339
(default),Unix
, orUnixNano
), Sampling rate and enable redaction forCVE-2021-44228
. - Under Filters you can select the events to include and/or remove from your logs. For more information, refer to Filters. Not all datasets have this option available.
-
In Select a destination, choose New Relic.
-
Enter the New Relic Logs Endpoint:
US:
"https://log-api.newrelic.com/log/v1?Api-Key=<NR_LICENSE_KEY>&format=cloudflare"
EU:
"https://log-api.eu.newrelic.com/log/v1?Api-Key=<NR_LICENSE_KEY>&format=cloudflare"
Use the region that matches the one that has been set on your New Relic account. The
<NR_LICENSE_KEY>
field can be found on the New Relic dashboard. It can be retrieved by following these steps. -
Select Validate access.
-
Select Save and Start Pushing to finish enabling Logpush.
Once connected, Cloudflare lists New Relic as a connected service under Logs > Logpush. Edit or remove connected services from here.
Manage via API
Ensure Log Share permissions are enabled, before attempting to read or configure a Logpush job. For more information refer to the Roles section. 1. Create a job
To create a job, make a POST
request to the Logpush jobs endpoint with the following fields:
-
name (optional) - Use your domain name as the job name.
-
logpull_options (optional) - To configure fields, sample rate, and timestamp format, refer to API configuration options.
-
destination_conf - A log destination consisting of an endpoint URL, a license key and a format in the string format below.
-
<NR_ENDPOINT_URL>
: The New Relic HTTP logs intake endpoint, which ishttps://log-api.newrelic.com/log/v1
for US orhttps://log-api.eu.newrelic.com/log/v1
for the EU, depending on the region that has been set on your New Relic account. -
<NR_LICENSE_KEY>
: This key can be found on the New Relic dashboard and it can be retrieved by following these steps. -
format
: The format iscloudflare
.US:
"https://log-api.newrelic.com/log/v1?Api-Key=<NR_LICENSE_KEY>&format=cloudflare"
EU:
"https://log-api.eu.newrelic.com/log/v1?Api-Key=<NR_LICENSE_KEY>&format=cloudflare"
-
-
max_upload_records (optional) - The maximum number of log lines per batch. This must be at least 1,000 lines or more. Note that there is no way to specify a minimum number of log lines per batch. This means that log files may contain many fewer lines than specified.
-
max_upload_bytes (optional) - The maximum uncompressed file size of a batch of logs. This must be at least 5 MB. Note that there is no way to set a minimum file size. This means that log files may be much smaller than this batch size. Nevertheless, it is recommended to set this parameter to 5,000,000.
-
dataset - The category of logs you want to receive. Refer to Log fields for the full list of supported datasets.
Example request using cURL:
curl -s https://api.cloudflare.com/client/v4/zones/<ZONE_ID>/logpush/jobs -X POST -d '{"name": "<DOMAIN_NAME>","logpull_options": "fields=ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID×tamps=unix","destination_conf": "https://log-api.newrelic.com/log/v1?Api-Key=<NR_LICENSE_KEY>&format=cloudflare","max_upload_bytes": 5000000,"dataset": "http_requests","enabled": true}' \-H "X-Auth-Email: <EMAIL>" \-H "X-Auth-Key: <API_KEY>" | jq .
Response:
{"errors" : [],"messages" : [],"result" : {"dataset" : "http_requests","destination_conf" : "https://log-api.newrelic.com/log/v1?Api-Key=<NR_LICENSE_KEY>&format=cloudflare","enabled" : true,"error_message" : null,"frequency" : "high","id" : 100,"kind" : "","last_complete" : null,"last_error" : null,"logpull_options" : "fields=ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID×tamps=unix","logstream" : true,"max_upload_bytes" : 5000000,"name" : "<DOMAIN_NAME>"},"success" : true}
2. Enable (update) a job
To enable a job, make a PUT
request to the Logpush jobs endpoint. You will use the job ID returned from the previous step in the URL and send {"enabled": true}
in the request body.
Example request using cURL:
curl -s -X PUT \https://api.cloudflare.com/client/v4/zones/<ZONE_ID>/logpush/jobs/100 -d'{"enabled":true}' \-H "X-Auth-Email: <EMAIL>" \-H "X-Auth-Key: <API_KEY>" | jq .
Response:
{"errors" : [],"messages" : [],"result" : {"dataset" : "http_requests","destination_conf" : "https://log-api.newrelic.com/log/v1?Api-Key=<NR_LICENSE_KEY>&format=cloudflare","enabled" : true,"error_message" : null,"frequency" : "high","id" : 100,"kind" : "","last_complete" : "null","last_error" : null,"logpull_options" : "fields=ClientIP,ClientRequestHost,ClientRequestMethod,ClientRequestURI,EdgeEndTimestamp,EdgeResponseBytes,EdgeResponseStatus,EdgeStartTimestamp,RayID×tamps=unix","logstream" : true,"max_upload_bytes" : 5000000,"name" : "<DOMAIN_NAME>"},"success" : true}