Viewing metrics and logs from Azure
BigAnimal sends all metrics and logs from Postgres clusters to Azure. You can view metrics and logs directly from Azure.
Azure log analytics
When BigAnimal deploys workloads on Azure, the logs from the Postgres clusters are forwarded to the Azure Log Workspace. To query BigAnimal logs, you must use Azure log analytics and Kusto query language.
Query Postgres cluster logs
All logs from your Postgres clusters are stored in the Customer Log Analytics workspace. To find your Customer Log Analytics workspace:
Sign in to the Azure portal.
Select Resource Groups.
Select the resource group corresponding to the region where you choose to deploy your BigAnimal cluster. You see resources included in that resource group.
Select the resource of type Log Analytics workspace with the prefix
log-workspace-
.Select the logs in the menu on the left in the General section.
Close the dashboard with prebuilt queries, which brings you to the KQL editor.
The following tables containing BigAnimal logs are available in the Customer Log Analytic workspace.
Table name | Description | Logger |
---|---|---|
PostgresLogs_CL | Logs of the customer clusters databases (all Postgres-related logs) | logger = postgres |
PostgresAuditLogs_CL | Audit logs of the customer clusters databases | logger = pgaudit or edb_audit |
Logs are split into structured fields matching those of the Postgres
csvlog format
with a record_
prefix and a type suffix. For example, the application_name
is in the record_application_name_s
log field. The pg_cluster_id_s
field identifies the specific Postgres cluster that originated the log message.
You can use the KQL Query editor to compose your queries over these tables. For example:
Using BigAnimal's cloud account
To access your Postgres cluster logs, when using BigAnimal's cloud account, generate a SAS token from BigAnimal and use it to download the logs.
In the BigAnimal portal, select Clusters, select your cluster, and select the Monitoring & Logging tab.
Select Generate Token and copy the SAS token. The SAS token is a sensitive value, so don't make it publicly available. The following is a sample SAS token:
Enter the
azcopy
command to download the Postgres logs from BigAnimal. For example:
Metrics
BigAnimal collects a wide set of metrics about Postgres instances into the DpMetrics_CL
log analytics table. Most of these metrics are acquired directly from Postgres system tables, views, and functions. The Postgres documentation serves as the main reference for these metrics.
Available metrics
The following tables in the Customer Log Analytic workspace contain metrics specific to BigAnimal.
Table name | Description |
---|---|
DpMetrics_CL | Metrics streams from BigAnimal Prometheus |
You can use the KQL Query editor in the Log Workspace view to compose queries over these tables.
The forwarded Prometheus metrics use structured JSON fields, particularly the Message
and Message.labels
fields. To query these fields, you need to use the KQL function
todynamic()
in your queries.
Currently, all metrics forwarded from Prometheus are in the DpMetrics_CL
table. This might change in a future release.
The available set of metrics is subject to change. Metrics might be added, removed or renamed. Where feasible, we will change the metric name when the meaning or type of an existing metric changes.
Metrics labels
All Postgres metrics share a common labeling scheme. Entries generally have at least the following labels.
Name | Description |
---|---|
instance | IP address of the host the metric originated from |
postgresql | BigAnimal postgres cluster identifier, e.g., p-abcdef012345 |
role | Postgres instance role: primary or standby replica |
datname | Postgres database name (where applicable) |
pod | k8s pod name |
aks_cluster_name | AKS cluster name |
When querying for labels, for best performance, apply filters that don't require inspecting labels (for example, filters by metric name) before label-based filters.
The labels
field of a metrics entry is a nested field under the JSON-typed Message
field. To query the field for individual values, you JSON parse the Message
field and dot reference (m.labels
) the labels property. Some uses of values might require explicit casts to another type, for example, tostring(...)
.
Example usage:
For a detailed list of available metrics, see metrics.
Workbooks
You can view logs from your Postgres clusters using Azure Workbooks. See Azure Monitor Workbooks.
To access Azure Workbooks:
Sign in to the Azure portal.
Search for
workbooks
.Filter the search results as needed. For example, you might need to filter by subscription if your account has more than one and the location that corresponds with the region where you deployed your cluster. You can either use the filter controls or search to narrow the search results.
Select the workbook link in the Name column to open the workbook details. Names have the format
${GUID} (xxx-rg-${region}-management-xxx-customer)
, for example,62b6d449-e0e5-480a-9809-cae2ff6510e9 (123-rg-australiaeast-management-abcd-customer)
.Select Open Workbook in the workbook details to launch an interactive workbook with various metrics for your deployed clusters.
- On this page
- Azure log analytics
- Metrics
- Workbooks