Management of Notification

Feature Overview

With notifications, you can integrate the platform's monitoring and alerting features to promptly send pre-warning information to notification recipients, reminding relevant personnel to take necessary measures to resolve issues or avoid failures.

Key Features

  • Notification Server: The notification server provides services for sending notification messages to receiver groups on the platform, such as an email server.
  • Receiver Group: A receiver group is a set of notification recipients with similar logical characteristics, which can reduce your maintenance burden by allowing a categorization of entities that receive notification messages.
  • Notification Template: A notification template is a standardized structure composed of custom content, content variables, and content format parameters. It is used to standardize the content and format of alert notification messages for notification rules. For example, customizing the subject and content of email notifications.
  • Notification Rule: A notification rule is a collection of rules defining how to send notification messages to specific contacts. It is essential to use a notification rule for scenarios such as alerts, inspections, and login authentication that require notifying external services.

Notification Server

The notification server provides services for sending notification messages to recipients on the platform. The platform currently supports the following notification servers:

  • Corporate Communication Tool Server: Supports integration with WeChat Work, DingTalk, and Feishu built-in applications for sending notifications to individuals.
  • Email Server: Sends notifications via email using an email server.
  • Webhook Type Server: Supports integration with corporate WeChat group bots, DingTalk group bots, Feishu group bots, or sending WebHooks to your designated server.
WARNING

Only one corporate communication tool server can be added.

Corporate Communication Tool Server

WeChat Work

  1. Configure the notification server parameters as per the example below. Once parameters are filled in, switch to the global cluster in Cluster Management > Resource Management and create the resource object.

    # WeChat Work corpId, corpSecret, agentId acquisition methods can be referenced in the official documentation: https://developer.work.weixin.qq.com/document/path/90665
    apiVersion: v1
    kind: Secret
    type: NotificationServer
    metadata:
      labels:
        cpaas.io/notification.server.type: CorpWeChat
        cpaas.io/notification.server.category: Corp
      name: platform-corp-wechat-server
      namespace: cpaas-system
    data:
      displayNameZh: 企业微信 # Server's Chinese display name, encoded in base64 by default
      displayNameEn: WeChat # Server's English display name, encoded in base64 by default
      corpId: # Corporate ID, encoded in base64 by default
      corpSecret: # Application secret, encoded in base64 by default
      agentId: # Corporate application ID, encoded in base64 by default
  2. After the creation, you need to update the user's WeChat Work ID in the platform's User Role Management > User Management or in the user's Personal Information to ensure the user can receive messages normally.

DingTalk

  1. Configure the notification server parameters as per the example below. Once parameters are filled in, switch to the global cluster in Cluster Management > Resource Management and create the resource object.

    # DingTalk appKey, appSecret, agentId acquisition method: https://open-dev.dingtalk.com/fe/app#/corp/app
    apiVersion: v1
    kind: Secret
    type: NotificationServer
    metadata:
      labels:
        cpaas.io/notification.server.type: CorpDingTalk
        cpaas.io/notification.server.category: Corp
      name: platform-corp-dingtalk-server
      namespace: cpaas-system
    data:
      displayNameZh: 钉钉 # Server's Chinese display name, encoded in base64 by default
      displayNameEn: DingTalk # Server's English display name, encoded in base64 by default
      appKey: # Application key, encoded in base64 by default
      appSecret: # Application secret, encoded in base64 by default
      agentId: # Application agent_id, encoded in base64 by default
  2. After the creation, you need to update the user's DingTalk ID in the platform's User Role Management > User Management or in the user's Personal Information to ensure the user can receive messages normally.

Feishu

  1. Configure the notification server parameters as per the example below. Once parameters are filled in, switch to the global cluster in Cluster Management > Resource Management and create the resource object.

    # Feishu appId, appSecret acquisition methods: https://open.feishu.cn/app/
    apiVersion: v1
    kind: Secret
    type: NotificationServer
    metadata:
      labels:
        cpaas.io/notification.server.type: CorpFeishu
        cpaas.io/notification.server.category: Corp
      name: platform-corp-feishu-server
      namespace: cpaas-system
    data:
      displayNameZh: 飞书 # Server's Chinese display name, encoded in base64 by default
      displayNameEn: Feishu # Server's English display name, encoded in base64 by default
      appId: # Application ID, encoded in base64 by default
      appSecret: # Application secret, encoded in base64 by default
  2. After the creation, you need to update the user's Feishu ID in the platform's User Role Management > User Management or in the user's Personal Information to ensure the user can receive messages normally.

Email Server

  1. In the left navigation bar, click Platform Settings > Notification Server.

  2. Click Configure Now.

  3. Refer to the following instructions to configure the relevant parameters.

    ParameterDescription
    Service AddressThe address of the notification server supporting the SMTP protocol, e.g., smtp.yeah.net.
    PortThe port number for the notification server. When Use SSL is checked, the SSL port number must be entered.
    Server ConfigurationUse SSL: Secure Socket Layer (SSL) is a standard security technology. The SSL switch is used to control whether to establish an encrypted link between the server and client.
    Skip Insecure Verification: The insecureSkipVerify switch is used to control whether to verify the client certificate and server hostname. If enabled, certificates and the consistency between the hostname in the certificate and the server hostname will not be verified.
    Sender EmailThe sender's email account in the notification server, used for sending notification emails.
    Enable AuthenticationIf authentication is required, please configure the username and authorization code for the email server.
  4. Click OK.

Webhook Type Server

Supports integration with corporate WeChat group bots, DingTalk group bots, Feishu group bots, or sending HTTP requests to your designated Webhook server.

Corporate WeChat Group Bot

  1. In the left navigation bar, click Cluster Management > Cluster.

  2. Click the operation button next to the global cluster > CLI Tool.

  3. Execute the following command on the master node of the global cluster:

    kubectl patch secret -n cpaas-system platform-wechat-server -p '{"data":{"enable":"dHJ1ZQo="}}'

    Tip: dHJ1ZQo= is the base64 encoded value of true; to disable, replace dHJ1ZQo= with ZmFsc2UK, which is the base64 encoded value of false.

DingTalk Group Bot

  1. In the left navigation bar, click Cluster Management > Cluster.

  2. Click the operation button next to the global cluster > CLI Tool.

  3. Execute the following command on the master node of the global cluster:

    kubectl patch secret -n cpaas-system platform-dingtalk-server -p '{"data":{"enable":"dHJ1ZQo="}}'

    Tip: dHJ1ZQo= is the base64 encoded value of true; to disable, replace dHJ1ZQo= with ZmFsc2UK, which is the base64 encoded value of false.

Feishu Group Bot

  1. In the left navigation bar, click Cluster Management > Cluster.

  2. Click the operation button next to the global cluster > CLI Tool.

  3. Execute the following command on the master node of the global cluster:

    kubectl patch secret -n cpaas-system platform-feishu-server -p '{"data":{"enable":"dHJ1ZQo="}}'

    Tip: dHJ1ZQo= is the base64 encoded value of true; to disable, replace dHJ1ZQo= with ZmFsc2UK, which is the base64 encoded value of false.

Webhook Server

  1. In the left navigation bar, click Cluster Management > Cluster.

  2. Click the operation button next to the global cluster > CLI Tool.

  3. Execute the following command on the master node of the global cluster:

    kubectl patch secret -n cpaas-system platform-webhook-server -p '{"data":{"enable":"dHJ1ZQo="}}'

    Tip: dHJ1ZQo= is the base64 encoded value of true; to disable, replace dHJ1ZQo= with ZmFsc2UK, which is the base64 encoded value of false.

Configure Custom Request Headers for Webhook Notifications

If the target Webhook endpoint requires custom HTTP request headers, create a Secret in the cpaas-system namespace and associate it with the receiver group later.

  1. In the left navigation bar, click Cluster Management > Cluster.

  2. Click the operation button next to the global cluster > CLI Tool.

  3. Create a YAML file similar to the following on the master node of the global cluster.

    apiVersion: v1
    kind: Secret
    metadata:
      name: webhook-header-secret
      namespace: cpaas-system
    type: NotificationSender
    data:
      X-Secret: <base64-encoded-header-value>
  4. Save the YAML file as webhook-header.yaml and apply the resource.

    kubectl apply -f webhook-header.yaml
INFO
  1. Each key under data is used as an HTTP header name.
  2. Each value under data must be base64 encoded before it is applied to the cluster.
  3. If the endpoint requires multiple headers, add more key-value pairs under data.

Receiver Group

A receiver group is a set of notification recipients with similar logical characteristics. For example, you can set an operations and maintenance team as a receiver group for easy selection and management when configuring notification rules.

INFO
  1. The platform supports various notification servers, and the corresponding configuration options for notification types will be displayed based on the notification server configuration.
  2. If you need to use a Webhook type server as a notification recipient, you must configure the relevant URL in the receiver group.
  3. If the Webhook endpoint requires custom request headers, you must associate the receiver group with a Secret of type NotificationSender in the cpaas-system namespace.
  1. In the left navigation bar, click Operations Center > Notifications.

  2. Switch to the Receiver Group tab.

  3. Click Create Receiver Group and configure the relevant parameters as per the instructions below.

    ParameterDescription
    EmailAdd an email to the entire receiver group. The platform will send notifications to this email and all contacts' emails in the group.
    Webhook URL/WeChat Group Bot/DingTalk Group Bot/Feishu Group BotPlease fill in the corresponding notification method URL based on the configured notification server. Once configured, contacts in this group will be notified using this method.
    Contact ConfigurationClick Add Contact to add existing platform users to the receiver group. Ensure the accuracy of the selected contacts' contact information (phone, email, interface callback) to avoid missing message notifications.
  4. Click Add.

Associate a Receiver Group with a Header Secret

If the configured Webhook endpoint requires custom request headers, associate the receiver group with the Secret created in Configure Custom Request Headers for Webhook Notifications.

  1. In the left navigation bar, click Cluster Management > Cluster.

  2. Click the operation button next to the global cluster > CLI Tool.

  3. Edit the corresponding receiver group (NotificationGroup) resource on the master node of the global cluster.

    kubectl edit notificationgroups.ait.alauda.io -n cpaas-system <notification-group-name>
  4. Add the cpaas.io/notification.webhook.config annotation to metadata.annotations. Its value must be the name of the Secret created for the custom request headers.

    apiVersion: ait.alauda.io/v1beta1
    kind: NotificationGroup
    metadata:
      name: <notification-group-name>
      namespace: cpaas-system
      annotations:
        cpaas.io/notification.webhook.config: webhook-header-secret
INFO

The Webhook URL is configured in the receiver group, while custom request headers are configured through the referenced Secret.

Notification Template

A notification template is a standardized structure composed of custom content, content variables, and content format parameters. It is used to standardize the content and format of alert notification messages for notification rules.

Platform administrators or operations personnel can set notification templates to customize the content and format of notification messages based on different alert notification methods, helping users quickly get critical alert information and improve operational efficiency.

Platform administrators can make platform notification templates public. Public templates can be selected in project notification rules, so project administrators can reuse platform-maintained message formats without creating duplicate templates in each project.

INFO

The platform supports various notification servers, and the corresponding notification type templates will be displayed according to the notification server configuration. If no notification server is configured, the corresponding notification templates will not be displayed by default.

Create Templates

  1. In the left navigation bar, click Operations Center > Notifications.

  2. Switch to the Templates tab.

  3. Click Create Templates.

  4. In the Basic Information section, configure the following parameters.

    ParameterDescription
    Message TypeSelect the type of message according to the purpose of the notification.
    Alert Message: Sends alert messages triggered by alert rules, in conjunction with the platform's alerting functionality;
    Component Exception Message: Sends notification information triggered by exceptions in certain components.
    PublicMake the template available to project notification rules. This option is available for platform notification templates. After the template is made public, project administrators can select it from the platform public template group when creating or updating notification rules. Public templates are marked with a read-only Public label in the template list and show the public status on the details page. When you make a public template private again, confirm the impact in the dialog box before submitting the change.
  5. In the Template Configuration section, reference different template types to configure variables and content formatting parameters.

INFO
  1. The content of the template can only consist of variables, variable display names, and special formatting markup language supported by the platform. Variables and other elements can be freely combined as long as they comply with the syntax rules.
  2. Only variables supported by the platform can be used in the template. You can modify variable display names and content formats, but you cannot modify the variable itself. Refer to Reference Variables, and Special Formatting Markup Language in Emails.
  3. The platform provides default notification template content for various notification types based on actual operational scenarios, which can meet most notification message setting needs. If there are no special requirements, you may directly use the default template content.
  4. If a platform public template is made private again, it remains visible but disabled in the platform public template group. It cannot be selected for new notification rules, but existing notification rules that already use the template can continue to send notifications.
  5. If a platform public template is referenced by any project notification rule, it is protected from deletion. Remove all references before deleting the template.
  1. Click Create.

Reference Variables

Variables are the keys of labels or annotations in notification messages (NotificationMessage), formatted as {{.labelKey}}. To facilitate users in quickly obtaining key information, custom display names can be assigned to variables; for example: Alert Level: {{ .externalLabels.severity }}.

When a notification rule sends notification messages to users based on a notification template, the variables in the template will reference the corresponding label values in the notification message (actual monitoring data). Ultimately, monitoring data will be sent to users in a standardized content format.

The platform provides the following basic variables by default:

Display NameVariableDescription
Alert Status{{ .externalLabels.status }}For example: Alerting.
Alert Level{{ .externalLabels.severity }}For example: Critical.
Alert Cluster{{ .labels.alert_cluster }}For example: Cluster 1 where the alert occurred.
Alert Object{{ .externalLabels.object }}The type and name of the resource where the alert occurred, e.g., node 192.168.16.53.
rule Name{{ .labels.alert_resource }}The name of the alert rule, e.g., cpaas-node-rules.
Alert Description{{ .externalLabels.summary }}Description of the alert rule.
Trigger Value{{ .externalLabels.currentValue }}The monitored value that triggered the alert.
Alert Time{{ dateFormatWithZone .startsAt "2006-01-02 15:04:05" "Asia/Chongqing" }}The start time of the alert.
Recovery Time{{ dateFormatWithZone .endsAt "2006-01-02 15:04:05" "Asia/Chongqing" }}The end time of the alert.
Metric Name{{ .labels.alert_indicator }}Name of the monitoring metric.

Special Formatting Markup Language in Emails

In email notifications, common HTML format tags and their instructions are referenced in the table below:

Content ElementTagDescription
Text-Supports input of Chinese/English text content.
Font<font color="#FF0000">Set Font Color</font>
<b>Bold Font</b>
Set font format.
Title<h1>Level 1 Title</h1>, supports up to h6 (header 6).Set title level.
Paragraph<p>Paragraph</p>Insert regular paragraph text.
Quote<q>Quote</q>Insert short quoted content.
Hyperlink<a href="//www.example.com">Hyperlink</a>Insert a hyperlink.

Notification Rule

A notification rule is a collection of rules defining how to send notification messages to specific contacts. It is essential to use notification rules for scenarios requiring notification to external services, such as alerts, inspections, and login authentication.

INFO

The platform supports various notification servers, and the notification modes corresponding to notification types will be displayed based on the notification server configuration. If no notification server is configured, the corresponding notification modes will not be displayed by default.

Prerequisites

To use the Corporate Communication Tool Server to notify contacts, users must first modify their contact information in Personal Information by entering their WeChat Work ID.

Operation Procedures

  1. In the left navigation bar, click Operations Center > Notifications.

  2. Click Create Rule and configure the relevant parameters as per the following instructions.

    ParameterDescription
    Receiver GroupA receiver group is a logical set of notification recipients, which the platform will notify using the specified notification method.
    Notification RecipientsChoose to add one or more notification recipients, and the platform will send notifications according to the recipients' Personal Information contact methods.
    Notification MethodSupports multiple methods including WeChat Work, DingTalk, Feishu, Corporate WeChat Group Bot, DingTalk Group Bot, Feishu Group Bot, WebHook URL, and supports multiple selections.
    Note: Some parameters will be displayed after configuring the notification server.
    Notification TemplateSelect the notification template to display notification information. In the project view, templates are grouped by source. You can select a project template or a platform public template.
  3. Click Create.

Set Notification Rules for Projects

The platform's notification rules, notification templates, and receiver groups are tenant-isolated by default. As a project administrator, you cannot view or use notification rules, private notification templates, or receiver groups configured by other projects or platform administrators. Platform public notification templates are an exception: you can select them when creating or updating notification rules, but you cannot edit or delete them from the project view.

Prerequisites

  1. You have contacted the platform administrator to complete the notification server setup.

  2. If you need to notify through corporate communication tools, you also need to ensure that the contacts to be notified have correctly configured their communication tool IDs in Personal Information.

Operation Procedures

  1. In the Project Management view, click Project Name.

  2. In the left navigation bar, click Notifications.

  3. Switch to the Receiver Group tab, refer to Receiver Group to create a receiver group.

TIP

If you do not need to manage notification contacts through a receiver group or do not need to notify a webhook type notification server, you can skip this step.

  1. Switch to the Templates tab, refer to Notification Template to create a notification template. If a suitable platform public template is available, you can skip this step and select that template when creating the notification rule.

  2. Switch to the Rules tab, refer to Notification Rule to create a notification rule.