1. 基本信息
2. 安装
2.1 blackbox_exporter 安装
$ pwd/alidata1/admin/tools/exporter/blackbox$ ls ## 配置文件blackbox.yml 使用默认即可blackbox_exporter blackbox.yml LICENSE NOTICE## 配置开机自启文件$ sudo cat /etc/systemd/system/blackbox_exporter.service[Unit]Description=BlackBox Exporter[Service]User=adminGroup=adminExecStart=/alidata1/admin/tools/exporter/blackbox/blackbox_exporter --config.file=/alidata1/admin/tools/exporter/blackbox/blackbox.yml --web.listen-address=192.168.242.205:20011 --log.level=warn Restart=on-failure[Install]WantedBy=multi-user.target## 启动并配置开机自启$ sudo systemctl enable --now blackbox_exporter
2.2 prometheus安装
安装过程见 Prometheus安装 , 此处不再安装, 只是配置需要做些改动
## 目录结构$ pwd/alidata1/admin/tools/prometheus-2.37.5$ $ ll总用量 206448drwxr-xr-x 2 admin admin 22 6月 6 17:02 authdrwxrwxr-x 2 admin admin 90 6月 7 14:44 confdrwxr-xr-x 2 admin admin 38 12月 9 21:06 console_librariesdrwxr-xr-x 2 admin admin 173 12月 9 21:06 consoles-rw-r--r-- 1 admin admin11357 12月 9 21:06 LICENSE-rw-r--r-- 1 admin admin3773 12月 9 21:06 NOTICE-rwxr-xr-x 1 admin admin 109779661 12月 9 20:49 prometheus-rw-r--r-- 1 admin admin2173 6月 7 11:44 prometheus.yml-rwxr-xr-x 1 admin admin 101601052 12月 9 20:52 promtooldrwxrwxr-x 2 admin admin 30 6月 7 14:33 rules## 1. 在配置文件prometheus.yml 中追加关于blackbox的配置......## 应用状态- job_name: services_statusmetrics_path: /probefile_sd_configs:- files: ['./conf/blackbox_service_status.yml']params:module: [tcp_connect]relabel_configs:- source_labels: [__address__]target_label: __param_target- source_labels: [__param_target]target_label: instance- replacement: "192.168.242.205:20011"target_label: __address__## 2. 在配置文件prometheus.yml中新增告警配置# Alertmanager configuration## 将告警打开alerting:alertmanagers:- static_configs:- targets:- 192.168.242.205:9093 rule_files: ## 告警配置文件- "rules/*.yml"## 3. 新增配置文件 ./conf/blackbox_service_status.yml 里面是关于应用的配置$ cat ./conf/blackbox_service_status.yml- targets: ['192.168.242.205:9090']labels: {appName: frontend, env: prd }- targets: ['192.168.84.56:8635', '192.168.242.205:9091']labels: {appName: backend, env: prd }## 4. 新增告警规则配置文件 rules/blackbox_service.ymlgroups:- name: appstatusrules:- alert: 服务已停止expr: probe_success{job=~"services_status"} == 0 for: 1mlabels:severity: critical annotations:summery: "当前应用 {{ $labels.appName }} : {{ $labels.instance }} 服务已停止, 请尽快处理"## 5. 重新加载prometheus配置$ curl -XPUT http://192.168.242.205:9090/-/reload
2.3 alertmanager安装
$ pwd/alidata1/admin/tools/alertmanager-0.25.0$ lsalertmanager alertmanager.yml amtool LICENSE NOTICE template## 1. 查看配置文件alertmanager.ymlglobal:resolve_timeout: 5msmtp_smarthost: ":465"smtp_from: "939545179@"smtp_auth_username: "939545179@"smtp_auth_password: "xxxxxxx" ## qq邮箱授权码smtp_require_tls: falsetemplates:- './template/*.tmpl'route:group_by: ['Alert']group_wait: 10sgroup_interval: 10srepeat_interval: 5mreceiver: 'mail'routes:- receiver: 'mail'match_re:severity: critical|warning receivers:- name: "mail"email_configs:- to: '123456789@'html: '{{ template "email.html" . }}'headers: { Subject: '{{ .CommonLabels.appName }}: {{ .CommonLabels.alertname }} ' }## 2. 邮件内容配置$ cat template/mail.tmpl {{ define "email.html" }}{{ range .Alerts }}<pre><strong>应用名:</strong> {{ .Labels.appName }}<strong>环境 :</strong> {{ .Labels.env }}<strong>实例 :</strong> {{ if gt (len .Labels.instance) 0 -}} {{ .Labels.instance }} {{ else }} grafana {{ end }}<strong>信息 :</strong> {{ .Annotations.summery }}<strong>时间 :</strong> {{ (.StartsAt.Add 28800e9).Format "-01-02 15:04:05" }}</pre>{{ end }}{{ end }}## 3.配置开机自启$ cat /etc/systemd/system/alertmanager.service[Unit]Description=AlertManager ServiceAfter=network.target[Service]User=adminGroup=adminExecStart=/alidata1/admin/tools/alertmanager-0.25.0/alertmanager --config.file=/alidata1/admin/tools/alertmanager-0.25.0/alertmanager.yml --storage.path=/alidata1/admin/data/alertmanager --web.listen-address=192.168.242.205:9093 --log.level=warnRestart=on-failureRestartSec=10[Install]WantedBy=multi-user.target$ sudo systemctl enable alertmanager --now
3. 验证
如果觉得《prometheus + alertmanager + blackbox_exporter 实现应用监控并发送告警邮件》对你有帮助，请点赞、收藏，并留下你的观点哦！
