Ceph Dashboard介绍

Ceph 的监控可视化界面方案很多—-grafana、Kraken。但是从Luminous开始，Ceph 提供了原生的Dashboard功能，通过Dashboard可以获取Ceph集群的各种基本状态信息。

mimic版 (nautilus版) dashboard 安装。如果是 (nautilus版) 需要安装 ceph-mgr-dashboard

配置Ceph Dashboard

1、在每个mgr节点安装（安装之前确保每个节点上都安装了mgr: ceph-deploy mgr create node-1 node-2 node-3）

# yum install ceph-mgr-dashboard

2、开启mgr-dashboard功能

# ceph mgr module enable dashboard

3、生成并安装自签名的证书

# ceph dashboard create-self-signed-cert

4、创建一个dashboard登录用户名密码

# ceph dashboard set-login-credentials admin fxkjnj.com

5、查看服务访问方式

# ceph mgr services

{

“dashboard”: “https://node-1:8443/”

}

修改默认配置

指定集群dashboard的访问端口（默认的端口是8443 ）

# ceph config set mgr mgr/dashboard/server_port 8443

指定集群 dashboard的访问IP（通过ceph -s 查看 mgr 在那个主节点上，就把IP 设置在哪个节点上）

# ceph config set mgr mgr/dashboard/server_addr 192.168.31.12

修改完之后，记得在各个重启下ceph-mgr

systemctl restart ceph-mgr@node-1.service

systemctl restart ceph-mgr@node-2.service

systemctl restart ceph-mgr@node-3.service

访问 ceph dashboared

https://192.168.31.12:8443

ceph-dashboard

开启Object Gateway管理功能

1、创建test用户

[root@node-2 ~]# radosgw-admin user create --uid=test --display-name="test" --email=test@example.com

{

"user_id": "test",

"display_name": "test",

"email": "test@example.com",

"suspended": 0,

"max_buckets": 1000,

"subusers": [],

"keys": [

{

"user": "test",

"access_key": "KFS3VX0X3ZLC0HD0Y7C5",

"secret_key": "3qgTAHvb9fuzMSIPMY6owHzEsENvOW8PCn3k9271"

}

],

"swift_keys": [],

"caps": [],

"op_mask": "read, write, delete",

"default_placement": "",

"default_storage_class": "",

"placement_tags": [],

"bucket_quota": {

"enabled": false,

"check_on_raw": false,

"max_size": -1,

"max_size_kb": 0,

"max_objects": -1

},

"user_quota": {

"enabled": false,

"check_on_raw": false,

"max_size": -1,

"max_size_kb": 0,

"max_objects": -1

},

"temp_url_keys": [],

"type": "rgw",

"mfa_ids": []

}

2、提供Dashboard证书

# ceph dashboard set-rgw-api-access-key KFS3VX0X3ZLC0HD0Y7C5

# ceph dashboard set-rgw-api-secret-key 3qgTAHvb9fuzMSIPMY6owHzEsENvOW8PCn3k9271

3、配置rgw主机地址:

# ceph dashboard set-rgw-api-host 192.168.31.12

4、刷新web页面

Object Gateway

Prometheus 中添加ceph 监控

二进制部署promethues

官网下载对于的包：https://prometheus.io/download/

[root@prometheus ~]# wget https://github.com/prometheus/prometheus/releases/download/v2.5.0/prometheus-2.5.0.linux-amd64.tar.gz

[root@prometheus ~]# tar -xf prometheus-2.5.0.linux-amd64.tar.gz

[root@prometheus ~]# cd prometheus-2.5.0.linux-amd64/

[root@prometheus prometheus-2.5.0.linux-amd64]# cp prometheus promtool /usr/local/bin

 #查看下prometheus 版本
[root@prometheus prometheus-2.5.0.linux-amd64]# prometheus --version

#创建prometheus 配置文件目录
[root@prometheus prometheus-2.5.0.linux-amd64]# mkdir /etc/prometheus 

[root@prometheus prometheus-2.5.0.linux-amd64]# cp prometheus.yml /etc/prometheus/

  
#为了方便管理，创建prometheus.service 文件，使用systemd 去管理

[root@prometheus ~]# vim /etc/systemd/system/prometheus.service

[Unit]

Description=Prometheus Monitoring System

Documentation=Prometheus Monitoring System

[Service]

ExecStart=/usr/local/bin/prometheus 

--config.file=/etc/prometheus/prometheus.yml 

--web.listen-address=:9090

[Install]

WantedBy=multi-user.target

[root@prometheus ~]# systemctl daemon-reload

[root@prometheus ~]# systemctl enable prometheus.service

[root@prometheus ~]# systemctl start prometheus.service


#查看服务是否运行

[root@prometheus ~]# ps -ef | grep prometheus

root 19302 1 0 11:38 ? 00:00:04 /usr/local/bin/prometheus --config.file=/etc/prometheus/prometheus.yml --web.listen-address=:9090

root 20532 16875 0 11:55 pts/1 00:00:00 grep --color=auto prometheus

登录prometheus 控制器查看

prometheus-dashboard

添加监控节点

（1）、ceph mgr prometheus 插件配置 ,开启prometheus 模块

# 启用 mgr 中 prometheus 模块

[root@node-1 ~]# ceph mgr module enable prometheus

#查看服务访问方式

[root@node-1 ~]# ceph mgr services

{

“dashboard”: “https://node-2:8443/”,

“prometheus”: “http://node-2:9283/”

}

#登录到node-2 上，查看端口

[root@node-2 ~]# netstat -ntlup | grep 9283

tcp6 0 0 :::9283 :::* LISTEN 12973/ceph-mgr

#测试返回值

[root@node-2 ~]# curl 192.168.31.12:9283/metrics

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0

# HELP ceph_mds_mem_dir_minus Directories closed

# TYPE ceph_mds_mem_dir_minus counter

ceph_mds_mem_dir_minus{ceph_daemon="mds.node-3"} 0.0

ceph_mds_mem_dir_minus{ceph_daemon="mds.node-1"} 1.0

# HELP ceph_paxos_commit_latency_count Commit latency Count

# TYPE ceph_paxos_commit_latency_count counter

ceph_paxos_commit_latency_count{ceph_daemon="mon.node-2"} 5.0

ceph_paxos_commit_latency_count{ceph_daemon="mon.node-3"} 0.0

ceph_paxos_commit_latency_count{ceph_daemon="mon.node-1"} 13949.0

# HELP ceph_mds_mem_dir_plus Directories opened

# TYPE ceph_mds_mem_dir_plus counter

ceph_mds_mem_dir_plus{ceph_daemon="mds.node-3"} 12.0

ceph_mds_mem_dir_plus{ceph_daemon="mds.node-1"} 12.0

# HELP ceph_paxos_collect_keys_count Keys in transaction on peon collect Count

# TYPE ceph_paxos_collect_keys_count counter

ceph_paxos_collect_keys_count{ceph_daemon="mon.node-2"} 1.0

ceph_paxos_collect_keys_count{ceph_daemon="mon.node-3"} 2.0

ceph_paxos_collect_keys_count{ceph_daemon="mon.node-1"} 0.0

# HELP ceph_osd_flag_norebalance OSD Flag norebalance

# TYPE ceph_osd_flag_norebalance untyped

ceph_osd_flag_norebalance 0.0

......................

ceph-metrics 数据

（2）、prometheus 配置监控节点

1、在prometheus 主配置文件中的 scrape_configs 位置添加监控节点

[root@prometheus-server ~]# cat /etc/prometheus/prometheus.yml

………………

scrape_configs:

– job_name: ‘ceph_cluster’

static_configs:

– targets: [‘192.168.31.12:9283’]

………………

2、热加载prometheus 配置：

[root@prometheus-server ~]# ps -ef | grep prometheus | grep -v color | awk '{print $2}' | xargs kill -HUP $1

3、登录prometheus 控制器中查看targets

# 浏览器-》 http://x.x.x.x:9090 -》status -》Targets

可以看到 ceph-mgr export节点的信息已经显示在prometheus 上

grafana 数据展示ceph 监控数据

（1）、安装grafana

1、配置yum源文件

[root@prometheus-server ~]# vim /etc/yum.repos.d/grafana.repo

[grafana]

name=grafana

baseurl=https://packages.grafana.com/oss/rpm

repo_gpgcheck=1

enabled=1

gpgcheck=1

gpgkey=https://packages.grafana.com/gpg.key

sslverify=1

sslcacert=/etc/pki/tls/certs/ca-bundle.crt