flume日志、zookeeper偏移量查询与设置

xiaoxiao2025-04-05  9

1.查询flume日志(INFO,ERROR) 法1:(在网页cloudera manager-flume-配置-类别-日志-Flume Agent 日志目录 找/lvm/log/flume-ng) http://10.2.5.65:7180/cmf/process/all/logs/search 法2:在命令行 [root@hadoop6 flume-ng]# cd /lvm/log/flume-ng [root@hadoop6 flume-ng]# cat flume-cmf-flume-AGENT-hadoop6.log |grep sideward_systemdata (所有日志中筛选需要检查的表的日志 >=IFNO级别) [root@hadoop6 flume-ng]# cat flume-cmf-flume-AGENT-hadoop6.log |grep -i error |grep sideward_systemdata (所有日志中筛选错误的,再在错误中筛选需要检查的表的日志,-i表示不区分大小写) [root@hadoop6 flume-ng]# tail -f flume-cmf-flume-AGENT-hadoop6.log (最近10条日志,实时更新) 2.1 查看zookeeper 中保存的kafka的偏移量 [root@hadoop6 data]# zookeeper-client [zk: localhost:2181(CONNECTED) 8] get /consumers/BusiSystem2/offsets/BusiSystem/0 //获取所有消费者中消费者组 为BusiSystem2且topic为BusiSystem的第0个分区 的偏移量。 422459392 (这个东西就是当前消费到的位置) cZxid = 0x1600a8df29 ctime = Thu Jun 22 11:21:56 CST 2017 mZxid = 0x19000c52b3 mtime = Thu May 24 13:53:50 CST 2018 pZxid = 0x1600a8df29 cversion = 0 dataVersion = 163881 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 9 numChildren = 0 [zk: localhost:2181(CONNECTED) 11] ls /consumers/FaultData4/offsets/FaultData [0] [zk: localhost:2181(CONNECTED) 12] get /consumers/FaultData4/offsets/FaultData/0 683 cZxid = 0x1d000b0b31 ctime = Thu Aug 09 19:28:25 CST 2018 mZxid = 0x1d000b194b mtime = Thu Aug 09 21:06:45 CST 2018 pZxid = 0x1d000b0b31 cversion = 0 dataVersion = 1 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 3 numChildren = 0 #将消费者为FaultData4,topic为FaultData的第0个分区的偏移量重新设置为600(本来是683) [zk: localhost:2181(CONNECTED) 15] set /consumers/FaultData4/offsets/FaultData/0 600 cZxid = 0x1d000b0b31 ctime = Thu Aug 09 19:28:25 CST 2018 mZxid = 0x1d000b83aa mtime = Fri Aug 10 09:13:05 CST 2018 pZxid = 0x1d000b0b31 cversion = 0 dataVersion = 2 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 3 numChildren = 0

 

转载请注明原文地址: https://www.6miu.com/read-5027538.html

最新回复(0)