linuxea: lsyncd快速配置安装


Lsyncd-实时同步相比较之前的rsync同步的方式,他可以通过ssh的方式进行同步,这也是为什么要写这篇笔记的原因。通常而言,越简单越高效的东西都有必要和价值来进行分享。

rsync+ssh配置起来更加复杂,ssh操作文件和目录。通过配置文件来做自定义配置,使用shell脚本和lua语言编写。配置简单,灵活,详细查看官网手册:https://axkibe.github.io/lsyncd/

lsyncd2.2.1需要在源节点和目标节点安装rsync>=3.1。这个需求通常来讲系统自带的就足够了。

因此,我们要安装的依赖包就包括lua,liblua,以及cmake:yum install lua lua-devel cmake -y

I. 克隆代码

[root@linuxea_node ~]# git clone https://github.com/axkibe/lsyncd.git /usr/local/lsyncd
Cloning into '/usr/local/lsyncd'...
remote: Enumerating objects: 4333, done.
remote: Total 4333 (delta 0), reused 0 (delta 0), pack-reused 4333
Receiving objects: 100% (4333/4333), 1.76 MiB | 780.00 KiB/s, done.
Resolving deltas: 100% (2577/2577), done.

II. 安装依赖包

安装几个依赖包

[root@linuxea_node ~]# cd /usr/local/lsyncd
[root@linuxea_node /usr/local/lsyncd]# yum install lua lua-devel cmake -y

III. 编译安装

安装完成后进行编译即可

[root@linuxea_node /usr/local/lsyncd]# cmake .
-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Found Lua: /usr/lib64/liblua.so;/usr/lib64/libm.so (found version "5.1.4") 
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/local/lsyncd
[root@linuxea_node /usr/local/lsyncd]# make
[ 12%] Generating defaults.out
Compiling built-in default configs
[ 25%] Generating runner.out
Compiling built-in runner
[ 37%] Generating runner.c
Generating built-in runner linkable
[ 50%] Generating defaults.c
Generating built-in default configs
Scanning dependencies of target lsyncd
[ 62%] Building C object CMakeFiles/lsyncd.dir/lsyncd.c.o
[ 75%] Building C object CMakeFiles/lsyncd.dir/runner.c.o
[ 87%] Building C object CMakeFiles/lsyncd.dir/defaults.c.o
[100%] Building C object CMakeFiles/lsyncd.dir/inotify.c.o
Linking C executable lsyncd
[100%] Built target lsyncd
[root@linuxea_node /usr/local/lsyncd]# make install
[100%] Built target lsyncd
Install the project...
-- Install configuration: ""
-- Installing: /usr/local/bin/lsyncd
-- Installing: /usr/local/man/lsyncd.1

IV. ssh互通

通常而言,我们需要打通目标节点

在目标节点放行源节点的端口,假如是iptables,就如下添加
iptables -I INPUT 5 -s 172.25.110.49 -p tcp -m tcp -m state --state NEW -m multiport --dports 22,2222 -m comment --comment "ssh" -j ACCEPT

在源节点复制key到目标节点

[root@linuxea_node /usr/local/lsyncd]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:0vgvn8tziq24dxuhg0yLTsVl3OGQNakxeWJUIUWNJPg root@172_30_2_44.cluster.com
The key's randomart image is:
+---[RSA 2048]----+
|       o=OX=     |
|      ..B*+o.    |
|       o+*o      |
|     . =E        |
|      * S .      |
|     = = . .     |
|    o + + .      |
|   o   .o*oo.    |
|    . ooo=X*     |
+----[SHA256]-----+
[root@linuxea_node /usr/local/lsyncd]# ssh-copy-id 172.25.110.50
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '172.25.110.50 (172.25.110.50)' can't be established.
ECDSA key fingerprint is SHA256:3BVoJtcE0SnDZN3NA9tRMIoiae7d4vi0FkX6ZltAixE.
ECDSA key fingerprint is MD5:e0:75:bd:aa:75:13:c7:e0:d6:2f:21:24:1a:45:42:f1.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@172.25.110.50's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '172.25.110.50'"
and check to make sure that only the key(s) you wanted were added.

V. 配置文件

我们将nginx的配置文件实时镜像到172.25.110.50上,配置文件如下

[root@DT_Node-172_25_244_4 ~]# cat /usr/local/etc/lsyncd.conf 
-- this file is sync from cserver
settings {
    logfile ="/var/log/nginx_conf_lsyncd.log",
    statusFile ="/var/run/nginx_conf_lsyncd.status",
    inotifyMode = "CloseWrite",
    maxProcesses = 1,
    insist = true,
    }
servers = {
    -- "主机位置"
    "172.25.110.50", -- server2
}
for _, server in ipairs(servers) do
sync {
    default.rsyncssh,
    source="/usr/local/nginx/conf",
    host=server,
    targetdir="/usr/local/nginx/conf",
    maxDelays=5,
    delay=0,
    delete="true",
    exclude={ ".*", "*.tmp" },
    rsync = {
        binary="/usr/bin/rsync",
        archive = true,
        compress = true,
        verbose = true,
        rsh = "/usr/bin/ssh -p 22992 -o StrictHostKeyChecking=no"
        }
    }
end

启动

[root@linuxea_node /usr/local/nginx]# lsyncd -log Exec /usr/local/etc/lsyncd.conf 
03:25:20 Normal: --- Startup, daemonizing ---

而后,我们清空日志,手动添加文件测试

[root@linuxea_node /usr/local/nginx]# > /var/log/nginx_conf_lsyncd.log
[root@linuxea_node /usr/local/nginx]# tail -f /var/log/nginx_conf_lsyncd.log

touch一个text.txt

root@linuxea_node ~]# touch /usr/local/nginx/conf/test.txt

日志已经显示同步

[root@linuxea_node /usr/local/nginx]# tail -f /var/log/nginx_conf_lsyncd.log 
Sat Mar 21 03:27:19 2020 Normal: Calling rsync with filter-list of new/modified files/dirs
/test.txt
/
Sat Mar 21 03:27:19 2020 Exec: /usr/bin/rsync [<] [/test.txt
/] [-gvzsolptD] [--rsh=/usr/bin/ssh -p 22992 -o StrictHostKeyChecking=no] [-r] [--force] [--from0] [--include-from=-] [--exclude=*] [/usr/local/nginx/conf/] [172.25.110.50:/usr/local/nginx/conf/]
Sat Mar 21 03:27:19 2020 Exec: one-sweeped pipe
sending incremental file list
./
test.txt

sent 106 bytes  received 38 bytes  288.00 bytes/sec
total size is 0  speedup is 0.00
Sat Mar 21 03:27:19 2020 Normal: Finished (list): 0

可以通过远程查看

[root@linuxea_node ~]# ssh 172.25.110.50 ls /usr/local/nginx/conf/te*
/usr/local/nginx/conf/test.txt

VI. 配置文件

  • 全局设置
logfile         定义日志文件
stausFile       定义状态文件
statusInterval  将lsyncd的状态写入上面的statusFile的间隔,默认10秒
nodaemon=true   表示不启用守护模式,默认
inotifyMode     指定inotify监控的事件,默认是CloseWrite,还可以是Modify或CloseWrite or Modify
maxProcesses    同步进程的最大个数。假如同时有20个文件需要同步,而maxProcesses = 8,则最大能看到有8个rysnc进程
maxDelays       累计到多少所监控的事件激活一次同步,即使后面的delay延迟时间还未到

定义同步参数,可以继续使用maxDelays来重写settings的全局变量。

模式运行

default.rsync       本地目录间同步,使用rsync,也可以达到使用ssh形式的远程rsync效果,或daemon方式连接远程rsyncd进程;
default.direct      本地目录间同步,使用cp、rm等命令完成差异文件备份;
default.rsyncssh    同步到远程主机目录,rsync的ssh模式,需要使用key来认证

目录设置

source                      同步的源目录,使用绝对路径。
target                      定义目的地址.对应不同的模式有几种写法:
/tmp/dest                   本地目录同步,可用于direct和rsync模式
172.25.100.10:/tmp/dest     同步到远程服务器目录,可用于rsync和rsyncssh模式
excludeFrom 排除选项,后面指定排除的列表文件,如excludeFrom = "/etc/lsyncd.exclude",如果是简单的排除,可以使用exclude = LIST。

这里的排除规则写法与原生rsync有点不同,更为简单:

监控路径里的任何部分匹配到一个文本,都会被排除,例如foo可以匹配/bin/foo/bar
如果规则以斜线/开头,则从头开始要匹配全部
如果规则以/结尾,则要匹配监控路径的末尾
?匹配单个任何字符,但不包括/
*匹配0或多个字符,但不包括/
**匹配0或多个字符,可以是/

delay

delay           累计事件,等待rsync同步延时时间,默认15秒(最大累计到1000个不可合并的事件)。也就是15s内监控目录下发生的改动,会累积到一次rsync同步,避免过于频繁的同步。(可合并的意思是,15s内两次修改了同一文件,最后只同步最新的文件)

delete 为了保持target与souce完全同步,Lsyncd默认会delete = true来允许同步删除。它除了false,还有startup、running

true        Lsyncd将在目标上删除任何不在源中的内容。 在启动时和正常操作中被删除的内容。
false       在lsyncd启动后将在目标上不删除任何不在源中的内容, 在启动时和正常操作中被删除的内容。
startup     启动时将执行一次完全文件同步,保证完全一致;正常运行过程中不会删除target中的文件
running     启动前,增加的会同步,删除的不同步;正常运行过程中会删除target中的文件
bwlimit         限速,单位kb/s,与rsync相同
compress        压缩传输默认为true。在带宽与cpu负载之间权衡,本地目录同步可以考虑把它设为false
perms           默认保留文件权限。

如果你打不开官方例子,配置部分可参考中文文档,或者其他

0 分享

您可以选择一种方式赞助本站

支付宝扫码赞助

支付宝扫码赞助

日期: 2020-03-21分类: rsync

标签: rsync

发表评论