语言入口:简体中文 | English | 北亚博客
 
    
 
北亚Linux数据恢复

首席工程师:张宇

手机:18600440055

座机:4006505646-801

传真:4006505646-809

Email:zy@datahf.net

地址:北京市海淀区永丰基地丰慧中路7号新材料创业大厦B座205室
QQ:点击这里给我发消息
当前位置:首页>>Linux修复知识>>Linux文件误删除恢复>>正文
Linux修复知识

Postfix+ldap的性能测试及分析

Postfix+ldap的性能测试及分析

1.测试平台
PIII 933 × 2 + 512M + SCSI 18G
Postfix 2.0.0.1 + openldap 2.0.11(默认安装) + virtual 投递
2.使用HASH格式作为查询表(lookukp table)时,SMTP 注入速度大概都有5000多封信/分钟。最高能达到87.5封/s。
这个时候采用的方式为只测试信头,100个并发连接。将postfix的默认进程限制提升到500,即:default_process_limit = 500,这也加大了并发速度。其中对syslogd采用了异步写优化(在设置的目录前加-号) 也能大幅度提高速度。
mail.*              -/var/log/maillog
3.将用户信息保存在ldap里,SMTP注入速度基本都有2500-2800/分钟,再简化了lookup的方式,transport_maps采用固定文本的方式,只有virtual_mailbox_maps才用ldap,这样速度提高到3200-3500封/分钟左右。测试recipients地址固定为10个
4.在ldap里存放50万条用户记录。进行500,1000,2000地址的并发测试,结果发现SMTP注入速度和使用的用户地址多少关系不大,以下是2000地址的结果:
Postal result:
time,msg/m,KB,error,connection,SSL
10:26,3779,1516,2,1930,0
10:27,3889,1556,0,1953,0
10:28,3987,1592,0,1997,0
10:29,4174,1667,0,2067,0
10:30,4308,1720,0,2166,0
10:31,4039,1613,0,1986,0
10:32,2660,1061,0,1338,0
10:33,2436,972,0,1216,0
SMTP注入速度比较快,这里信件内容只包含了信头,采用50个并发连接,每个连接只发2-3封信。感觉效果还是不错的。不过10:32之后速度开始有点问题了,我检查了/var/spool/postfix目录,发现incoming的数量大幅提高,估计是cleanup不能那么快处理incoming的信件,使队列堆积起来;但也可能是ldap反应速度开始变慢。
小结:性能优化技巧·关闭ldap的log使之=0
·将maillog等需要频繁进行write/sync的日志设置为async write,方法见上文
·调整postfix的default_process_limit > 150
·调整/var/spool/postfix目录的属性为async,使用chattr命令
5. 50个并发2000地址,邮件只包含信头,每个连接3封信的结果:

11:17,3613,1447,0,1849,0
11:18,3876,1547,0,1930,0
11:19,4088,1634,0,2080,0
11:20,3920,1566,0,1977,0
11:21,4057,1621,0,2021,0
11:22,3946,1573,0,1960,0
11:23,3316,1324,0,1695,0
11:24,2576,1030,0,1295,0
11:25,2889,1155,0,1439,0
^^^^^^^^^^^^^^^^^^^^^^*
11:26,3984,1596,0,1990,0
11:27,3986,1588,0,1993,0
11:28,4164,1662,0,2064,0
11:29,3973,1591,0,1986,0
11:30,3991,1592,0,1982,0
11:31,3521,1406,0,1758,0
11:32,2590,1035,0,1297,0
^^^^^^^^^^^^^^^^^^^^^^*
11:33,2655,1058,0,1322,0
11:34,2745,1097,0,1348,0
注意^^*所标记的时间段,SMTP注入性能大幅度下降。检查/var/spool/postfix目录,发现qmgr已达到处理上限了,该上限由qmgr_message_active_limit 参数设置。故此incoming开始增长很多,Postfix开始放慢处理速度。
以上所有测试都是在default_destination_concurrency_limit = 10的条件下完成的,所以在virtual进行本地mailbox投递时,并发数很少,而active目录增长得很快,没有足够多的virtual进行投递,结果是无法快速地将邮件写到硬盘。
6.提升default_destination_concurrency_limit:
default_destination_concurrency_limit = 1000
此时出现另外一个问题:active队列大概只维持在200-300百封信的样子,但defer/defered目录开始有所增长,并且SMTP注入速度下降,大概只有1850封信/分钟的速度。此时virtual的并发数目达到了400个!
可以推断,这是因为virtual投递速度和数量大幅度增加,致使磁盘写(write)操作过于频繁,使磁盘I/O性能吃紧,导致性能下降。
解决办法:使用分布式存储,利用mailswitch这一类技术,将最终的邮件投递由存储机器上的软件完成。
将这个default_destination_concurrency_limit改成100,速度提升了一些,但很多出现了很多lookup failure:
time,messages,data(K),errors,connections,SSL connections
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
11:57,2773,1124,2,1459,0
11:58,2302,920,0,1156,0
11:59,2492,996,0,1264,0
12:00,2204,879,0,1087,0
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
12:01,2311,924,2,1146,0
Server error:451 : Temporary lookup failure
.
12:02,2487,993,1,1265,0
12:03,2284,912,0,1112,0
12:04,1515,602,81,808,0
              ^^ 81个错误
12:05,1969,787,11,1012,0
12:06,2139,858,0,1073,0
12:07,2174,868,0,1095,0
12:08,2214,884,0,1093,0
12:09,2252,900,0,1118,0
Server error:451 : Temporary lookup failure
并发降至50:
default_destination_concurrency_limit = 50
测试的log:
Server error:451 : Temporary lookup failure
.
12:21,2645,1064,2,1379,0
12:22,3033,1211,0,1503,0
12:23,2996,1206,0,1472,0
Server error:451 : Temporary lookup failure
.
12:24,2911,1152,1,1455,0
12:25,2716,1093,0,1359,0
12:26,2844,1131,0,1434,0
12:27,2748,1096,0,1360,0
12:28,2738,1107,0,1396,0
12:29,2652,1051,0,1352,0
12:30,2759,1108,0,1355,0
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
12:31,2472,987,13,1253,0
Postal测试时打开100个并发,每个邮件最大10k,每个连接1封,发5000封,产生了非常多的failure,不过速度还不错。
接下来Postal 使用50并发,每并发1个邮件,打开了openLDAP cache支持:
time,messages,data(K),errors,connections,SSL connections
23:03,3540,19196,46,3685,0
23:04,4246,23136,0,4247,0
23:05,4182,22873,19,4194,0
                ^^19个错误
23:06,3736,19922,74,3808,0
                ^^ 70多个错误!
23:33,3683,3359,0,3733,0
23:34,3935,3632,0,3935,0
23:35,4041,3664,0,4041,0
23:36,3990,3636,0,3989,0
23:37,4164,3798,0,4165,0
23:38,3872,3512,0,3872,0
23:39,3210,2945,0,3209,0
23:40,2624,2400,0,2625,0
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
23:41,2502,2286,3,2505,0
23:42,2760,2508,0,2759,0
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
23:43,2528,2305,2,2531,0
23:44,2078,1972,0,2078,0
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
23:45,1520,1387,4,1524,0
23:46,1715,1536,0,1715,0
23:47,1475,1362,0,1475,0
23:48,1508,1371,0,1507,0
可见active目录的上限已经达到了,所以SMTP注入速度开始下降,邮件已经开始处理不过来了。
调整Postal 参数,使用50并发,每个并发1个msg,尽最大能力发送,打开openLDAP cache模式,qmgr_messages_active_limit=50000,测试持续了近一个小时,日志如下:
[root@ns1 postal-0.61]# ./do.sh
time,messages,data(K),errors,connections,SSL connections
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
23:52,3663,3353,3,3714,0
23:53,3784,3454,0,3786,0
23:54,3747,3409,0,3747,0
23:55,3861,3498,0,3861,0
23:56,3922,3577,0,3922,0
23:57,3822,3491,0,3822,0
23:58,3748,3429,0,3747,0
23:59,3608,3264,0,3609,0
00:00,3320,3027,0,3319,0
00:01,3410,3117,0,3411,0
00:02,3317,3042,0,3316,0
00:03,3580,3286,0,3581,0
00:04,3507,3205,0,3507,0
00:05,3475,3114,0,3475,0
00:06,3259,2995,0,3258,0
00:07,3186,2912,0,3186,0
00:08,3552,3216,0,3553,0
00:09,2887,2635,0,2887,0
00:10,2517,2292,0,2517,0
00:11,2721,2505,0,2721,0
00:12,2574,2317,0,2574,0
00:13,2587,2357,0,2587,0
00:14,2573,2342,0,2573,0
00:15,2704,2463,0,2704,0
00:16,2647,2405,0,2647,0
00:17,2585,2383,0,2585,0
00:18,2421,2235,0,2421,0
00:19,2573,2349,0,2572,0
00:20,2378,2170,0,2378,0
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
...... 期间Temporary lookup failure有12个
00:21,2330,2144,15,2346,0
00:22,2610,2376,0,2610,0
00:23,2551,2317,0,2550,0
00:24,2586,2380,0,2587,0
00:25,2530,2282,0,2528,0
00:26,2558,2306,0,2560,0
00:27,2444,2232,0,2444,0
00:28,2509,2250,0,2508,0
00:29,2523,2293,0,2524,0
00:30,2445,2210,0,2445,0
00:31,2555,2345,0,2554,0
00:32,2455,2225,0,2456,0
00:33,2477,2266,0,2476,0
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
00:34,2236,2038,2,2238,0
00:35,2245,2061,0,2245,0
00:36,2536,2311,0,2537,0
00:37,2210,2004,0,2210,0
00:38,2385,2150,0,2385,0
00:39,2463,2242,0,2463,0
00:40,2496,2291,0,2495,0
00:41,2471,2261,0,2471,0
00:42,2422,2208,0,2422,0
00:43,2370,2159,0,2371,0
00:44,2387,2197,0,2386,0
00:45,2203,2028,0,2203,0
00:46,2494,2301,0,2494,0
00:47,2313,2133,0,2314,0
00:48,2255,2016,0,2255,0
00:49,2428,2236,0,2428,0
00:50,2327,2113,0,2326,0
00:51,2436,2244,0,2437,0
00:52,2334,2127,0,2334,0
00:53,2399,2202,0,2398,0
00:54,2271,2078,0,2272,0
00:55,2255,2046,0,2255,0
00:56,2397,2205,0,2397,0
00:57,2392,2183,0,2392,0
00:58,2364,2161,0,2364,0
00:59,2241,2031,0,2240,0
01:00,2354,2137,0,2355,0
01:01,2223,2000,0,2223,0
01:02,2326,2118,0,2325,0
01:03,2343,2155,0,2344,0
01:04,2388,2209,0,2388,0
队列里积压了13万封信,70多分钟内注入了22万封信,正确投递并写入用户$HOME目录的有95000多封,由于人为限制了投递的并发数,因此这个结果也在意料之内了。
[root@ns2 postfix]# postsuper -d ALL
postsuper: Deleted: 137410 messages
[root@ns2 postfix]# du -sk /home/domains/bigmail.hzqbbc.com/
392124  /home/domains/bigmail.hzqbbc.com
[root@ns2 postfix]# find /home/domains/bigmail.hzqbbc.com/ | wc -l
95349
6. 新一轮测试
Postal 使用50/100个并发,1-3封邮件/每连接,将Postfix升级到Snapshot 2.0.2,并打开proxymap daemon,做如下的测试:

[root@ns1-bjcnc postal-0.61]# ./do.sh  
time,messages,data(K),errors,connections,SSL connections
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
14:02,3743,3366,3,1880,0
14:03,4244,3809,0,2129,0
14:04,4345,3915,0,2204,0
14:05,4187,3790,0,2114,0
[root@ns1-bjcnc postal-0.61]# ./do.sh
time,messages,data(K),errors,connections,SSL connections
14:10,4380,3973,5,2289,0
14:11,4481,4065,0,2215,0
14:12,4541,4083,0,2260,0
.
14:13,2741,2455,14,1389,0
14:14,3927,3549,0,1949,0
14:15,3611,3235,0,1807,0
14:16,3634,3304,0,1796,0
14:17,3889,3475,0,1933,0
[root@ns1-bjcnc postal-0.61]# ./do.sh
time,messages,data(K),errors,connections,SSL connections
14:22,1838,1644,0,961,0
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
14:23,1910,1721,2,959,0
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
14:24,1989,1769,2,1007,0
Server error:451 : Temporary lookup failure
.
Server error:451 : Temporary lookup failure
.
14:25,1828,1652,2,915,0
[root@ns1-bjcnc postal-0.61]# ./do.sh
Can't open config file "conver".  Doing no expansion.
time,messages,data(K),errors,connections,SSL connections
Server error:451 : Temporary lookup failure
.
14:27,4110,3721,1,2105,0
14:28,289,221,0,149,0
[root@ns1-bjcnc postal-0.61]# cat do.sh
#!/bin/sh
MAXMSGSIZE=1
PROC=50
MSGPERCONN=3
MSGPERMIN=5000
SSLPERCENT=0
SMTP_HOST=210.82.193.91
./postal -m $MAXMSGSIZE -p $PROC -c $MSGPERCONN -r $MSGPERMIN -a \
  -b netscape $SMTP_HOST myuser conver
[root@ns1-bjcnc postal-0.61]# ./do.sh
time,messages,data(K),errors,connections,SSL connections
.
14:38,4213,3793,10,2176,0
14:39,4308,3869,0,2174,0
14:40,4296,3835,0,2160,0
14:41,4261,3850,0,2136,0
14:42,4342,3930,0,2168,0
14:43,4258,3843,0,2139,0
14:44,4172,3769,0,2067,0
14:45,4111,3684,0,2052,0
14:46,3920,3577,0,1961,0
14:47,3987,3590,0,1987,0
14:48,3729,3365,0,1851,0
14:49,3702,3318,0,1842,0
14:50,3745,3371,0,1879,0
14:51,3961,3558,0,1969,0
14:52,3753,3415,0,1876,0
14:53,3498,3139,0,1781,0
14:54,3337,3001,0,1661,0
14:55,2671,2398,0,1341,0
14:56,2722,2453,0,1365,0
14:57,2648,2388,0,1342,0
14:58,2466,2199,7,1227,0
14:59,2601,2344,0,1299,0
通过上述多次测试表明,在打开proxymap daemon后,SMTP注入速度明显提高,结果是提升了10%-15%。
接下来将virtual投递并发限制
[root@ns1-bjcnc postal-0.61]# ./do.sh
time,messages,data(K),errors,connections,SSL connections
20:44,4289,3902,0,2166,0
20:45,4329,3874,0,2169,0
Server error:451 : Temporary lookup failure
.
期间产生12个Temporary lookup failure
20:46,3993,3594,13,1983,0
20:47,4422,4018,0,2216,0
20:48,4639,4163,0,2323,0
以上部分是final_destination 限制在20个并发virtual
========================================
接着改成10个,速度提升。说明bottleneck仍然在disk i/o(投递多了就明显影响i/o)ldap仍然有潜力可挖掘。
[root@ns1-bjcnc postal-0.61]# ./do.sh
Can't open config file "conver". Doing no expansion.
time,messages,data(K),errors,connections,SSL connections
20:53,5172,4690,0,2636,0
20:54,5074,4530,0,2529,0
20:55,4988,4529,0,2494,0
20:56,4856,4376,0,2456,0
20:57,5114,4602,0,2530,0
20:58,4720,4251,0,2345,0
20:59,4490,4038,0,2230,0
21:00,4716,4234,0,2363,0
21:01,4355,3931,0,2188,0
21:02,4488,4046,0,2241,0
21:03,4495,4062,0,2271,0
21:04,4680,4224,0,2363,0
21:05,4295,3882,0,2141,0
21:06,3822,3436,0,1921,0
21:07,2899,2597,0,1425,0
21:08,2863,2587,0,1447,0
完整结果如下:
[root@ns1-bjcnc postal-0.61]# ./do.sh
Can't open config file "conver". Doing no expansion.
time,messages,data(K),errors,connections,SSL connections
20:53,5172,4690,0,2636,0
20:54,5074,4530,0,2529,0
20:55,4988,4529,0,2494,0
20:56,4856,4376,0,2456,0
20:57,5114,4602,0,2530,0
20:58,4720,4251,0,2345,0
20:59,4490,4038,0,2230,0
21:00,4716,4234,0,2363,0
21:01,4355,3931,0,2188,0
21:02,4488,4046,0,2241,0
21:03,4495,4062,0,2271,0
21:04,4680,4224,0,2363,0
21:05,4295,3882,0,2141,0
21:06,3822,3436,0,1921,0
21:07,2899,2597,0,1425,0
21:08,2863,2587,0,1447,0
21:09,2887,2600,0,1473,0
21:10,2772,2494,0,1372,0
21:11,2106,1896,0,1033,0
21:12,1188,1079,0,584,0
21:13,1493,1355,0,742,0
21:14,1696,1540,0,828,0
21:15,1860,1677,0,920,0
21:16,1581,1428,0,784,0
21:17,2022,1837,0,1018,0
21:18,1477,1317,0,735,0
21:19,1511,1375,0,760,0
21:20,1877,1691,0,950,0
21:21,1459,1314,0,725,0
21:22,1494,1321,0,740,0
21:23,1526,1386,0,756,0
21:24,1309,1188,0,643,0
21:25,1446,1288,0,751,0
21:26,1396,1252,0,722,0
21:27,1431,1260,0,713,0
21:28,1312,1199,0,663,0
21:29,1290,1149,0,658,0
21:30,1190,1078,0,608,0
21:31,949,864,0,464,0
21:32,969,875,0,484,0
21:33,1210,1095,0,625,0
21:34,1384,1255,0,681,0
21:35,1332,1207,0,645,0
21:36,1018,928,0,503,0
21:37,882,795,0,442,0
21:38,1222,1095,0,613,0
21:39,1171,1051,0,581,0
21:40,1222,1123,0,609,0
21:41,1223,1088,0,606,0
21:42,1220,1086,0,607,0
21:43,1328,1208,0,665,0
21:44,1409,1263,0,719,0
21:45,1488,1349,0,723,0
21:46,1237,1115,0,628,0
21:47,1160,1035,0,586,0
21:48,1270,1146,0,626,0
21:49,1400,1261,0,708,0
到了21:10开始,actvie已接近上限,处理速度开始来不及了。此外,postal里限制了一分钟只发5000封信,所以本次测试的峰值51xx信/分未必可信,应能提高更多。
提高注入速度的新测试笔者根据在IBM developerWorks上得到的灵感,将Postfix queue的所在的磁盘分区挂接参数(mount)改为noatime,这样实际减少了磁盘的写入操作(noatime表示对于访问操作,不更新文件的atime记录,这可减少磁盘的write操作),提高了5%的性能。
通过elvtune调整了磁盘write操作的延时,可明显提高速度。
注意使用linux的iostat及相关的I/O监视工具。。以下是调整后的数值:
time,messages,data(K),errors,connections,SSL connections
23:57,5325,4804,0,2703,0
23:58,5480,4905,0,2702,0
23:59,5160,4624,0,2573,0
00:00,5211,4689,0,2609,0
00:01,4918,4441,0,2453,0
00:02,4872,4377,0,2395,0
00:03,5039,4564,0,2492,0
00:04,4694,4223,0,2308,0
效果有了明显进展。而再进行多次测试发现,有些时段可达到峰值63xx封/分,估计这因为是Postfix的Multiplex技术及磁盘I/O的cache所带来的收益,multiplex可复用已打开的smtp通道,减少了进程的创建带来的开销。
[root@ns1-bjcnc postal-0.61]# ./do.sh

本新闻共2页,当前在第1页  1  2  

上一篇:linux下安装mysql
下一篇:没有了
JFS文件系统简介
XFS文件系统简介
Reiserfs文件系统简介
EXT3文件系统简介
常见Linux文件系统简介
GRUB简介与配置
LILO的使用和配置
Linux单用户模式维护系统
引导加载程序简介
重置Linux用户口令
  版权所有@北京北亚时代科技有限公司   京ICP备09039053号    
  总部电话:(010)82488636  中国·北京·海淀区中关村E世界C座8层879室               站点地图   联系我们  RSS阅读