Приветствую.Есть cisco 2811. Держит два туннеля. По туннелям передается голос(три колцентра)
1ая точка - Колцентр Cisco.
2ая точка - Колцентр Avaya, клиентские места аваевского и цисковского колцентра(~10 раб мест)
3яя точка - вынос Аваевского колцентра, клиентские места аваевского и цисковского колцентра.(~20+ раб мест)cisco 2811 стоит на 2 точке и держит туннель с ISA 2006(1ая точка) и на 3й точке с juniper ssg.
show proc cpu hist - показывает 99% загрузку.
Даже на внутреннем интерфейсе циски потери пакетов и пинги в районе 100-200мс.Через циску ходит только голос, всё остальное ходит через другую проксю.
Помогите, может чего поправить надо ? или без замены 2811 на что то другое не обойтись ?
----------------------------------------------------------------------
Если что, конфиг:
cisco-n-a-c#sh run
Building configuration...Current configuration : 4544 bytes
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname cisco-n-a-c
!
boot-start-marker
boot system flash c2800nm-advipservicesk9-mz.124-21a.bin
boot-end-marker
!
logging buffered 4096 debugging
!
aaa new-model
!
aaa authentication login default local
aaa authentication ppp default local
!
aaa session-id common
!
ip cef
!
ip auth-proxy max-nodata-conns 3
ip admission max-nodata-conns 3
!
voice-card 0
no dspfarm
!
crypto isakmp policy 10
encr 3des
authentication pre-share
group 2
lifetime 28800
crypto isakmp key xxxxxxxxxx address 94.xx.xx.xx
crypto isakmp key xxxxxxxxxx address 81.xx.xx.xx
!
crypto ipsec transform-set des-sha esp-des esp-sha-hmac
crypto ipsec transform-set 3des-sha esp-3des esp-sha-hmac
!
crypto ipsec profile VTI
set transform-set 3des-sha
set pfs group2
!
!
crypto map IPSEC_TUNNEL 20 ipsec-isakmp
set peer 81.xx.xx.x
set transform-set 3des-sha
set pfs group2
match address SB
!
interface Loopback1
no ip address
!
interface Tunnel1
ip unnumbered FastEthernet0/1
ip accounting output-packets
ip mtu 1400
tunnel source 89.28.xx.xx
tunnel destination 94.xx.xx.xx
tunnel mode ipsec ipv4
tunnel protection ipsec profile VTI
!
interface FastEthernet0/0
description --- WAN ---
ip address 89.28.xx.xx 255.255.255.224
ip access-group BlockFromInternet in
ip accounting output-packets
ip nat outside
ip virtual-reassembly
duplex auto
speed auto
crypto map IPSEC_TUNNEL
!
interface FastEthernet0/1
description --- LAN ---
ip address 192.168.104.254 255.255.255.0 secondary
ip address 192.168.103.254 255.255.255.0 secondary
ip address 192.168.101.254 255.255.255.0 secondary
ip address 192.168.3.254 255.255.255.0
ip accounting output-packets
ip flow ingress
ip flow egress
ip nat inside
ip virtual-reassembly
duplex auto
speed auto
!
ip local pool PPTP xxxxxxxxxxxxxxxxxxxxx
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 89.28.xxx.xxx
ip route 192.168.1.0 255.255.255.0 87.226.231.126
ip route 192.168.2.0 255.255.255.0 87.226.231.126
ip route 192.168.4.0 255.255.254.0 Tunnel1
ip route 192.168.52.0 255.255.255.0 Tunnel1
ip route 192.168.102.0 255.255.255.0 192.168.3.251
!
!
no ip http server
no ip http secure-server
ip nat inside source list NAT interface FastEthernet0/0 overload
ip nat inside source static tcp 192.168.3.1 22 89.28.xx.xx 22 extendable
ip nat inside source static tcp 192.168.3.1 80 89.28.xx.xx 80 extendable
ip nat inside source static tcp 192.168.3.1 443 89.28.xx.xx 443 extendable
ip nat inside source static tcp 192.168.3.35 3389 89.28.xx.xx 3390 extendable
ip nat inside source static tcp 192.168.3.9 3389 89.28.xx.xx 4083 extendable
ip nat inside source static tcp 192.168.3.50 3389 89.28.xx.xx 4084 extendable
ip nat inside source static tcp 192.168.3.1 5022 89.28.xx.xx 5022 extendable
!
ip access-list extended BlockFromInternet
............................
ip access-list extended NAT
..............................
ip access-list extended SB
permit ip 192.168.3.0 0.0.0.255 192.168.2.0 0.0.0.255 log
permit ip 192.168.2.0 0.0.0.255 192.168.3.0 0.0.0.255 log
permit ip 192.168.3.0 0.0.0.255 192.168.1.0 0.0.0.255 log
permit ip 192.168.1.0 0.0.0.255 192.168.3.0 0.0.0.255 log
permit ip 192.168.4.0 0.0.0.255 192.168.2.0 0.0.0.255 log
permit ip 192.168.2.0 0.0.0.255 192.168.4.0 0.0.0.255 log
permit ip 192.168.4.0 0.0.0.255 192.168.1.0 0.0.0.255 log
permit ip 192.168.1.0 0.0.0.255 192.168.4.0 0.0.0.255 log
!
access-list
............................................
!
control-plane
!
line con 0
line aux 0
line vty 0 4
access-class 1 in
!
scheduler allocate 20000 1000
!
endcisco-n-a-c#
Что именно грузит рутер?
Покажите в момент нагрузки
sh proc cpu sort 1 | exc 0.00
sh int fa0/какой?
sh ip traffic | i fra
> Что именно грузит рутер?
> Покажите в момент нагрузки
> sh proc cpu sort 1 | exc 0.00
> sh int fa0/какой?
> sh ip traffic | i fracisco-n-a-c#sh proc cpu sort 1 | exc 0.00
CPU utilization for five seconds: 99%/85%; one minute: 99%; five minutes: 98%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
180 103731896 82248934 1261 2.65% 2.69% 2.79% 0 Crypto PAS Proc
230 532 246 2162 0.16% 0.11% 0.03% 322 Virtual Exec
5 24379300 1850105 13177 0.08% 0.08% 0.08% 0 Check heaps
36 678976 7154572 94 0.08% 0.04% 0.02% 0 Net Background
132 1493188 162959551 9 0.08% 0.02% 0.02% 0 RBSCP Background
cisco-n-a-c#sh int fa0/0
FastEthernet0/0 is up, line protocol is up
Hardware is MV96340 Ethernet, address is 001b.2aa3.8fe8 (bia 001b.2aa3.8fe8)
Description: --- WAN ---
Internet address is 89.xx.x.xxx/27
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 8/255, rxload 8/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, 100BaseTX/FX
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 4d22h
Input queue: 0/75/1347/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 3382000 bits/sec, 1600 packets/sec
5 minute output rate 3509000 bits/sec, 1650 packets/sec
254601591 packets input, 3056519813 bytes
Received 2231803 broadcasts, 0 runts, 0 giants, 411 throttles
5455 input errors, 0 CRC, 0 frame, 0 overrun, 5455 ignored
0 watchdog
0 input packets with dribble condition detected
258680351 packets output, 4124462648 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
25018 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out
cisco-n-a-c#sh int fa0/1
FastEthernet0/1 is up, line protocol is up
Hardware is MV96340 Ethernet, address is 001b.2aa3.8fe9 (bia 001b.2aa3.8fe9)
Description: --- LAN ---
Internet address is 192.168.3.254/24
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 210/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, 100BaseTX/FX
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 4d22h
Input queue: 26/75/319756/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 760000 bits/sec, 449 packets/sec
5 minute output rate 644000 bits/sec, 389 packets/sec
65303920 packets input, 237810178 bytes
Received 1220349 broadcasts, 0 runts, 0 giants, 107519 throttles
4972467 input errors, 0 CRC, 0 frame, 0 overrun, 4972467 ignored
0 watchdog
0 input packets with dribble condition detected
57703321 packets output, 3485172364 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
15 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out
cisco-n-a-c#sh ip traffic | i fra
99147 fragmented, 218131 fragments, 213 couldn't fragment
Вот на момент выполения всех этих команд загрузка процессора в течении последнего часа была 99%.
Для начала уберите:
ip accounting output-packets
С какой целью ведется подсчет?
> Для начала уберите:
> ip accounting output-packets
> С какой целью ведется подсчет?убрал )))
> Received 2231803 broadcasts, 0 runts, 0 giants,
> 411 throttles
> 5455 input errors, 0 CRC, 0 frame,
> Received 1220349 broadcasts, 0 runts, 0 giants,
> 107519 throttles
> 4972467 input errors, 0 CRC, 0 frame,эм...
ошибки на интерфейсе... попробуйте поставить speed и duplex НЕ auto, проверить кабель.так же похоже на перегрузку из за обработки большого количества прерываний... много потоков?
http://www.cisco.com/en/US/products/hw/routers/ps359/product...
"Throttles are a good indication of an overloaded router. They show the number of times the receiver on the port has been disabled, possibly due to buffer or processor overload. Together with high CPU utilization on an interrupt level, throttles indicate that the router is overloaded with traffic."
>[оверквотинг удален]
>> 411 throttles
>> 5455 input errors, 0 CRC, 0 frame,
>> Received 1220349 broadcasts, 0 runts, 0 giants,
>> 107519 throttles
>> 4972467 input errors, 0 CRC, 0 frame,
> эм...
> ошибки на интерфейсе... попробуйте поставить speed и duplex НЕ auto, проверить кабель.
> так же похоже на перегрузку из за обработки большого количества прерываний... много
> потоков?
>Ну а чего кабель? Пакеты пропадают только во время работы. Сейчас 5-10% загрузка - пинги 1мс, все нормально.
!!! clear counters по всем1. Загрузка по прерываниям 85%, причин может быть несколько:
- большое количество пакетов
- отключен или неправильно настроен cef
- большое кол-во arp-запросов
Покажите sh ip int и sh adj.2. AIM-VPN у вас на борту, видимо, нет, соответственно, шифрованием туннелей у вас занимается исключительно процессор. Варианты: упрощать механизмы шифрования.
3. clear ip traffic и через некоторое время опять sh ip traffic | i fra
как быстро увеличивается счетчик? Возможно, ввиду неверно настроенных туннелей процессор занимается еще и сборкой фрагментированных пакетов.4. Оч.много дропов на WAN fa0/0. Сделайте на интерфейсе load-interval 30, потом раз в минуту см.sh int fa0/0; интересует динамика input rate/output rate. Накопите 7-8 мин и еще раз покажите сюда.
Вообще, заявленная пропуская способность 2811 - ~61mbps, но из опыта, это значение рекордное и зависит от живущего трафика. Возможно, вы просто уперлись в её пределы.
> !!! clear counters по всемсделал
> 1. Загрузка по прерываниям 85%, причин может быть несколько:
> - большое количество пакетов
> - отключен или неправильно настроен cef
> - большое кол-во arp-запросов
> Покажите sh ip int и sh adj.cisco-nn-analitik-center#sh ip int
FastEthernet0/0 is up, line protocol is up
Internet address is 89.xx.xx.xx/27
Broadcast address is 255.255.255.255
Address determined by setup command
MTU is 1500 bytes
Helper address is not set
Directed broadcast forwarding is disabled
Outgoing access list is not set
Inbound access list is BlockFromInternet
Proxy ARP is enabled
Local Proxy ARP is disabled
Security level is default
Split horizon is enabled
ICMP redirects are always sent
ICMP unreachables are always sent
ICMP mask replies are never sent
IP fast switching is enabled
IP fast switching on the same interface is disabled
IP Flow switching is disabled
IP CEF switching is enabled
IP CEF Feature Fast switching turbo vector
IP multicast fast switching is enabled
IP multicast distributed fast switching is disabled
IP route-cache flags are Fast, CEF
Router Discovery is disabled
IP output packet accounting is disabled
IP access violation accounting is disabled
TCP/IP header compression is disabled
RTP/IP header compression is disabled
Policy routing is disabled
Network address translation is enabled, interface in domain outside
BGP Policy Mapping is disabled
WCCP Redirect outbound is disabled
WCCP Redirect inbound is disabled
WCCP Redirect exclude is disabled
FastEthernet0/0.2 is deleted, line protocol is down
Internet protocol processing disabled
FastEthernet0/1 is up, line protocol is up
Internet address is 192.168.3.254/24
Broadcast address is 255.255.255.255
Address determined by non-volatile memory
MTU is 1500 bytes
Helper address is not set
Directed broadcast forwarding is disabled
Secondary address 192.168.104.254/24
Secondary address 192.168.103.254/24
Secondary address 192.168.101.254/24
Outgoing access list is not set
Inbound access list is not set
Proxy ARP is enabled
Local Proxy ARP is disabled
Security level is default
Split horizon is enabled
ICMP redirects are always sent
ICMP unreachables are always sent
ICMP mask replies are never sent
IP fast switching is enabled
IP fast switching on the same interface is disabled
IP Flow switching is enabled
IP CEF switching is enabled
IP CEF Flow Fast switching turbo vector
IP multicast fast switching is enabled
IP multicast distributed fast switching is disabled
IP route-cache flags are Fast, Flow cache, CEF, Subint Flow
Router Discovery is disabled
IP output packet accounting is disabled
IP access violation accounting is disabled
TCP/IP header compression is disabled
RTP/IP header compression is disabled
Policy routing is disabled
Network address translation is enabled, interface in domain inside
BGP Policy Mapping is disabled
WCCP Redirect outbound is disabled
WCCP Redirect inbound is disabled
WCCP Redirect exclude is disabled
NVI0 is up, line protocol is up
Interface is unnumbered. Using address of NVI0 (0.0.0.0)
Broadcast address is 255.255.255.255
MTU is 1514 bytes
Helper address is not set
Directed broadcast forwarding is disabled
Outgoing access list is not set
Inbound access list is not set
Proxy ARP is enabled
Local Proxy ARP is disabled
Security level is default
Split horizon is enabled
ICMP redirects are always sent
ICMP unreachables are always sent
ICMP mask replies are never sent
IP fast switching is disabled
IP fast switching on the same interface is disabled
IP Flow switching is disabled
IP CEF switching is disabled
IP Null turbo vector
IP multicast fast switching is enabled
IP multicast distributed fast switching is disabled
IP route-cache flags are Fast, CEF
Router Discovery is disabled
IP output packet accounting is disabled
IP access violation accounting is disabled
TCP/IP header compression is disabled
RTP/IP header compression is disabled
Policy routing is disabled
Network address translation is disabled
BGP Policy Mapping is disabled
WCCP Redirect outbound is disabled
WCCP Redirect inbound is disabled
WCCP Redirect exclude is disabled
Virtual-Access1 is down, line protocol is down
Internet protocol processing disabled
Virtual-Access2 is up, line protocol is up
Internet protocol processing disabled
Virtual-Access3 is down, line protocol is down
Broadcast address is 255.255.255.255
MTU is 1500 bytes
Helper address is not set
Directed broadcast forwarding is disabled
Outgoing access list is not set
Inbound access list is not set
Proxy ARP is enabled
Local Proxy ARP is disabled
Security level is default
Split horizon is enabled
ICMP redirects are always sent
ICMP unreachables are always sent
ICMP mask replies are never sent
IP fast switching is enabled
IP fast switching on the same interface is enabled
IP Flow switching is disabled
IP CEF switching is enabled
IP CEF Feature Fast switching turbo vector
IP multicast fast switching is enabled
IP multicast distributed fast switching is disabled
IP route-cache flags are Fast, CEF
Router Discovery is disabled
IP output packet accounting is disabled
IP access violation accounting is disabled
TCP/IP header compression is disabled
RTP/IP header compression is disabled
Policy routing is disabled
Network address translation is disabled
BGP Policy Mapping is disabled
WCCP Redirect outbound is disabled
WCCP Redirect inbound is disabled
WCCP Redirect exclude is disabled
Loopback1 is up, line protocol is up
Internet protocol processing disabled
Tunnel1 is up, line protocol is up
Interface is unnumbered. Using address of FastEthernet0/1 (192.168.3.254)
Broadcast address is 255.255.255.255
MTU is 1400 bytes
Helper address is not set
Directed broadcast forwarding is disabled
Outgoing access list is not set
Inbound access list is not set
Proxy ARP is enabled
Local Proxy ARP is disabled
Security level is default
Split horizon is enabled
ICMP redirects are always sent
ICMP unreachables are always sent
ICMP mask replies are never sent
IP fast switching is enabled
IP fast switching on the same interface is disabled
IP Flow switching is disabled
IP CEF switching is enabled
IP CEF Feature Fast switching turbo vector
IP multicast fast switching is enabled
IP multicast distributed fast switching is disabled
IP route-cache flags are Fast, CEF
Router Discovery is disabled
IP output packet accounting is disabled
IP access violation accounting is disabled
TCP/IP header compression is disabled
RTP/IP header compression is disabled
Policy routing is disabled
Network address translation is disabled
BGP Policy Mapping is disabled
WCCP Redirect outbound is disabled
WCCP Redirect inbound is disabled
WCCP Redirect exclude is disabled
Tunnel100 is up, line protocol is up
Internet address is 10.xx.xx.xx/30
Broadcast address is 255.255.255.255
Address determined by setup command
MTU is 1400 bytes
Helper address is not set
Directed broadcast forwarding is disabled
Outgoing access list is not set
Inbound access list is not set
Proxy ARP is enabled
Local Proxy ARP is disabled
Security level is default
Split horizon is enabled
ICMP redirects are always sent
ICMP unreachables are always sent
ICMP mask replies are never sent
IP fast switching is enabled
IP fast switching on the same interface is disabled
IP Flow switching is disabled
IP CEF switching is enabled
IP CEF Feature Fast switching turbo vector
IP multicast fast switching is enabled
IP multicast distributed fast switching is disabled
IP route-cache flags are Fast, CEF
Router Discovery is disabled
IP output packet accounting is disabled
IP access violation accounting is disabled
TCP/IP header compression is disabled
RTP/IP header compression is disabled
Policy routing is disabled
Network address translation is disabled
BGP Policy Mapping is disabled
WCCP Redirect outbound is disabled
WCCP Redirect inbound is disabled
WCCP Redirect exclude is disabled
cisco-n-a-c# sh adj
Protocol Interface Address
IP FastEthernet0/1 192.168.3.107(5)
IP FastEthernet0/1 192.168.3.105(5)
IP FastEthernet0/0 89.xx.xx.xx(476)
IP Tunnel1 point2point(869351)
IP FastEthernet0/0 89.xx.xx.xx(5)
IP FastEthernet0/1 192.168.3.100(5)
IP Tunnel100 point2point(4)
IP FastEthernet0/0 88.xx.xx.xx(3) (incomplete)
IP FastEthernet0/1 192.168.3.67(5)
IP FastEthernet0/1 192.168.3.86(5)
IP FastEthernet0/1 192.168.3.49(5)
IP FastEthernet0/1 192.168.3.53(5)
IP FastEthernet0/1 192.168.3.11(5)
IP FastEthernet0/1 192.168.3.10(5)
IP FastEthernet0/1 192.168.3.9(5)
IP FastEthernet0/1 192.168.3.15(5)
IP FastEthernet0/1 192.168.3.13(5)
IP FastEthernet0/1 192.168.3.12(5)
IP FastEthernet0/1 192.168.3.2(5)
IP FastEthernet0/1 192.168.3.1(5)
IP FastEthernet0/1 192.168.3.6(5)
IP FastEthernet0/1 192.168.3.5(5)
IP FastEthernet0/1 192.168.3.27(5)
IP FastEthernet0/1 192.168.3.25(5)
IP FastEthernet0/1 192.168.3.29(5)
IP FastEthernet0/0 192.168.2.22(5)
IP FastEthernet0/1 192.168.3.22(5)
IP FastEthernet0/1 192.168.3.251(9)
IP FastEthernet0/1 192.168.3.252(5)
IP FastEthernet0/1 192.168.3.244(5)
IP FastEthernet0/0 192.11.13.5(5)> 2. AIM-VPN у вас на борту, видимо, нет, соответственно, шифрованием туннелей у
> вас занимается исключительно процессор. Варианты: упрощать механизмы шифрования.а как ?
> 3. clear ip traffic и через некоторое время опять sh ip traffic
> | i fra
> как быстро увеличивается счетчик? Возможно, ввиду неверно настроенных туннелей процессор
> занимается еще и сборкой фрагментированных пакетов.А не увеличивается он. Как сбросил, так до сих пор:
cisco-n-a-c#sh ip traffic | include fra
0 fragmented, 0 fragments, 0 couldn't fragment
> 4. Оч.много дропов на WAN fa0/0. Сделайте на интерфейсе load-interval 30, потом
> раз в минуту см.sh int fa0/0; интересует динамика input rate/output rate.
> Накопите 7-8 мин и еще раз покажите сюда.Не то чтобы прям раз в минуту, но как то так, интервалы в 1-5 минут:
cisco-n-a-c#show interfaces fa0/030 second input rate 652000 bits/sec, 330 packets/sec
30 second output rate 669000 bits/sec, 326 packets/sec
129 unknown protocol drops
30 second input rate 525000 bits/sec, 279 packets/sec
30 second output rate 549000 bits/sec, 277 packets/sec
135 unknown protocol drops30 second input rate 568000 bits/sec, 268 packets/sec
30 second output rate 905000 bits/sec, 281 packets/sec
140 unknown protocol drops
30 second input rate 481000 bits/sec, 247 packets/sec
30 second output rate 485000 bits/sec, 239 packets/sec
150 unknown protocol drops30 second input rate 774000 bits/sec, 372 packets/sec
30 second output rate 776000 bits/sec, 369 packets/sec
160 unknown protocol drops30 second input rate 473000 bits/sec, 236 packets/sec
30 second output rate 477000 bits/sec, 232 packets/sec
169 unknown protocol drops30 second input rate 494000 bits/sec, 254 packets/sec
30 second output rate 499000 bits/sec, 252 packets/sec
182 unknown protocol drops30 second input rate 489000 bits/sec, 247 packets/sec
30 second output rate 488000 bits/sec, 242 packets/sec
197 unknown protocol drops30 second input rate 499000 bits/sec, 259 packets/sec
30 second output rate 525000 bits/sec, 257 packets/sec
211 unknown protocol drops> Вообще, заявленная пропуская способность 2811 - ~61mbps, но из опыта, это значение
> рекордное и зависит от живущего трафика. Возможно, вы просто уперлись в её пределы.Ночью с циской все хорошо, наверное надо все это днем проделать.
Упсь, у вас же 2811, у неё пропуская способность 46mbps при 64kb-пакетах (это на 2821 - 61mbps, был невнимателен), вы просто уперлись в её максимум. Фрагментации у вас нет.Можно попытаться разобраться что за дропы на fa0/0, хотя общей ситуации не изменит, нужен апгрейд по железу:
1. Проверить патч-корд
2. ВРЕМЕННО включить на fa0/0 ip nbar protocol-discovery.
потом смотреть, распознает ли, чьи дропы на sh int fa0/0