RT – starting from a ticket number larger than 1

Posted on April 23, 2013 by danmassey99

Ok, so not a request you would get every day, but when setting up our new RT (Request Tracker) server, we had a requirement to start at ticket number 200,000. This requirement was to ensure the new RT server’s tickets did not overlap with the old server. We have some great new features with the version 4 and we have also installed the SLA add on – all very exciting! Anyway, how is this change made?

The first thing you need to know is there is no convenient, easy to use button to click or setting to make in the config file. This needs to be done on the database direct, in my case this is a MySQL database, but it could be any of the supported database types. From the command line, log into the MySQL console and navigate to the rt4 database:

# mysql -u root -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 108
Server version: 5.5.30 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use rt4
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+-------------------------+
| Tables_in_rt4           |
+-------------------------+
| ACL                     |
| Articles                |
| Attachments             |
| Attributes              |
| CachedGroupMembers      |
| Classes                 |
| CustomFieldValues       |
| CustomFields            |
| GroupMembers            |
| Groups                  |
| Links                   |
| ObjectClasses           |
| ObjectCustomFieldValues |
| ObjectCustomFields      |
| ObjectTopics            |
| Principals              |
| Queues                  |
| ScripActions            |
| ScripConditions         |
| Scrips                  |
| Templates               |
| Tickets                 |
| Topics                  |
| Transactions            |
| Users                   |
| sessions                |
+-------------------------+
26 rows in set (0.00 sec)

mysql>

So we can see the tables and the one that interests us is the Tickets table. We need to alter the auto increment setting to start at 200000 from here:

mysql> ALTER TABLE Tickets AUTO_INCREMENT = 200000;

Job done!
 

Posted in FreeBSD Administration, MySQL | Tagged , | Leave a comment

Updating FreeBSD – The modern way!

Posted on April 19, 2013 by danmassey99

The time has finally come for me to update my update method. For years I have been a ‘dyed-in-the-wool’ cvsup guy, but the last server I installed (FreeBSD 9.1) started bleating about the ports tree not being updated any more, and to be honest I should have taken more notice of my emails! The full story can be found ***here***

So whats the ‘new way’?

FreeBSD has a great utility called freebsd-update which will take care of security fixes as well as minor and major upgrades to the kernel. The freebsd-update does not need to be installed or, even configured unless you have specific requirements. You can control behaviour with the configuration file located in /etc/freebsd-update.conf. So lets just do the first update from the command line:

# freebsd-update fetch
# freebsd-update install

I want to automate this with cron so first I added my email address to the config file. The I added the special 'cron' feature as such:

@daily                                  root    freebsd-update cron

that takes care of the kernel…

What about the Ports?

Portsnap takes care of the ports collection without much effort at all. As always there is has a configuration file, but unless you have some specific issues it can probably left alone. For reference it lives here: /etc/portsnap.conf. To run the program for the first time:

# portsnap fetch
# portsnap extract

thereafter

# portsnap fetch update

Thats pretty much it, you can cron the job if you wish, however I tend to do it on demand.

Posted in FreeBSD Administration | Tagged , , | Leave a comment

Fun (?!) with Cisco OSPF part 1

Posted on April 13, 2013 by danmassey99

There are several factors involved in successfully forming an OSPF neighbourship, these are:

  1. Must pass the authentication process
  2. Must be in the same subnet with a matching mask
  3. Must be in the same OSPF area
  4. Stub flag must match
  5. Unique router-id required
  6. Dead and hello timers must match

This post really just refers to item 3 – the OSPF area. In the past I have come across routers set in area 0.0.0.0 while being neighbours of routers in area 0. This always struck me as weird but I had assumed it was an old IOS thing and pushed it to the back of my mind! It seems however that the area ID can be entered in either decimal or dotted quad notation, more interestingly they are interoperable! Here is the configuration in standard format:

router ospf 1
 log-adjacency-changes
 network 10.1.0.4 0.0.0.0 area 456

All pretty standard, but the neighbour is configured thus:

router ospf 1
 log-adjacency-changes
 network 10.1.0.5 0.0.0.0 area 0.0.1.200

Interesting huh? How does 0.0.1.200 become 456? Well, the dotted quad format should be treated as 32 bit single binary number so in full binary it would be:

0000 0000 0000 0000 0000 0001 1100 1000

When you take into account that the router shows this when we use the question mark after the ‘area’ part of the command:

R5(config-router)#network 1.1.1.1 0.0.0.0 area ?
  <0-4294967295>  OSPF area ID as a decimal value
  A.B.C.D         OSPF area ID in IP address format

And as you would predict, the binary number 1111 1111 1111 1111 1111 1111 1111 1111 is equal to 4294967295 or, should you want to enter the area in dotted quad format, it is of course 255.255.255.255

It’s not possible to mix up the formats on the same router, it seems that once an area has been defined in dotted quad format, there it must stay, see this example:

R4(config)#router ospf 1
R4(config-router)#network 4.4.4.4 0.0.0.0 area 255.255.255.255 
R4(config-router)#area 4294967295 stub
R4(config-router)#exit
R4#sh run | s ospf
router ospf 1
 log-adjacency-changes
 area 255.255.255.255 stub
 network 4.4.4.4 0.0.0.0 area 255.255.255.255

But I suppose that would have been VERY confusing!

Posted in Cisco | Tagged , , | Leave a comment

Setting up Syslog on FreeBSD 9.1 for Cisco

Posted on April 9, 2013 by danmassey99

Just a quick note on how to make a syslog server to collect Cisco logging messages. From a fresh installation of FreeBSD 9.1

1. Allow syslogd to accept connections from anywhere (I use an external firewall to limit access)

echo 'syslogd_flags="-a *:*"' >> /etc/rc.conf

2. Modify the syslogd configuration file to accept the messages from the Cisco devices

echo 'local7.*                                        /var/log/cisco/cisco.log' >> /etc/syslogd.conf

You can also stop duplicating the recored by adding “local7.none” to the following line (final result shown):

*.notice;local7.none;authpriv.none;kern.debug;lpr.info;mail.crit;news.err       /var/log/messages

3. Create the file structure for the messages

mkdir /var/log/cisco
touch /var/log/cisco/cisco.log
chmod 0600 /var/log/cisco/cisco.log

4. Restart the syslogd service

service syslogd restart
Posted in Cisco, FreeBSD Administration | Tagged , | Leave a comment

Creating SSH keys on a Cisco PIX (how retro!)

Posted on March 22, 2013 by danmassey99

Today I was faced with replacing an old Cisco PIX 515 (not the 515E) with another equally old Cisco PIX 515, equipped with a mighty 32MB RAM and only capable of single DES. With its FOS 6.4 operating system compiled in 2004, this was blast from the past. Of course when I changed the hostname to reflect the new location of the device, the warning came up about the ssh keys being invalid. In order to SSH onto a PIX/ASA the device requires the following:

  • set the domain-name
  • set the hostname
  • allow incoming SSH with the ssh command
  • generate keys

No problem, but in the new parlance, the command for key generation is:

crypto key generate rsa modulus 1024

however in the ‘old’ days (pre FOS 7) it was:

ca zerioze rsa
ca generate rsa key 2048
ca save all

Oh how I miss those days, no auto complete, no hair-pinning, no SSL VPN, etc, etc.

Posted in Cisco | Tagged , | Leave a comment

Cisco PPP Revision Lab

Posted on March 17, 2013 by danmassey99

Another day of ongoing study brings me to the ‘Wide Area Network’ topics such as Frame-Relay and PPP. I’m using GNS3 which makes the whole process a bit easier to demonstrate the concepts and help it become less theoretical and more ‘real’! Here is my setup:

ppp_lab

 

Here are my requirements which covers a lot of what the syllabus is asking for:

R1 -> R3
1. Set the clock speed and bandwidth to reflect 128kbps
2. Ensure that if the traffic drops more than 10% of the packets in either direction, the line is dropped.
3. R1 should authenticate R3 using CHAP (password: ‘s3cur3_chap’)
4. R3 should authenticated R1 using EAP (password: ‘s3cur3_3ap’)
5. Configure ‘fast’ payload compression
6. Ensure header compression for tcp based applications is configured as well as voip applications

R1 -> R2
1. Set speed and bandwidth to 128kbps
2. Ensure packets are load balanced over both lines
3. Ensure that the links will only come up if both members are active
4. All links should use bi-directional plain text authentication (password ‘PLAIN’)
5. To ensure small packets are not ‘serialized out’ configure interleaving with a fragment delay of 10ms

R2 -> R3
1. Set speed and bandwidth to 128kbps on both links
2. Configure multilinking only when both lines are up
3. Links should authenticate via chap using UN: router2 PW:fish2 and UN:router3 and PW:fish3

Have fun!

Posted in Cisco | Tagged , , | Leave a comment

Are Cisco moving EIGRP into the public domain?

Posted on March 14, 2013 by danmassey99

It looks as if Cisco may be moving their proprietary distance vector based EIGRP into the public domain. EIGRP is quick and easy to deploy, but is only available on Cisco devices – Maybe this will make it more popular? Here are the details: http://tools.ietf.org/html/draft-savage-eigrp-00

Posted in Cisco | Tagged , , | Leave a comment

EIGRP Authentication on Frame Relay (just for recreation!)

Posted on March 13, 2013 by danmassey99

As part of my ongoing studies I’ve been looking at running EIGRP over a hybrid Frame-Relay network. This is mainly an exercise in switching off split horizon – and a very long winded way of getting there. Whilst doing the EIGRP authentication part an interesting point came up up where by it became apparent that having 2 matching key-chains was not good enough. The key numbers also have to match! So to start with we need to set up the key chain:

key chain EIGRP_KEY
 key 1
   key-string sup3r_s3cr3t

Then I set up the EIGRP on the router (nothing complicated):

router eigrp 20
 network 10.10.10.1 0.0.0.0
 network 10.11.0.1 0.0.0.0
 network 192.168.1.0
 no auto-summary

Now I add the 2 lines of config to the s1/0.2 interface to first tell the interface to use MD5 with EIGRP and the second line to specify the key chain.

interface Serial1/0.2 multipoint
 ip address 10.10.10.1 255.255.255.0
 ip authentication mode eigrp 20 md5
 ip authentication key-chain eigrp 20 EIGRP_KEY
 no ip split-horizon eigrp 20
 snmp trap link-status
 frame-relay map ip 10.10.10.3 104 broadcast
 frame-relay map ip 10.10.10.4 105 broadcast

When I add the authentication configuration into the serial interface, the neighbourship is immediately trashed! Note the date on the GNS3 servers never got set – oops!

*Mar  1 02:01:51.175: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 20: Neighbor 10.10.10.3 (Serial1/0.2) is down: authentication mode changed
*Mar  1 02:01:51.175: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 20: Neighbor 10.10.10.4 (Serial1/0.2) is down: authentication mode changed

Now the interesting bit…(!)

On router 4 (which is in 10.10.10.4 which you can see from the logs above fell off the air when the authentication went on) I test to see the neighbours:

R4#sh ip eigrp neighbors 
IP-EIGRP neighbors for process 20

So no neighbours there. Now i add the config with a different keychain name, but the same key-string and same key number:

key chain MY_CHAIN
 key 1
   key-string sup3r_s3cr3t

interface Serial1/0
 ip address 10.10.10.4 255.255.255.0
 ip authentication mode eigrp 20 md5
 ip authentication key-chain eigrp 20 MY_CHAIN
 encapsulation frame-relay
 serial restart-delay 0
 frame-relay map ip 10.10.10.1 501 broadcast
 frame-relay map ip 10.10.10.3 501 broadcast
 no frame-relay inverse-arp

And up it comes:

*Mar  1 02:13:29.083: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 20: Neighbor 10.10.10.1 (Serial1/0) is up: new adjacency

Now I do R3 which is another neighbour router but use the same keychain name, same secret but a different key number:

key chain EIGRP_KEY
 key 2
   key-string sup3r_s3cr3t

interface Serial1/0
 ip address 10.10.10.3 255.255.255.0
 ip authentication mode eigrp 20 md5
 ip authentication key-chain eigrp 20 EIGRP_KEY
 encapsulation frame-relay
 serial restart-delay 0
 no arp frame-relay
 frame-relay map ip 10.10.10.1 401 broadcast
 frame-relay map ip 10.10.10.4 401 broadcast
 no frame-relay inverse-arp

Not a murmor! Now I’ll change the key number as follows:

key chain EIGRP_KEY
 key 1
   key-string sup3r_s3cr3t

It took a moment, but:

*Mar  1 02:21:40.975: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 20: Neighbor 10.10.10.1 (Serial1/0) is up: new adjacency

Just for completeness, here is R3 showing its EIGRP neighbours.

R3#sh ip eigrp neighbors 
IP-EIGRP neighbors for process 20
H   Address                 Interface       Hold Uptime   SRTT   RTO  Q  Seq
                                            (sec)         (ms)       Cnt Num
0   10.10.10.1              Se1/0            139 00:03:23   12   200  0  39

So the lesson is, match the key number and the key string but don’t worry about the key-chain name.

Posted in Cisco | Tagged , , | Leave a comment

Nagios jitter testing

Posted on March 1, 2013 by danmassey99

As part of our new SLA we are required to test and repost on jitter across the core of our network on a per MPLS VPN basis. That seemed ok to start with, but a Nagios does not have an ‘out-of-the-box’ jitter test which, when you think about it, is not that surprising. The Cisco routers we use can measure jitter so our solution was to get the Cisco routers to do all the fiddly jitter measuring and Nagios to just jump in, collect the data and the glory! Naturally we’ll need a new test written in Perl to bridge the two together.

1. Set up the Cisco router SLA

ip sla 100
 icmp-jitter 10.0.0.1
 vrf VRFNAME

2. Now schedule the test

ip sla schedule 100 life forever start-time now

Now you can see the results by typing:

# show ip sla statistics 100

Round Trip Time (RTT) for	Index 100
Type of operation:		icmpJitter
	Latest RTT: 7 milliseconds
Latest operation start time: 22:03:37.093 GMT Fri Mar 1 2013
Latest operation return code: OK
RTT Values:
	Number Of RTT: 10		RTT Min/Avg/Max: 7/7/9 
Latency one-way time:
	Number of Latency one-way Samples: 0
	Source to Destination Latency one way Min/Avg/Max: 0/0/0 
	Destination to Source Latency one way Min/Avg/Max: 0/0/0 
Jitter Time:
	Number of SD Jitter Samples: 9
	Number of DS Jitter Samples: 9
	Source to Destination Jitter Min/Avg/Max: 0/1/1 
	Destination to Source Jitter Min/Avg/Max: 0/1/1 
Packet Late Arrival: 0
Out Of Sequence: 0
	Source to Destination: 0	Destination to Source 0
	In both Directions: 0
Packet Skipped: 0	Packet Unprocessed: 0
Packet Loss: 0
	Loss Period Length Min/Max: 0/0
Number of successes: 24
Number of failures: 0
Operation time to live: Forever

We are only after one value here which is near the top ‘ Latest RTT: 7 milliseconds and its just the number ’7′ we need. the next task is to write a per test which will extract that number periodically and the test should alarm if it goes out of our defined SLA parameters. Here is the script we used:

#! /usr/bin/perl -w

use Net::Telnet::Cisco;
use strict;

my $host = shift;
my $sla_ref = shift;
my $warn = shift;
my $critical = shift;

my $conn = Net::Telnet::Cisco->new(Host => "$host");

$conn->login('username', 'password');

my @shipsla = $conn->cmd("sh ip sla statistics $sla_ref");
my $result;
foreach (@shipsla){
        if (/^\s+Latest RTT: ([0-9]+) milliseconds/){
                $result = $1;
                last;
        }
}

if ($result >= $critical){
print "CRITICAL - Jitter reading : $result ms\n";
exit(2);
}
if ($result >= $warn){
print "WARNING - Jitter reading : $result ms\n";
exit(1);
}

print "OK - Jitter reading : $result ms\n";
exit(0);

With that plugged into Nagios we are now monitoring multiple jitter values for multiple clients.

Posted in Cisco, FreeBSD Administration, Perl, test_cat | Tagged , , | Leave a comment

IPv6 Tunnel using BGP on a Cisco 877

Posted on March 1, 2013 by danmassey99

The last post was about basic IPv6 tunnelling using GRE and a static route from the ISP router to the Cisco 887 at the remote site. That all worked , but lacked ‘interest’. I had considered putting in a Cisco 1841 which was lying around to attempt a full IPv6 BGP session but then thought ‘Why not try the 877 in use already in the office?’ – and here we are. For info I have a Cisco 877 with a massive 128MB RAM and and Advanced IP services IOS, which when compared to a border router that Gconnect use is very, very small. That said there is not much overhead in the IPv6 table at the moment (~ 10 – 15 K routes). I’m using the same layout (apart from the router model) as last time:

tunnel_diagram

 

So the tunnels are set up as per the previous post, however I have now removed the static route at the ISP end and the default (::/0) from the remote end, ie we just have point to point connectivity between the the routers. Starting on the head end:

1. Make an appropriate prefix list

ipv6 prefix-list GCONNECT-OFFICE seq 5 permit 2A01:570:Y:XXXY::/64

2. Create the peer – I’m updating the source to Tunnel0 to make it work nicely.

router bgp 33941
neighbor 2A01:570:Y:XXXX::2 remote-as 65000
neighbor 2A01:570:Y:XXXX::2 update-source Tunnel0

3. We need to disable the IPv4 address family activation:

 
router bgp 33941
 address-family ipv4
 no neighbor 2A01:570:Y:XXXX::2 activate

4. Now add the prefix list and activate the peer in the IPv6 address family

 
router bgp 33941
 address-family ipv6
  neighbor 2A01:570:Y:XXXX::2 activate
  neighbor 2A01:570:Y:XXXX::2 prefix-list GCONNECT-OFFICE in

Thats the head end sorted, now the trusty 877 is pretty much a mirror of the head end, so I’ll compress into one lump:

ipv6 prefix-list OFFICE seq 5 permit 2A01:570:Y:XXXY::/64
!
router bgp 65000
 bgp log-neighbor-changes
 neighbor 2A01:570:Y:XXXX::1 remote-as 33941
 neighbor 2A01:570:Y:XXXX::1 update-source Tunnel0
 !
 address-family ipv4
  no neighbor 2A01:570:Y:XXXX::1 activate
  no auto-summary
  no synchronization
 exit-address-family
 !
 address-family ipv6
  neighbor 2A01:570:Y:XXXX::1 activate
  neighbor 2A01:570:Y:XXXX::1 prefix-list OFFICE out
  network 2A01:570:Y:XXXY::/64
 exit-address-family

Note the network statement at the end announcing the route to the ISP site. This route needs to be in the local routing table so if you are using a /48 and don’t have it in the table you should null route it. My vlan 1 is using the /64 so I’ve no issues here. I’m using a prefix list to filter outgoing routes as when I hook up a second ISP router i dont want to transiting via my DSL line! So we nee to run some verification commands from the Office Cisco 877:

#sh ip bgp ipv6 unicast summary 
BGP router identifier 192.168.X.X, local AS number 65000
BGP table version is 37022, main routing table version 37022
12133 network entries using 1844216 bytes of memory
12133 path entries using 922108 bytes of memory
7768/7762 BGP path/bestpath attribute entries using 963232 bytes of memory
7250 BGP AS-PATH entries using 184120 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
Bitfield cache entries: current 1 (at peak 2) using 32 bytes of memory
BGP using 3913708 total bytes of memory
BGP activity 12176/43 prefixes, 24313/12180 paths, scan interval 60 secs

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
2A01:570:Y:XXXX::1
                4 33941   16547     158    37022    0    0 00:31:58    12132

There are a few good points to note here. Firstly we have 12,132 IPv6 routes which is good. Second the memory that the BGP is taking up is minimal (circa 4GB) so the 877 does not have any issues. We need to check we are announcing the /64 to the ISP with the following command:

#sh ip bgp ipv6 unicast neighbors 2A01:570:Y:XXXX::1 advertised-routes 
BGP table version is 37046, local router ID is 192.168.X.X
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 2A01:570:Y:XXXY::/64
                    ::                       0         32768 i

Total number of prefixes 1

This all looks good, you need that ‘*>’ meaning valid and best and just the one route. Last thing is to actually test it, so we’ll ping our friends at Google.

ping 2a00:1450:400c:c00::93 source vlan1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2A00:1450:400C:C00::93, timeout is 2 seconds:
Packet sent with a source address of 2A01:570:Y:XXXY::1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 48/52/56 ms

Well that just about wraps it up i think.

Posted in Cisco, IPv6 | Tagged , , | Leave a comment