VLAN Configuration in VMWare 5.5

External Switch Tagging (EST)

The is where no virtual tagging is performed within vmware and all packets coming out of the virtual switch have a vlan tag set to 0 (default to be not tagged in VMWare). This untagged port connects to a vlan access port of a physical switch. This will mark all traffic which goes into the physical port of the switch onto the vlan defined in the vlan. All references to tagging is using dot1q.

Vmware Network configuration using External Switch Tagging

Virtual Switch Tagging (VST)

This is where the virtual environment is connected to a trunk port on the physical switch and the virtual environment sends tagged packets.

Vmware Network configuration using Virtual Switch Tagging

Virtual Guest Tagging (VGT)

This is where the virtual machines themselves rather than the virtual network performs the tagging. The VMWare environment passes these tagged packets along without altering them. The connection to the physical switch should be configured as a trunk.  For this to occur the port group of the virtual machine should be configured with VLAN of 4095. Vlan 4095 should not be used for any other purpose as it tells the VMWare virtual switch to trust the tags for interfaces on VLAN 4095 effectively extending the trunk to these systems.

Vmware Network configuration using Virtual Guest Tagging


CCIE Quarterly Call May 2010

Today I was in the CCIE quarterly call there are a could of noteworthy points.

Core Knowledge waiver seen as an unfair advantage and as such will be eliminated on May the 10th, the extra 30 mins will be allocated to the configuration section. They also mentioned that the skills they were looking for in the core knowledge section are covered in the troubleshooting section. Additionally many candidates need the extra time, particularly the non english speaking ones.

They were also plugging the 360 program and showed a couple of stats, if you get rid of the rubbish its interesting to see the average attempts between the 360 program and outside of that.

The mobile testing centers are working out ways to move forward this and the mobile testing locations will expand, and this will be in a similar way to how the CCDE is deployed.

There was an interesting comment from Fred Weiller – “Dynamips is neither recommended to support and it enables infringement on the IOS licence”, but they are going to be using some virtualistion technologies to make studying more cost effective & accessible.

That’s about it, off topic I am scheduling in for just over 3 months so am quite glad they didn’t announce any big changes like IOS 15.0 🙂 Also any rumours about version 5.0 are false, but there is nothing on the development plan for that.

Just taken the lab

Here is a copy I have just posted to Group Study about my experience at the lab yesterday;

I took the CCIE R&S lab yesterday in Brussels and had a couple of comments that you might find interesting, now some of this came from the proctor over lunch and I may have misheard him, got confused so please don’t take it as gospel;
– All the CCIE labs are graded remotely at other CCIE lab locations with the exception of Voice which requires the person who marks it to be physically there. This is how they can mark the exam out of hours.
– I asked the proctor about the exams at the Person professional testing centers and he said that the idea would be that they are not mobile labs and you will connect to equipment remotely and they are intended to eventually run proctorless, although he mentioned that a proctor would be available in the early stages while they test the exam in these locations if needed, I asked if this would be via telepresence and it was a no.

Comments on the lab;
– The proctor was brilliant, I asked quite a few questions and at one point thought I had a bug and he said “I think this is something you need to fix yourself” which was when I realised they were trying to play mind tricks on me.
– There was 8 candidates taking a variety of tracks
– For both the troubleshooting and configuration you are presented with a topology diagram which when you click on the device icon opens up a Putty window to the device
– I had enough space, the big monitors were just enough for me organise and I could easily view 3-4 putty windows and the relevant items of the topology. Although 2 smaller monitors may have suited me better if I could snap the topology diagram and questions on one side
– I read about candidates taking ear plugs in the past so I took some but they weren’t needed it was a quiet environment as the equipment was in another room.

Other comments;
– The food at Cisco is amazing, I had a steak with chips and veg, cooked how I liked it while I waited, although continental Europeans looked at me strangely when I mentioned I was looking for vinegar to put on my chips.
– Not really associated but when a continental European hotel says they do a cooked breakfast, its nothing like an Englishman’s idea of a cooked breakfast. Bacon should not be cut up into 1inch squares..
– I stayed at the NH Brussels Airport hotel, its directly outside the Diegem train station and just the behind Cisco. Expensive at 155 Euros advance rate for B&B but worth every penny as its so close.
– Brussels has a great train network if your in Europe on a Eurostar line I highly recommend travelling with them, as a single change at Brussels Midi takes you directly to the hotel/testing center, so no worrying about Shuttles/Taxis/Buses/Trams.

Personal note;
I havn’t got the marks back yet but I think I just missed out on the Configuration section, will definitely nail it next time its just a mind game.

Hope this helps, and please don’t bother unicasting me asking for the material on the lab.



CCIE R&S Reading List

One of the first things I repeatedly came across when I was beginning my preparation for the CCIE lab was don’t focus on the commands, having a deep understanding the core technologies is key.

The official CCIE Book List by Cisco is mental, I don’t think its really practical to read through the entire list in any reasonable amount of time. Below is some of the books I have read in my preparations, although I am not suggesting that what was right for me will be right for you, I am also only going to mention hard copy books stuff like the DocCd and RFCs are invaluable aswell.

I started this with a decent enough CCNA/CCNP level of knowledge, from the CCNP track I would recommend
the books from the BSCI and BCMSN as a casual warm up. The smaller portable command guides for BSCI and BCMSN are also quite handy for a quick reference on how to set stuff up, but they are not at CCIE level so don’t plan on using them for long unless you have a temporary brain malfunction.

I also read through a generic non vendor orientated book on networking and would highly recommend it, reading though a non Cisco book really clarified alot of things for me. This is as the book focuses entirely on the technology and protocols and less about a vendors implementation or view of protocols.

Routing TCP/IP, Volume 1 (2nd Edition) – People sometimes refer to this book as the bible of networking, its deep thorough and enjoyable. Doyle and Carroll have done an amazing job on this and its definitely a must read for the core technologies, I personally consider this the best book to read for IGPs.

Routing TCP/IP, Volume II (CCIE Professional Development) – the second volume of Routing TCP/IP focuses on technologies not covered in the original mainly BGP, Multicast, IPv6 and IP Services. I don’t believe its the best book for BGP but it does have alot of detail and is written well although it could do with an update notably in the IPv6 section.

Troubleshooting IP Routing Protocols (CCIE Professional Development Series) – This book is incredible, I don’t know why this wasn’t in my life sooner. When the troubleshooting section came out, I was quite concerned about my methodology. This book as shown me that any routing protocol problem can be solved with the right approach. It has a nice and reasonably brief overview of each routing protocol and then provides systematic processes to solves problems with them, it has flow charts, example problems, show command outputs with the problems, its freaking awesome!

MPLS Fundamentals – although the title has “fundamentals” in it, don’t be tricked into thinking this is a basic level book. It goes from basic knowledge to SP level CCIE stuff in a reasonable short amount of time. Its all good stuff and I particularly enjoyed the section on the background and the false truths about MPLS. I read through chapters 1 to 7 and the chapter on MPLS L3 VPNs, its very well written and I am sure I will be referring to it many times in the future past my CCIE.

Cisco QOS Exam Certification Guide (IP Telephony Self-Study) (2nd Edition)Now this is a good book and it goes into detail in all the right and interesting places, for instance a random fact I found in this book was, the speed of light in a vacuum in 3.0 x 108 m/s, we have all probably come across at some point but the speed of transmission over copper and optical media is generally measured not at the speed of light but 2.1 x108m/s. But please dont get me wrong I had real difficulty reading this as my own personal tastes just find QoS boring, I genuinely couldn’t read more than 20 to 50 pages (if that) while staying awake. The book is well written and a great reference guide I just couldn’t personally read it cover to cover as I don’t enjoy QoS.

CCIE Routing and Switching Exam Certification Guide (4th Edition) – This is the official guide for the written, its designed for the 350-001 written and not so much for lab but it does have most of the topics in a single book. I haven’t read the latest version 4 as of yet.

Books which I am yet to read, but planning to

Developing IP Multicast Networks, Volume I by Beau Williamson – this has been recommended in some other places in regards to multicast, I have heard it could do with a bit of an update but is still very vaulable. I am planning on getting it soon.

Internet Routing Architectures (2nd Edition)
by Wendell Odom, Michael J. Cavanaugh – Another one on my to read list, I have heard this is THE book for BGP, will defiantly be getting this next.

Understanding Zone Based Firewalls

One of the new topics to the blueprint is zone based firewalls, in my head (and possibly not in reality) its like CBAC with some extra bells and whistles but between virtual zones, I wasn’t particularly hot on the subject but after a review its really not that bad, you have have to follow the key steps.

1. Define your security zones
Firstly choose what security zones you wish to define, you can choose almost any name you like I have just chosen internal and external for my test, there’s no reason why you couldn’t have 3, 4 or more zones if required. There is a default zone called “local” which is for traffic originated/sent to the router as opposed to passing through it. So as far as I am aware if you don’t play around with the local zone it shouldn’t affect traffic your routing protocol traffic, as it does not pass through it. Define the zones with the following commands;

Router(config)#zone security internal
Router(config)#zone security external

2. Assign interfaces to the relevant zones
Under each interface you wish to link to a zone go under the interface and use the following command where the interface is linked to the zone “internal”.

Router(config-if)#zone-member security internal

3. Define your class maps
Split your traffic up into classes and make sure you use “type inspect” option when defining them, the simple one below uses NBAR to classify http packets. I wont explain class or policy maps in much detail as most other candidates should be familiar with them from MQC.

class-map type inspect match-all http
 match protocol http

4. Tie up the class maps in a policy map
Now to define the action for each of the classes defined in the class maps, make sure you use the “inspect” option if you wish to inspect the traffic to allow return traffic. In the example below I have named the policy zonebasedfw.

policy-map type inspect zonebasedfw
 class type inspect http
 class class-default

5. Tie it all together in a zone pair
Finally create a zone pair with your own name I chose “internal-to-external”, define your source and destination zones and the pre defined policy.

zone-pair security internal-to-external source internal destination external
 service-policy type inspect zonebasedfw

There are 2 handy verification commands which I am aware of, firstly sh zone security as shown below, this indicates which interfaces are linked to which zone;

Router#sh zone security
zone self
  Description: System defined zone

zone internal
  Member Interfaces:

zone external
  Member Interfaces:

And sh zone-pair security which shows the details from the zone pair, notably which policy has been applied in which direction.

Router#sh zone-pair security
Zone-pair name internal-to-external
    Source-Zone internal  Destination-Zone external
    service-policy zonebasedfw

Understanding uRPF – Unicast Reverse Path Forwarding

Spoofed packets are a big problem with on the Internet, they are commonly used in DNS amplification attacks, and TCP SYN floods. Unfortunately there is no simple way to totally fix all spoofed packets on the Internet but if service providers implement ingress filtering on their network, it effectively stops such attacks with spoofed source addresses coming from their patch.

The process is actually standardised Best Practice in BCP 38 “Network Ingress Filtering” which all service providers should implement if they have Internet facing services for good karma.

There are a number of ways of implementing ingress filtering, one of the technically simplest is to create ACLs of your customers global address ranges and only allow packets sourced from those ranges to leave your network. Configuration wise Unicast Reverse Path Forwarding (uRPF) is in my opinion the simplest way of managing this and it has a couple of extra features.

uRPF checks incoming unicast packets and validates that a return path exists, there is not much point in forwarding a packet if it doesnt know how to return it right?

There are 2 methods of implementation of uRPF strict and loose. Strict mode is where the source of the packet is reachable via the interface that it came from, this is nice for extra security on the edge of your network but not so good if you have multiple edges towards the Internet eg you peer at multiple IXPs where you might expect asymmetric routing. In such cases loose mode is used which checks that a return route exists in the routing table.

The configuration is super simple, after CEF has been enabled just go to the interface you wish to check inbound traffic and use the following command, with the “rx” option for strict mode or “any” for loose mode.

Router(config-if)#ip verify unicast source reachable-via ?
  any  Source is reachable via any interface
  rx   Source is reachable via interface on which packet was received

Obviously you can check the running config to see if its configured but if your a fan of using other show commands its visible under the sh cef interface and sh ip interface as shown below;

Router#sh cef  interface fastEthernet 0/0 | i RPF
  IP unicast RPF check is enabled
Router# sh ip int fa0/0 | i verify
  IP verify source reachable-via RX


Netflow is a great tool developed by Cisco which is commonly used for bandwidth monitoring & traffic analysis. Its used quite heavily where I work for the detecting and dealing with security related incidents (I talk about it at the last UKNOF meeting here). Although originally developed by Cisco other vendors have support for it under their own product names. And there are standardised versions of it under the name IPFIX.

Like with anything I thoroughly recommend you test this out before rolling it out to your production systems as on high traffic networks it can cause CPU problems in such cases most people change the amount of sampling. If you are worried about this Cisco has produced a white paper on working out the resource utilisation of Netflow.


First of all to configure it choose the interfaces you want to monitor and decide if you want to monitor the ingress, egress or both;

Router(config-if)#ip flow ?
egress Enable outbound NetFlow
ingress Enable inbound NetFlow

There are plenty of extra global options available, below is an example of the configuration to capture the packet length, TTL and the MAC addresses.

ip flow-capture packet-length
ip flow-capture ttl
ip flow-capture mac-addresses

Most implementations of Netflow export the data to a remote server for analysis. There are plenty of Netflow analysis software choices. There are 3 versions supported on Cisco routers at present 1, 5 & 9. Version 1 was designed for classful networks and is almost obsolete, Version 5 is designed for IPv4 Unicast flows, and Version 9 is the newest built and can carry BGP Next Hop information, IPv6, Multicast, and MPLS.

Netflow wont export each packets headers as a single packet but it will collect the packet headers together and report them in a single UDP packet via export. In my example at the bottom of this 715 flows have been exported in 48 UDP packets.

ip flow-export version 9
ip flow-export destination 555

Another handy option is configuring logging of the top talkers, this is handy if you have a problem with a remote site where they are complain of problems with this enabled you can see the biggest bandwidth hogs in the period of the cache timeout.

ip flow-top-talkers
        top 100
        sort-by bytes
        cache-timeout 3600000
        match protocol tcp


Most importantly verification its actually working, show ip flow interface below confirms which interfaces are sampling for netflow

Router#sh ip flow interface
  ip flow ingress
  ip flow egress
  ip flow ingress
  ip flow egress

If you configured top talkers then show ip flow top-talkers will show you the bandwidth hogs.

Router#sh ip flow top-talkers

SrcIf         SrcIPaddress    DstIf         DstIPaddress    Pr SrcP DstP Bytes
Di0   BV1*    06 0050 D89F    92K
BV1     Local      06 D7DC 0017    19K
BV1     Di0   06 D89F 0050  7381
BV1     Di0   06 D89E 0050  1466
Di0   BV1*    06 0050 D89E  1379
5 of 100 top talkers shown. 5 of 9 flows matched.

Finally if you configured netflow export verify it with the show ip flow export command.

Router#sh ip flow export
Flow export v9 is enabled for main cache
  Export source and destination details :
  VRF ID : Default
    Destination(1) (555)
  Version 9 flow records
  715 flows exported in 48 udp datagrams
  0 flows failed due to lack of export packet
  48 export packets were sent up to process level
  0 export packets were dropped due to no fib
  0 export packets were dropped due to adjacency issues
  0 export packets were dropped due to fragmentation failures
  0 export packets were dropped due to encapsulation fixup failures

IPExpert v4 VoD first thoughts

Just received version of the IPExpert Video on Demand (VoD) the updated for the v4 R&S CCIE track. I have gone through the first day over the weekend and thought it would be helpful to post my initial thoughts.

For your money you get 45 videos, which are 116kbps MP4 files, consisting of 84kbps of video (only slides so this is fine), and 32kbps of audio. The menu splits the video up over 4 days but from the amount of content it could easily be over 5 days,.

The product gets shipped on a DVD and the files are 4GB, I have copied the files straight off the DVD onto my hard drive and they have worked fine. I have tested it with Windows 7 x64 and on my Mac with Snow Leopard and it worked really fine, just click on the .exe or .app file and you get presented the menu below.
IPExpert Video on Demand Menu

Below is an example of the video interface which is the same on Windows or Mac. You might notice that there is no full screen option as there was in the v3 Video, I hope they put that in a future version as the videos could be a little larger especially if you have a large screen with high resolution.
IPExpert Video on Demand

One of my main issues with the old VoD was that the slides did not appear for long enough and the video was split between the slides and Scot Morris. It was sometimes a welcome break to pause it and try to get Scott Morris to pull strange faces mid talk. 🙂 With the update there is no video of the instructor just the slides, command line and audio which is best but no funny faces of Joe like we could get with Scott. 🙂
Scott Morris presenting IPExpert Video on Demand

Bypassing Obfuscation
One of the downsides to the product is they dont give you easy access to the MP4 files, personally I would like to have the media on my phone, ipod etc. Now from the looks of they havn’t encrypted the video files just obfuscated them. If you look in the /com/ipexpert/data.dat/ folder you will see 45 files with the .ipx extension. Changing the extension to MP4 allows you to play them in any player you like. You just need to match up the file names with the actual video content, it would be pretty simply for somebody to write a batch/shell script to do this 🙂
IPExpert Video on Demand MP4 files

Quality of Content
The videos are presented by Joe Astorino who trained with the v3 BLS as he explained in the videos. From what I have seen so far (after day 1) Joe presents a lecture on the theory which lasts around and hour long and then a few 30 to 60 mins hands on labs on the topics talked about. The level of detail is really good and he doesn’t spend to long on the basics. There are 45 video files which are split over 4 official days.

Joe is really good at explaining every little detail, but is especially on the configuring sections. Sometimes there is a lot of detail on the slides, but if you want to refer back to the slides they are available to download from the members area of the IPExpert web site, which is a nice touch. There are a couple of really minor problems which should get fixed eg after one of the video Joe says words to the affect of “thats the end of day 1” when there is another video, but that should get fixed pretty quickly and does not affect the quality of the training.

The quality of the audio is also excellent, almost too good as you can hear every creak of Joe’s chair and gulp of water!

Excellent quality instruction by Joe
Really high quality video
Good value for money, much cheaper than a boot camp

No full screen button
No easy access to the MP4 files

Its a damn good set of videos updated for the new track, I am very happy with it but could only be a little more if it had a full screen button or easier access to the MP4 files. Joe has done a brilliant job and I am defiantly feeling better about the configuration section regarding the topics he has covered. Its part of the BLS so contact IPExpert if you have the old version to get them to send you the updated one.

OSPF Network Types

Quick bit of a refresh on OSPF network types, these are specified on a per link basis and it changes the way OSPF behaves on that link specifying the wrong type can cause bad things to happen such as if you mix DR & Non DR types adjacencies wont form correctly.

There are 5 network types,

  1. Point to Point – Should be used as the name implies on P2P links which support broadcast, nearly all OSPF traffic is sent to the ALLSPFRouters address which is, although retransmitted LSAs are sent to the unicast address as such these links should support broadcasting.
  2. Broadcast – This is the default network type for Ethernet networks, as this network type is capable of having adjacencies with multiple routers. To prevent unnecessary meshing DR & BDR elections take place. Communications from the DR & BDR take place on the ALLDRRouters address whilst all other communications on As the name implies the layer 2 network must support broadcasting.
  3. Non Broadcast Multiple Access (NBMA) – This is the default configuration on a serial interface, and is classified as a link which is able to send to multiple other routers via the single link but with no broadcasting, therefore neighbour statements must be manually set. A DR & BDR will still be elected but all communication is unicast, the hello and dead timers change from the defaults to 30 and 120 seconds.
  4. Point to Multipoint networks – All communication is unicast and no DR or BDR election takes place an ideal placement for this would be in the hub in a hub and spoke topology.
  5. Virtual Links – Packets sent over Virtual Links are unicast and perceived as unnumbered point to point connections

Its pretty simple to change the network type its all done under the interface like so.

Router(config-if)#ip ospf network ?
  broadcast            Specify OSPF broadcast multi-access network
  non-broadcast        Specify OSPF NBMA network
  point-to-multipoint  Specify OSPF point-to-multipoint network
  point-to-point       Specify OSPF point-to-point network

ODR is freaking awesome!

In a nutshell On-Demand Routing (ODR) is simply freaking awesome for hub and spoke environments.

In hub and spoke enviornemnts before ODR there was generally two choices to the deployment

  1. Have default routes on the spokes pointing to the hub and static routes back. But frankly this was a pain as there was an administrative burden as you have to manually add/remove each spoke as needed.
  2. Run a routing protocol on the spokes which communicates reachability information with the hub but this is unnecessary as the spokes are dead end streets they don’t need more topology information than a default route and generally the spoke routers aren’t the best in the network so it an unneeded overhead.

But this is were On-Demand Routing (ODR) comes to the rescue . Now ODR is not a routing protocol, the spoke routers send details of attached networks via CDP to neighbouring routers and when ODR is enabled the network information is simply installed into the IP routing table. Each of the spokes only needs to have a default route pointing to the hub and have CDP enabled which is on by default and thats it. The routes it collects have an AD of 160 with a metric of 1 (only 1 hop away), the routes that ODR collected can then be redistributed into your IGP of choice. 😀

If you don’t believe me that its that simple check out the configuration below
Simple Hub and Spoke Cisco Topology

Firstly we just enable it with the

router odr

command, this is a single command which only needs to be configured on the hub, the spokes dont need it.

R0(config)#router odr

There are not many options in the protocol itself the most common you might wish to change is the timers

Router configuration commands:
  default            Set a command to its defaults
  default-metric     Set metric of redistributed routes
  distance           Define an administrative distance
  distribute-list    Filter networks in routing updates
  exit               Exit from routing protocol configuration mode
  help               Description of the interactive help system
  maximum-paths      Forward packets over multiple paths
  neighbor           Specify a neighbor router
  network            Enable routing on an IP network
  no                 Negate a command or set its defaults
  passive-interface  Suppress routing updates on an interface
  redistribute       Redistribute information from another routing protocol
  timers             Adjust routing timers
  traffic-share      How to compute traffic share over alternate paths

We can validate that its running and see the routes it has collected.

R0#sh ip protocols
Routing Protocol is "odr"
  Sending updates every 60 seconds, next due in 47 seconds
  Invalid after 180 seconds, hold down 0, flushed after 240
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Maximum path: 4
  Routing Information Sources:
    Gateway         Distance      Last Update             160      00:00:30             160      00:00:47
  Distance: (default is 160)

And on the hub se should be able to see the routes installed into the IP routing table with the AD of 160 and metric of 1

R0#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set is subnetted, 2 subnets
o [160/1] via, 00:01:45, Serial0/1
                 [160/1] via, 00:00:45
o [160/1] via, 00:00:02, Serial0/0
                 [160/1] via, 00:02:02, Serial0/0
                 [160/1] via, 00:01:02 is variably subnetted, 2 subnets, 2 masks
C is directly connected, Serial0/0
C is directly connected, Serial0/0 is variably subnetted, 2 subnets, 2 masks
C is directly connected, Serial0/1
C is directly connected, Serial0/1

Now to test it the view from R2 just shows the default route up to the hub

R2#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is to network is subnetted, 1 subnets
C is directly connected, Loopback0 is variably subnetted, 2 subnets, 2 masks
C is directly connected, Serial0/0
C is directly connected, Serial0/0
S* is directly connected, Serial0/0

Now to test if we can ping the other spoke router


Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to, timeout is 2 seconds:
Success rate is 100 percent (5/5), round-trip min/avg/max = 12/39/60 ms

I am almost speechless at how simple it was to get spoke to spoke connectivity! Beautiful!