CCIE Blog

Hi Brian,

Can we use NBAR on the gateway router to prevent internal users from watching video streams from any video web site (like Youtube.com)?

Ahmed

Hi Ahmed,

Yes, NBAR can be used to apply application based filters such as blocking youtube.com traffic. To accomplish this we can categorize traffic based on the HTTP hostname. Next we will create a policy-map that matches the youtube.com class and drops the traffic. Lastly the policy is applied outbound to the Internet. Syntax-wise this would read:

R1#
class-map match-all YOUTUBE
 match protocol http host "*youtube.com*"
!
policy-map DROP_YOUTUBE
 class YOUTUBE
   drop
!
interface FastEthernet0/0
 description TO INTERNET
 service-policy output DROP_YOUTUBE

NBAR for HTTP can also be used to match based on URL string or IANA MIME type. For more information see:

Network-Based Application Recognition and Distributed Network-Based Application Recognition

Classification, Policing and Marking on the 3560 model.

As we remember, 3560 uses the concept of internal QoS label, which contains both the CoS and DSCP values for a packet. Only one value is actually taken (trusted) from the packet or interface (either CoS or DSCP), and the other one is deduced using the configurable mapping tables. As it has been described previously, we are allowed to map CoS/DSCP values to queues/thresholds separately. This allows for different service policies to be configured with respect to non-IP and IP traffic. The value that has been trusted (CoS/DSCP) is used as the key selector to either the CoS or DSCP based mapping tables.

What are the available classification options? You can match an access-list (MAC, IP or IPv6), an input interface (only for hierarchical policy-maps to be discussed later) and IP DSCP or Precedence values. Note that CoS value could not be matched, you may configure to trust them. You can trust either CoS, DSCP or IP precedence values from IP/IPv6 or non-IP packet. If a packet has no CoS assigned, the default CoS value is set when interface is set to trust CoS.

Now for the “set” operation – you can only set either IP/IPv6 DSCP or Precedence fields, but not CoS field directly. Does that mean we are not allowed to change the CoS value? No, it is still possible to change the CoS value by using “set dscp” command and DSCP to CoS mapping table – the indirect way. This is a way to extend DSCP-based operations to CoS only bearing packets by use of mapping tables. For example the following configuration will change CoS value of any non-IP packet to “5” (equivalent to DSCP EF):

!
! Match any ether-type, e.g. IPX packets
!
mac access-list extended TEST
 permit any any 0x0 0xFFFF
!
class-map match-all TEST
  match access-group name TEST

!
! Match the Non-IP packets and set DSCP value
!
policy-map TEST
  class TEST
   set dscp ef

!
! Connected to R1
!
interface FastEthernet 0/1
 service-policy input TEST

Verification – configure IPX on R1 and R3, e.g. using the IPX network “135” and SNAP encapsulation.

Rack21R1#ping 135.0011.920e.5620
Type escape sequence to abort.
Sending 5, 100-byte IPX Novell Echoes to 135.0011.920e.5620, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms

Rack21SW1#show mls qos interface fastEthernet 0/3 statistics
FastEthernet0/3

  cos: incoming
——————————-

  0 -  4 :         175            0            0            0            0
  5 -  7 :           0            0            0
  cos: outgoing
——————————-

  0 -  4 :         664            0            0            0           11
  5 -  7 :           5            0            1
Policer: Inprofile:            0 OutofProfile:            0

For example, with policer markdown action, we use the configurable mapping table to map DSCP value to a new policed-down equivalent. When the markdown action is applied to a CoS-marked packet (non-IP) the CoS-to-DSCP mapping table is used to deduce the DSCP value, then the other table (DSCP to Policed DSCP) is used mark the DSCP value down, and finally the marked-down DSCP value is mapped back to the CoS value (DSCP to CoS map).

!
! DSCP to Policed-DSCP map
!
mls qos map policed-dscp 0 to 8

!
! Match the Non-IP packets and set DSCP value
!
policy-map TEST
  class TEST
   police 64000 8000 exceed policed
!
! Connected to R1
!
interface FastEthernet 0/1
 service-policy input TEST

As with the 3550 model, 3560 allows policers to be applied ingress on any port. The limit is 64 policers per port ingress, and 256 policers per ASIC. However, no egress policers are available on this platform. Aggregate policers could also be configured, with the same restrictions as the 3550 had.

Another feature, known on the 3550 models as the Per-Port Per-VLAN classification is implemented using the hierarchical policy-maps. The concept is better illustrated using an example:

!
!  Any non-IP traffic
!
mac access-list extended MAC_ANY
 permit any any 0x0 0xFFFF

!
! Any IP traffic
!
ip access-list extended IP_ANY
 permit ip any any

!
! Class for any non-IP traffic
!
class-map MAC_ANY
 match access-group name MAC_ANY

!
! Class for any IP traffic
!
class-map IP_ANY
 match access-group name IP_ANY

!
! Class to match the port connected to R1
!
class-map PORT_R1
 match input-interface FastEthernet 0/1

!
! Class to match the port connected to R3
!
class-map PORT_R3
 match input-interface FastEthernet 0/3

!
! Inteface-level policy-maps, limit rate per-port (R1 & R3)
!
policy-map PORT_R1
 class PORT_R1
  police  64000 8000

!
policy-map PORT_R3
  class PORT_R3
   police 512000 64000

!
! VLAN policy-map; two levels
!
policy-map VLAN_POLICY
 class IP_ANY
  set dscp 24
  service-policy PORT_R1
 class MAC_ANY
  set dscp ef
  service-policy PORT_R3
!
! Attach a switch-wide VLAN policy
!
interface VLAN 1
 service-policy input VLAN_POLICY
!
! Enabe VLAN based-QoS on some ports
!
interface range FastEthernet 0/1, FastEthernet 0/3
 mls qos vlan-based

What is that configuration intended to do? It defines two VLAN-wide classes, which apply to all ports belonging to this VLAN (including trunks) that has VLAN-based QoS enabled. Therefore, for any IP traffic on VLAN1 we set DSCP to 24, and for any non-IP traffic we set CoS to 5 (implicitly, using the “set dscp” command). Additionally, for IP traffic only, we limit port Fa 0/1 to 64Kbps, and for non-IP traffic we limit port Fa0/3 to 512Kbps. The policing is the only thing that is allowed to be configured on the “child” policy-map, and you can only match input-interfaces as a classification criteria for second-level policy-maps. The “child” (second-level) policy maps are unnecessary, and could be omitted from your configuration (i.e. you can use only a single-level policy-map). However, you must enable vlan-based QoS on the ports that needs to be affect by SVI-level policy-map.

The biggest difference from the 3550 model is that per-port, per-VLAN classification is now switch-wide (not port-wide), so it’s better called “per-VLAN” classification. You still may control which ports are affected, and use second-level policy-map to control the incoming rate per-class/per-port. For example, you may use the same range of ports under all class-maps and specify different rates for different types of traffic.

And finally, for verification techniques. The most used command to verify your QoS implementation is probably show mls qos interface x/y statistics which displays the number of packets with respective DSCP/CoS values entering/leaving a given port.

Rack21SW1#show mls qos interface fastEthernet 0/3 statistics
FastEthernet0/3

  dscp: incoming
——————————-

  0 -  4 :        2124            0            0            0            0
  5 -  9 :           0            0            0            0            0
 10 - 14 :           0            0            0            0            0
 15 - 19 :           0            0            0            0            0
 20 - 24 :           0            0            0            0            0
 25 - 29 :           0            0            0            0            0
 30 - 34 :           0            0            0            0            0
 35 - 39 :           0            0            0            0            0
 40 - 44 :           0            0            0            0            0
 45 - 49 :           0            0            0            0            0
 50 - 54 :           0            0            0            0            0
 55 - 59 :           0            0            0            0            0
 60 - 64 :           0            0            0            0
  dscp: outgoing
——————————-

  0 -  4 :        3053            0            0            0            0
  5 -  9 :           0            0            0            0            0
 10 - 14 :           0            0            0            0            0
 15 - 19 :           0            0            0            0            0
 20 - 24 :           0            0            0            0           98
 25 - 29 :           0            0            0            0            0
 30 - 34 :           0            0            0            0            0
 35 - 39 :           0            0            0            0            0
 40 - 44 :           0            0            0            0            0
 45 - 49 :           0            0            0            0            0
 50 - 54 :           0            0            0            0            0
 55 - 59 :           0         1458            0            0            0
 60 - 64 :           0            0            0            0
  cos: incoming
——————————-

  0 -  4 :        5914            0            0            0            0
  5 -  7 :           0            0            0
  cos: outgoing
——————————-

  0 -  4 :        7991            0            0           98           11
  5 -  7 :        2762            0            2
Policer: Inprofile:            0 OutofProfile:            0

The other show commands are also useful, like show policy-map, show class-map, show access-list, show mls qos interface. There are also «esoteric» commands under the show platform but those are «deep technical» and are not well documented on the DocCD.

The 3560 QoS processing model is tightly coupled with it’s hardware architecture borrowed from the 3750 series switches. The most notable feature is the internal switch ring, which is used for the switch stacking purpose. Packets entering a 3560/3750 switch are queued and serviced twice: first on the ingress, before they are put on the internal ring, and second on the egress port, where they have been delivered by the internal ring switching. In short, the process looks as follows:

[Classify/Police/Mark] -> [Ingress Queues] -> [Internal Ring] -> [Egress Queues]

For more insights and detailed overview of StackWise technology used by the 3750 models, visit the following link:

Cisco StackWise Technology White Paper

Read the rest of this entry »

Catalyst QoS configuration for IP Telephony endpoints is one of the CCIE Voice labs topics. Many people have issues with that one, because of need to memorize a lot of SRND recommendations to do it right. The good news is that during the lab exam you have full access to the QoS SRND documents and UniverCD content. The bad news is that you won’t probably have enough time to navigate the UniverCD with comfort plus the reference configurations often have a lot of typos and mistakes in them.

Read the rest of this entry »

QoS features available on Catalyst switch platforms have specific limitations, dictated by the hardware design of modern L3 switches, which is heavily optimized to handle packets at very high rates. Catalyst switch QoS is implemented using TCAM (Ternary Content Addressable Tables) - fast hardware lookup tables - to store all QoS configurations and settings. We start out Catalyst QoS overview with the old, long time available in the CCIE lab, the Catalyst 3550 model.

Read the rest of this entry »

This is a good example of fragmentation and interleaving, applied in a complex context. To begin with, why whould anyone need to run Multilink PPP (MLPPP or MLP) with Interleaving over Frame-Relay? Well, back in days, when Frame-Relay and ATM were really popular, there was a need to interwork the two technologies: that is, transparently pass encapsulated packets between FR and ATM PVCs. (This is similar in concept with modern L2 VPN interworking, however it was specific to ATM and Frame-Relay). Let’s imagine a situation where we have slow ATM and Frame-Relay links, used to transport a mix of VoIP and data traffic. As we know, some sort of fragmentation and interleaving scheme should be implemented, in order to keep voice quality under control. Since there was no fragmentation scheme common to both ATM and Frame-Relay, people came with idea to run PPP (yet another L2 tech) over Frame-Relay and ATM PVCs and use PPP multilink and interleave feature to implement fragmentation. (Actually there was no good scheme for native fragmentation and interleaving with VoIP over ATM - the cell mode technology - how ironic!)

Before coming up with a configuration example, let’s discuss briefly how PPP Multilink and Interleave works. MLPPP is defined under RFC 1990, and it’s purpose is to group a number of physical links into one logical channel with larger “effective” bandwidth. As we discussed before, MLPPP uses a fragmentation algorithm, where one large frame is being split at Layer2 and replaced with a bunch of sequenced (by the use of additional MLPPP header) smaller frames which are then being sent over multiple physical links in parallel. The receiving side will then accept fragments, reorder some of them if needed, and assemble the pieces into complete frame using the sequence numbers.

So here comes the interleave feature: small voice packets are not fragmented by MLPPP (no MLPPP header and sequence number added) and are simply inserted (intermixed) among the fragments of large data packet. Of course, a special interleaving priority queue is used for this purpose, as we have discussed before.

To summarize:

1) MLPPP uses fragmentation scheme where large packets are sliced in pieces and sequence numbers are added using special MLPPP headers
2) Small voice packets are interleaved with fragments of large packets using a special priority queue

We see that MLPPP was originally designed to work with multiple physical links at the same time. However, PPP Multilink Interleave only works with one physical link. The reason is that voice (small) packets are being sent without sequence numbers. If we were using multiple physical links, the receiving side may start accepting voice packets out of their original order (due to different physical link latencies). And since voice packets bear no fragmentation headers, there is no way to reorder them. In effect, packets may arrive to their final destination out of order, degrading voice quality.

To overcome this obstacle, Multiclass Multilink PPP (MCMLPPP or MCMLP) has been introduced in RFC 2886. Under this RFC, different “fragment streams” or classes are supported at sending and receiving sides, using independent sequence numbers. Therefore, with MCMLPPP voice packets may be sent using MLPPP header with separate sequence numbers space. In result, MCMPPP permits the use of fragmentation and interleaving over multiple physical links at time.

Now back to our MLPPPoFR example. Let’s imagine the situation where we have two routers (R1 and R2) connected via FR cloud, with physical ports clocked at 512Kpbs and PVC CIR values equal to 384Kbps (There is no ATM interworking in this example). We need to provide priority treatment to voice packets and enable PPP Multilink and Interleave to decrease serialization delays.

[R1]---[DLCI 112]---[Frame-Relay]---[DLCI 211]---[R2]

Start by defining MQC policy. We need to make sure that software queue gives voice packets priority treatmet, or else interleaving will be useless

R1 & R2:

!
! Voice bearer
!
class-map VOICE
 match ip dscp ef

!
! Voice signaling
!
class-map SIGNALING
 match ip dscp cs3

!
! CBWFQ: priority treatment for voice packets
!
policy-map CBWFQ
 class VOICE
  priority 48
 class SIGNALING
  bandwidth 8
 class class-default
  fair-queue

Next create a Virtual-Template interface for PPPoFR. We need to calculate the fragment size for MLPPP. Since physical port speed is 512Kpbs, and required serialization delay should not exceed 10ms (remember, fragment size is based on physical port speed!), the fragment size must be set to 512000/8*0,01=640 bytes. How is the fragment size configured with MLPPP? By using command ppp multilink fragment delay - however, IOS CLI takes this delay value (in milliseconds) and multiplies it by configured interface (virtual-template) bandwidth (in our case 384Kbps). We can actually change the virtual-template bandwidth to match the physical interface speed, but this would affect the CBWFQ weights! Therefore, we take the virtual-template bandwidth (384Kpbs) and adjust the delay to make sure the fragment size matches the physical interace rate is 512Kpbs. This way, the “effective” delay value would be set to “640*8/384 = 13ms” (Fragment_Size/CIR*8) to accomodate the physical and logical bandwidth discrepancy. (This may be unimportant if our physical port speed does not differ much from PVC CIR. However, if you have say PVC CIR=384Kbps and port speed 768Kbps you may want to pay attention to this issue)

R1:
interface Loopback0
 ip address 177.1.101.1 255.255.255.255
!
interface Virtual-Template 1
 encapsulation ppp
 ip unnumbered Loopback 0
 bandwidth 384
 ppp multilink
 ppp multilink interleave
 ppp multilink fragment delay 13
 service-policy output CBWFQ

R2:
interface Loopback0
 ip address 177.1.102.1 255.255.255.255
!
interface Virtual-Template 1
 encapsulation ppp
 ip unnumbered Loopback 0
 bandwidth 384
 ppp multilink
 ppp multilink interleave
 ppp multilink fragment delay 13
 service-policy output CBWFQ

Next we configure PVC shaping settings by using legacy FRTS configuration. Note that Bc is set to CIR*10ms.

R1 & R2:
map-class frame-relay SHAPE_384K
frame-relay cir 384000
frame-relay mincir 384000
frame-relay bc 3840
frame-relay be 0

Finally we apply all the settings to the Frame-Relay interfaces:

R1:
interface Serial 0/0/0:0
 encapsulation frame-relay
 frame-relay traffic-shaping
!
! Virtual Template bound to PVC
!
interface Serial 0/0/0:0.1 point-to-point
 no ip address
 frame-relay interface-dlci 112 ppp virtual-template 1
  class SHAPE_384K

R2:
interface Serial 0/0/1:0
 encapsulation frame-relay
 frame-relay traffic-shaping
!
! Virtual Template bound to PVC
!
interface Serial 0/0/1:0.1  point-to-point
 no ip address
 no frame-relay interface-dlci 221
 frame-relay interface-dlci 211 ppp virtual-Template 1
  class SHAPE_384K

Verification

Two virtual-access interfaces have been cloned. First for the member link:

R1#show interfaces virtual-access 2
Virtual-Access2 is up, line protocol is up
  Hardware is Virtual Access interface
  Interface is unnumbered. Using address of Loopback0 (177.1.101.1)
  MTU 1500 bytes, BW 384 Kbit, DLY 100000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation PPP, LCP Open, multilink Open
  Link is a member of Multilink bundle Virtual-Access3   <---- MLP bundle member
  PPPoFR vaccess, cloned from Virtual-Template1
  Vaccess status 0x44
  Bound to Serial0/0/0:0.1 DLCI 112, Cloned from Virtual-Template1, loopback not set
  Keepalive set (10 sec)
  DTR is pulsed for 5 seconds on reset
  Last input 00:00:52, output never, output hang never
  Last clearing of "show interface" counters 00:04:17
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo       <---------- FIFO is the member link queue
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     75 packets input, 16472 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     86 packets output, 16601 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions

Second for the MLPPP bundle itself:

R1#show interfaces virtual-access 3
Virtual-Access3 is up, line protocol is up
  Hardware is Virtual Access interface
  Interface is unnumbered. Using address of Loopback0 (177.1.101.1)
  MTU 1500 bytes, BW 384 Kbit, DLY 100000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation PPP, LCP Open, multilink Open
  Open: IPCP
  MLP Bundle vaccess, cloned from Virtual-Template1   <---------- MLP Bundle
  Vaccess status 0x40, loopback not set
  Keepalive set (10 sec)
  DTR is pulsed for 5 seconds on reset
  Last input 00:01:29, output never, output hang never
  Last clearing of "show interface" counters 00:03:40
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: Class-based queueing    <--------- CBWFQ is the bundle queue
  Output queue: 0/1000/64/0 (size/max total/threshold/drops)
     Conversations  0/1/128 (active/max active/max total)
     Reserved Conversations 1/1 (allocated/max allocated)
     Available Bandwidth 232 kilobits/sec
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     17 packets input, 15588 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     17 packets output, 15924 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions

Verify the CBWFQ policy-map:

R1#show policy-map interface
 Virtual-Template1 

  Service-policy output: CBWFQ

    Service policy content is displayed for cloned interfaces only such as vaccess and sessions
 Virtual-Access3 

  Service-policy output: CBWFQ

    Class-map: VOICE (match-all)
      0 packets, 0 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: ip dscp ef (46)
      Queueing
        Strict Priority
        Output Queue: Conversation 136
        Bandwidth 48 (kbps) Burst 1200 (Bytes)
        (pkts matched/bytes matched) 0/0
        (total drops/bytes drops) 0/0

    Class-map: SIGNALING (match-all)
      0 packets, 0 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: ip dscp cs3 (24)
      Queueing
        Output Queue: Conversation 137
        Bandwidth 8 (kbps) Max Threshold 64 (packets)
        (pkts matched/bytes matched) 0/0
        (depth/total drops/no-buffer drops) 0/0/0

    Class-map: class-default (match-any)
      17 packets, 15554 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: any
      Queueing
        Flow Based Fair Queueing
        Maximum Number of Hashed Queues 128
        (total queued/total drops/no-buffer drops) 0/0/0

Check PPP multilink status:

R1#ping 177.1.102.1 source loopback 0 size 1500

Type escape sequence to abort.
Sending 5, 1500-byte ICMP Echos to 177.1.102.1, timeout is 2 seconds:
Packet sent with a source address of 177.1.101.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 64/64/64 ms

R1#show ppp multilink

Virtual-Access3, bundle name is R2
  Endpoint discriminator is R2
  Bundle up for 00:07:49, total bandwidth 384, load 1/255
  Receive buffer limit 12192 bytes, frag timeout 1000 ms
  Interleaving enabled            <------- Interleaving enabled
    0/0 fragments/bytes in reassembly list
    0 lost fragments, 0 reordered
    0/0 discarded fragments/bytes, 0 lost received
    0x34 received sequence, 0x34 sent sequence   <---- MLP sequence numbers for fragmented packets
  Member links: 1 (max not set, min not set)
    Vi2, since 00:07:49, 624 weight, 614 frag size <------- Fragment Size
No inactive multilink interfaces

Verify the interleaving queue:

R1#show interfaces serial 0/0/0:0
Serial0/0/0:0 is up, line protocol is up
  Hardware is GT96K Serial
  MTU 1500 bytes, BW 1536 Kbit, DLY 20000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation FRAME-RELAY, loopback not set
  Keepalive set (10 sec)
  LMI enq sent  10, LMI stat recvd 11, LMI upd recvd 0, DTE LMI up
  LMI enq recvd 0, LMI stat sent  0, LMI upd sent  0
  LMI DLCI 1023  LMI type is CISCO  frame relay DTE
  FR SVC disabled, LAPF state down
  Broadcast queue 0/64, broadcasts sent/dropped 4/0, interface broadcasts 0
  Last input 00:00:05, output 00:00:02, output hang never
  Last clearing of "show interface" counters 00:01:53
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: dual fifo                        <--------- Dual FIFO
  Output queue: high size/max/dropped 0/256/0         <--------- High Queue
  Output queue: 0/128 (size/max)                      <--------- Low (fragments) queue
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     47 packets input, 3914 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     1 input errors, 1 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     47 packets output, 2149 bytes, 0 underruns
     0 output errors, 0 collisions, 4 interface resets
     0 output buffer failures, 0 output buffers swapped out
     1 carrier transitions
  Timeslot(s) Used:1-24, SCC: 0, Transmitter delay is 0 flags

Further Reading

Reducing Latency and Jitter for Real-Time Traffic Using Multilink PPP
Multiclass Multilink PPP
Using Multilink PPP over Frame Relay

The need for fragmentation

We are going to briefly discuss Layer2 fragmentation schemes, their purpose and configuration examples. Let’s start with a general discussion. Usually, Layer2 fragmentation is used to accomplish one of two goals:

a) Link aggregation, e.g. making a number of physical channels look like one logical link from Layer2 standpoint. A good example is PPP Multilink, which breaks large packets into smaller pieces, and send them other multiple physical links simulataneously. Another example is FRF.16 (Multilink Frame-Relay).

b) Decrease large packets serialization delay on slow links. By “slow link”, we mean a link with “physical” speed (e.g. clock-rate) less than 1 Mbps. The issue is usually to have a mix of bulk data and delay-sensitive traffic (e.g. voice) on the same link. This is because large bulky packets (say 1500 bytes in size) may block the interface transmission queue for a long time (with slow links), making small voice packets (e.g. 60 bytes) to wait for more than maximum tolerable threshold (say 10ms).

For example, if physical interface has clock rate of 384000bps, large 1500 byte packet would take 1500*8/384000 > 30ms to serialize. So here comes the solution: break large packets into small pieces at layer2, to decrease the serialization delay. Say if we break one 1500 packet into 3×500 byte frames on a 384Kpbs link, we’ll get 10ms transmission delay for each fragment. Look at the following picture ([V] is a voice packet, and [D] is a data packet)

Before fragmentation:

--[DDD][V][DDD][V][V][DDD]--->

After fragmentation:

--[D][D][D][V][D][D][D][V][V][D][D][D]--->

There is still something wrong here: Small pieces of a large packet are being sent in a row, effectively blocking the transmission qeueue the same way it was before. So just fragmenting alone is not enough - we need a way to make sure the fragments of large packets are “mixed” with voice packets. The technique is called “interleaving”, and it always accompanies fragmentation. With interleaving we get a picture like this:

---[D][V][D][V][V][D][D][D][D]--->

That is, voice packets are not separated by large “islands” of data packets.

So how does interleaving work? Usually, it is accomplished by inserting a special “interleaving” queue before interface transmission (FIFO) queue. Interleaving queue usually has two parts: “high” and “low” FIFO queues. Small packets (packets smaller than configured fragment size) go to “high” queue, and large packets are first fragmented, and then assigned to “low” queue. With this strategy, “high” queue is a priority queue - it’s always get emptied first, and only then the “low” queue gets served.

[Interface Software Queue, e.g. WFQ] --> 

If(Packet.Size lt FRAGMENT_SIZE) 

then 

{ put to High_Queue } 

else 

{ Fragment and put fragments to Low_Queue } 

--> { Service (High_Queue) then Service(Low_Queue) } --> [Interface Xmit Queue]

We are not over yet! You’ve probably noticed “Interface Software Queue” on the diagram above. It plays an important role too. Say, if this is a simple FIFO queue, and a bunch of large data packets sit there ahead of small voice packets. The data packets will get dequeud first, fragmented, and since “high” interleaving queue is empty, will be sent in line on their own. Therefore, the last component to make fragmentation and interleaving work properly, is a software interface queue that give voice packets priority treatment. This could be legacy WFQ or modern CBWFQ/LLQ - just remember that voice packets should be taken from software queue first!

So here are the important things to remember about fragmentation:

1) Fragmentation is not effective without interleaving
2) Interleaving is accomplished by use of additional priority queue
3) Decision on where to put a packet to “high” interleaving queue is based on packet size solely
4) Interleaving is inefficient without a software queue that gives small (e.g. voice) packets priority treatment

Situation becomes more complicated, when we have multiple logical channels (e.g. PVCs) multiplexed over the same physical channel. For example, with a number of Frame-Relay PVCs, assigned to the same physical interface, we get multiple software queues - one per each PVC. They all share the same interleaving queue at physical interface level. Due to the fact that large packets of one PVC may affect small packets serialization delay of the other PVC, fragmentation should be turned on for all PVCs simultaneously.

This is a “modern” way to configure FRTS, using MQC commands only to accomplish the task. With MQC approach, an unified interface has been introduced to configure all QoS settings, irrelevant of underlying technology.

In summary:

- Legacy command frame-relay traffic-shaping is incompatible with MQC-based FRTS (you can’t mix them)
- Fancy queueing could not be used as a PVC-queueing strategy: CBWFQ is the only option available
- Per-VC CBWFQ is configured via hierarchical policy-maps configuration: Parent policy sets shaping values, while child policy implements CBWFQ
- You may apply policy-map per-interface (subinterface) or per-VC, using match fr-dlci under class-map submode

Example: Shape PVC to 384Kbps and provide LLQ treatment for voice bearer packets on PVC queue

class-map VOICE
 match ip dscp ef
!
class-map DATA
 match ip dscp cs1

!
! "Child" policy-map, used to implement CBWFQ
!

policy-map CBWFQ
 class VOICE
  priority 64
 class DATA
  bandwidth 128
 class class-default
  fair-queue

!
! "Parent" policy map, used for PVC shaping
!

policy-map SHAPE_384K
 class class-default
  shape average 384000
  shape adaptive 192000
  service-policy CBWFQ

interface Serial 0/0/0:0.1
 ip address 177.0.112.1 255.255.255.0
 service-policy output SHAPE_384K
 frame-relay interface-dlci 112

Verification: check out policy map settings

Rack1R1#show policy-map interface serial 0/0/0:0.1

 Serial0/0/0:0.1 

  Service-policy output: SHAPE_384K

    Class-map: class-default (match-any)
      1942 packets, 1590741 bytes
      5 minute offered rate 48000 bps, drop rate 0 bps
      Match: any
      Traffic Shaping
           Target/Average   Byte   Sustain   Excess    Interval  Increment
             Rate           Limit  bits/int  bits/int  (ms)      (bytes)
           384000/384000    2400   9600      9600      25        1200     

        Adapt  Queue     Packets   Bytes     Packets   Bytes     Shaping
        Active Depth                         Delayed   Delayed   Active
        -      0         1936      1581717   0         0         no

      Service-policy : CBWFQ

        Class-map: VOICE (match-all)
          0 packets, 0 bytes
          5 minute offered rate 0 bps, drop rate 0 bps
          Match: protocol rtp
          Match: ip dscp ef (46)
          Queueing
            Strict Priority
            Output Queue: Conversation 40
            Bandwidth 64 (kbps) Burst 1600 (Bytes)
            (pkts matched/bytes matched) 0/0
            (total drops/bytes drops) 0/0

        Class-map: DATA (match-all)
          0 packets, 0 bytes
          5 minute offered rate 0 bps, drop rate 0 bps
          Match: ip dscp cs1 (8)
          Queueing
            Output Queue: Conversation 41
            Bandwidth 128 (kbps) Max Threshold 64 (packets)
            (pkts matched/bytes matched) 0/0
        (depth/total drops/no-buffer drops) 0/0/0

        Class-map: class-default (match-any)
          1942 packets, 1590741 bytes
          5 minute offered rate 48000 bps, drop rate 0 bps
          Match: any
          Queueing
            Flow Based Fair Queueing
            Maximum Number of Hashed Queues 32
        (total queued/total drops/no-buffer drops) 0/0/0

The amount of bandwidth, available for allocation to CBWFQ classes, is taken from shape adaptive value. If the latter is not configured, shape average
value is used instead. Note, that as you configure bandwidth settings for classes, their values are not subtracted from remaining bandwidth. This is in contraty with
“classic” CBWFQ, applied to a physical interface (not subinterface or PVC)

Verification (with the example above):

Rack1R1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
Rack1R1(config)#policy-map CBWFQ
Rack1R1(config-pmap)#class class-default
Rack1R1(config-pmap-c)#no fair-queue
Rack1R1(config-pmap-c)#bandwidth 256
I/f  shape  class class-default requested bandwidth 256 (kbps), available only 192 (kbps)

Note that available bandwidth is set to shape adaptive value, even though we have priority configured under class VOICE and bandwidth
settings under class DATA

- You can’t apply FRF.12 fragmentation with MQC commands - it should be applied at physical interface level. By doing so, FRF.12 is effectively enabled for all PVCs
- Physical interface queue could be set to any of WFQ/CQ/PQ or CBWFQ (not restricted to FIFO as with FRTS legacy) - though this is rarely needed

Example: Shape PVC DLCI 112 to 384Kpbs and enable FRF.12 fragmentation for all PVCs

class-map VOICE
 match ip dscp ef
!
class-map DATA
 match ip dscp cs1

!
! Match the specific DLCI
!
class-map DLCI_112
 match fr-dlci 112

!
! "Child" policy-map, used to implement CBWFQ
!

policy-map CBWFQ
 class VOICE
  priority 64
 class DATA
  bandwidth 128
 class class-default
  fair-queue

!
! "Parent" policy map, used for PVC shaping
!  With multiple classes, we can match different DLCIs
!  all at the same physical interface (where they belongs)
!

policy-map INTERFACE_POLICY
 class DLCI_112
  shape average 384000
  shape adaptive 192000
   service-policy CBWFQ

!
! Apply the parent policy map at physical interface level
! Also, configure FRF.12 "global" settings here
!

interface Serial 0/0/0:0
 service-policy output INTERFACE_POLICY
 frame-relay fragment 640 end-to-end

Verification:

Rack1R1#show policy-map interface serial 0/0/0:0

 Serial0/0/0:0 

  Service-policy output: INTERFACE_POLICY

    Class-map: DLCI_112 (match-all)
      1040 packets, 95287 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: fr-dlci 112
      Traffic Shaping
           Target/Average   Byte   Sustain   Excess    Interval  Increment
             Rate           Limit  bits/int  bits/int  (ms)      (bytes)
           384000/384000    2400   9600      9600      25        1200     

        Adapt  Queue     Packets   Bytes     Packets   Bytes     Shaping
        Active Depth                         Delayed   Delayed   Active
        -      0         1040      95287     4         1373      no

      Service-policy : CBWFQ

        Class-map: VOICE (match-all)
          0 packets, 0 bytes
          5 minute offered rate 0 bps, drop rate 0 bps
          Match: protocol rtp
          Match: ip dscp ef (46)
          Queueing
            Strict Priority
            Output Queue: Conversation 40
            Bandwidth 64 (kbps) Burst 1600 (Bytes)
            (pkts matched/bytes matched) 0/0
            (total drops/bytes drops) 0/0

        Class-map: DATA (match-all)
          0 packets, 0 bytes
          5 minute offered rate 0 bps, drop rate 0 bps
          Match: ip dscp cs1 (8)
          Match: fr-dlci 112
          Queueing
            Output Queue: Conversation 41
            Bandwidth 128 (kbps) Max Threshold 64 (packets)
            (pkts matched/bytes matched) 0/0
        (depth/total drops/no-buffer drops) 0/0/0

        Class-map: class-default (match-any)
          1040 packets, 95287 bytes
          5 minute offered rate 0 bps, drop rate 0 bps
          Match: any
          Queueing
            Flow Based Fair Queueing
            Maximum Number of Hashed Queues 32
        (total queued/total drops/no-buffer drops) 0/0/0

    Class-map: class-default (match-any)
      2594 packets, 153695 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: any

Verify fragmentation settings:

Rack1R1#show interface serial 0/0/0:0
Serial0/0/0:0 is up, line protocol is up
  Hardware is GT96K Serial
  MTU 1500 bytes, BW 1536 Kbit, DLY 20000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation FRAME-RELAY, loopback not set
  Keepalive set (10 sec)
  LMI enq sent  21224, LMI stat recvd 21224, LMI upd recvd 0, DTE LMI up
  LMI enq recvd 0, LMI stat sent  0, LMI upd sent  0
  LMI DLCI 1023  LMI type is CISCO  frame relay DTE
  FR SVC disabled, LAPF state down
  Fragmentation type: end-to-end, size 640, PQ interleaves 0   <--------- Fragment Size
  Broadcast queue 0/64, broadcasts sent/dropped 63160/0, interface broadcasts 56080
  Last input 00:00:03, output 00:00:02, output hang never
  Last clearing of "show interface" counters 2d10h
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 6
  Queueing strategy: weighted fair
  Output queue: 0/1000/64/0 (size/max total/threshold/drops)
     Conversations  0/1/256 (active/max active/max total)
     Reserved Conversations 0/0 (allocated/max allocated)
     Available Bandwidth 1152 kilobits/sec
  5 minute input rate 0 bits/sec, 1 packets/sec
  5 minute output rate 0 bits/sec, 1 packets/sec
     272202 packets input, 27557680 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     15 input errors, 15 CRC, 8 frame, 0 overrun, 0 ignored, 5 abort
     333676 packets output, 42152431 bytes, 0 underruns
     0 output errors, 0 collisions, 16 interface resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions
  Timeslot(s) Used:1-24, SCC: 0, Transmitter delay is 0 flags

This is the most well-known FRTS method, which has been available for quite a while on Cisco routers. It is now being outdated by MQC configurations.
The key characteristic is that all settings are configured under map-class command mode, and later are applied to a particular set PVCs. The
same configuration concept was used for legacy ATM configuration mode (map-class atm).

Legacy FRTS has the following characteristics:

- Enabled with frame-relay traffic-shaping command at physical interface level
- Incompatible with GTS or MQC commands at subinterfaces or physical interface levels
- With FRTS you can enforce bitrate per-VC (VC-granular, unlike GTS), by applying a map-class to PVC
- When no map-class is explicitly applied to PVC, it’s CIR and Tc are set to 56K/125ms by default
- Shaping parameters are configured under map-class frame-relay configuration submode
- Allows to configure fancy-queueing (WFQ/PQ/CQ) or simple FIFO per-VC
- No option to configure fancy-queueing at interface level: interface queue is forced to FIFO (if no FRF.12 is configured)
- Allows for adaptive shaping (throttling down to minCIR) on BECN reception (just as GTS) and option to reflect incoming FECNs as BECNs
- Option to enable adaptive shaping which responds to interface congestion (non-empty interface queue)

Example: Shape PVC to 384Kbps with minimal Tc (10ms) and WFQ as interface queue

map-class frame-relay SHAPE_384K
 frame-relay cir 384000
 frame-relay bc 3840
 frame-relay be 0
 !
 ! Adaptive shaping: respond to BECNs and interface congestion
 !
 frame-relay adaptive-shaping becn
 frame-relay adaptive-shaping interface-congestion
 !
 ! Per-VC fancy-queueing
 !
 frame-relay fair-queue
!
interface Serial 0/0/0:0
 frame-relay traffic-shaping
!
interface Serial 0/0/0:0.1
 ip address 177.0.112.1 255.255.255.0
 frame-relay interface-dlci 112
  class SHAPE_384K

Verification: Check FRTS settings for the configured PVC

Rack1R1#show frame-relay pvc 112

PVC Statistics for interface Serial0/0/0:0 (Frame Relay DTE)

DLCI = 112, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0/0:0

  cir 384000    bc 3840      be 0         byte limit 480    interval 10   <------ Shaping parameters
  mincir 192000    byte increment 480   Adaptive Shaping BECN and IF_CONG <---- Adaptive Shaping enabled
  pkts 0         bytes 0         pkts delayed 0         bytes delayed 0
  shaping inactive
  traffic shaping drops 0
  Queueing strategy: weighted fair  <---- WFQ is the per-VC queue
  Current fair queue configuration:
   Discard     Dynamic      Reserved
   threshold   queue count  queue count
    64          16           0
  Output queue size 0/max total 600/drops 0

The other PVC, with no class configured, has CIR set to 56Kbps and uses FIFO as per-VC queue:

Rack1R1#show frame-relay pvc 113

PVC Statistics for interface Serial0/0/0:0 (Frame Relay DTE)

DLCI = 113, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0/0:0.2

  cir 56000     bc 7000      be 0         byte limit 875    interval 125 <---- CIR=56K
  mincir 28000     byte increment 875   Adaptive Shaping none
  pkts 74        bytes 5157      pkts delayed 0         bytes delayed 0
  shaping inactive
  traffic shaping drops 0
  Queueing strategy: fifo <------------------ FIFO
  Output queue 0/40, 0 drop, 0 dequeued

Check the physical interface queue:

Rack1R1#show interfaces serial 0/0/0:0 | inc Queue
  Queueing strategy: fifo

- Interface queue could be changed to PIPQ (PVCs are assigned to 4 pririty groups, with PQ being physical interface queue)
- PIPQ stands for PVC Interface Priority queueing

Example: Map PVC 112 traffic to high interface queue, and PVC 113 to low interface queue, with WFQ being per-VC queueing

!
! Shape to 384K an assign to high interface level queue
!
map-class frame-relay SHAPE_384K_HIGH
 frame-relay cir 384000
 frame-relay bc 3840
 frame-relay be 0
 !
 ! Per-VC fancy-queueing
 !
 frame-relay fair-queue
 frame-relay interface-queue priority high

!
! Shape to 256k an assign to low interface level queue
!
map-class frame-relay SHAPE_256K_LOW
 frame-relay cir 256000
 frame-relay bc 2560
 frame-relay be 0
 !
 ! Per-VC fancy-queueing
 !
 frame-relay fair-queue
 frame-relay interface-queue priority low

!
! Enable PIPQ as interface queueing strategy
!
interface Serial 0/0/0:0
 frame-relay traffic-shaping
 frame-relay interface-queue priority
!
interface Serial 0/0/0:0.1
 ip address 177.0.112.1 255.255.255.0
 frame-relay interface-dlci 112
  class SHAPE_384K_HIGH
!
interface Serial 0/0/0:0.2
 ip address 177.0.113.1 255.255.255.0
 frame-relay interface-dlci 113
  class SHAPE_256K_LOW

Verfication: Check PVC interface-level priorities

Rack1R1#show frame-relay pvc 112

PVC Statistics for interface Serial0/0/0:0 (Frame Relay DTE)

DLCI = 112, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0/0:0.1

  cir 384000    bc 3840      be 0         byte limit 480    interval 10
  mincir 192000    byte increment 480   Adaptive Shaping none
  pkts 1687      bytes 113543    pkts delayed 0         bytes delayed 0
  shaping inactive
  traffic shaping drops 0
  Queueing strategy: weighted fair
  Current fair queue configuration:
   Discard     Dynamic      Reserved
   threshold   queue count  queue count
    64          16           0
  Output queue size 0/max total 600/drops 0
  priority high
  ^^^^^^^^^^^^^

Rack1R1#show frame-relay pvc 113

PVC Statistics for interface Serial0/0/0:0 (Frame Relay DTE)

DLCI = 113, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0/0:0.2

  cir 256000    bc 2560      be 0         byte limit 320    interval 10
  mincir 128000    byte increment 320   Adaptive Shaping none
  pkts 1137      bytes 79691     pkts delayed 0         bytes delayed 0
  shaping inactive
  traffic shaping drops 0
  Queueing strategy: weighted fair
  Current fair queue configuration:
   Discard     Dynamic      Reserved
   threshold   queue count  queue count
    64          16           0
  Output queue size 0/max total 600/drops 0
  priority low
  ^^^^^^^^^^^^

Verify interface-level queue:

Rack1R1#show interfaces serial 0/0/0:0 | inc (Output|high)
  Output queue (queue priority: size/max/drops):
     high: 0/20/0, medium: 0/40/0, normal: 0/60/0, low: 0/80/0

- With FRF.12 fragmentation configured per any PVC, physical interface queue is changed to dual-FIFO
- This is due to the fact that fragmentation is ineffective without interleaving
- Fragment size is calculated based on physical interface speed to allow minimum serialization delay

Example: Enable FRF.12 fragmentation for PVC DLCI 112 and physical interface speed 512Kbps

!
! PVC shaped to 384Kbps, with physical interface speed 512Kbps
!
map-class frame-relay SHAPE_384K_FRF12
 frame-relay cir 384000
 frame-relay bc 3840
 frame-relay be 0
 !
 ! Per-VC fancy-queueing
 !
 frame-relay fair-queue

 !
 ! Enable FRF.12 per VC. Fragment size = 512Kbps*0,01/8 = 640 bytes
 !
 frame-relay fragment 640
!
!
!
interface Serial 0/0/0:0
 frame-relay traffic-shaping
!
interface Serial 0/0/0:0.1
 ip address 177.0.112.1 255.255.255.0
 frame-relay interface-dlci 112
  class SHAPE_384K_FRF12

Verfication: Check PVC settings

Rack1R1#show frame-relay pvc 112

PVC Statistics for interface Serial0/0/0:0 (Frame Relay DTE)

DLCI = 112, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0/0:0.1

  fragment type end-to-end fragment size 640
  cir 384000    bc   3840      be 0         limit 480    interval 10
  mincir 192000    byte increment 480   BECN response no  IF_CONG no
  frags 1999      bytes 135126    frags delayed 0         bytes delayed 0
  shaping inactive
  traffic shaping drops 0
  Queueing strategy: weighted fair
  Current fair queue configuration:
   Discard     Dynamic      Reserved
   threshold   queue count  queue count
    64          16           0
  Output queue size 0/max total 600/drops 0

Look at physical interface queue:

Rack1R1#show interfaces serial 0/0/0:0 | inc Queu|Output q
  Queueing strategy: dual fifo
  Output queue: high size/max/dropped 0/256/0
  Output queue: 0/128 (size/max)

- You can map up to 4 priority queues to 4 different VCs (inverse PIPQ)
- This scenario usually implies multiple PVCs running between two sites (e.g. PVC for voice and PVC for data)

Example: Map voice packets to high interface-level priority queue and send them over PVC 112

!
! Voice bearer
!
access-list 101 permit udp any any range 16384 32767

!
! Simple priority list to classify voice bearer to high queue
!
priority-list 1 protocol ip high list 101

interface Serial 0/0/0:0
 ip address 177.1.0.1 255.255.255.0
 !
 ! We apply the priority group twice: first to implement queueing
 !
 priority-group 1
 !
 ! Next to map priority levels to DLCIs
 !
 frame-relay priority-dlci-group 1 112 112 113 113

Verfication:

Rack1R1#show queueing interface serial 0/0/0:0
Interface Serial0/0/0:0 queueing strategy: priority

Output queue utilization (queue/count)
	high/217 medium/0 normal/1104 low/55 

Rack1R1#show frame-relay pvc 112

PVC Statistics for interface Serial0/0/0:0 (Frame Relay DTE)

DLCI = 112, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0/0:0

  pvc create time 3d01h, last time pvc status changed 3d01h
  Priority DLCI Group 1, DLCI 112 (HIGH), DLCI 112 (MEDIUM)
  DLCI 113 (NORMAL), DLCI 113 (LOW)

- You can change per-VC queue to CBWFQ/LLQ, and still shape with FRTS
- Note that available bandwidth will be calculated from minCIR value, which is CIR/2 by default

Example: Implement CBWFQ Per-VC

!
! Classify voice using NBAR
!
class-map VOICE
 match protocol rtp

!
! Simple LLQ
!
policy-map CBWFQ
 class VOICE
  priority 64
 class class-default
  bandwidth 64

!
! Use CBWFQ as queueing strategy
! Note that MinCIR = 384/2=192Kbps
!
map-class frame-relay SHAPE_384K_CBWFQ
 frame-relay cir 384000
 frame-relay bc 3840
 frame-relay be 0
 service-policy output CBWFQ
!
!
!
interface Serial 0/0/0:0
 frame-relay traffic-shaping
!
interface Serial 0/0/0:0.1
 ip address 177.0.112.1 255.255.255.0
 frame-relay interface-dlci 112
  class SHAPE_384K_CBWFQ

Verfication: Check PVC queueing strategy

Rack1R1#show frame-relay pvc 112

PVC Statistics for interface Serial0/0/0:0 (Frame Relay DTE)

DLCI = 112, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0/0:0

  cir 384000    bc 3840      be 0         byte limit 480    interval 10
  mincir 192000    byte increment 480   Adaptive Shaping none
  pkts 0         bytes 0         pkts delayed 0         bytes delayed 0
  shaping inactive
  traffic shaping drops 0
  service policy CBWFQ
 Serial0/0/0:0: DLCI 112 -

  Service-policy output: CBWFQ

    Class-map: VOICE (match-all)
      0 packets, 0 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: protocol rtp
      Queueing
        Strict Priority
        Output Queue: Conversation 40
        Bandwidth 64 (kbps) Burst 1600 (Bytes)
        (pkts matched/bytes matched) 0/0
        (total drops/bytes drops) 0/0

    Class-map: class-default (match-any)
      32 packets, 2560 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: any
      Queueing
        Output Queue: Conversation 41
        Bandwidth 128 (kbps) Max Threshold 64 (packets)
        (pkts matched/bytes matched) 0/0
        (depth/total drops/no-buffer drops) 0/0/0
  Output queue size 0/max total 600/drops 0

To verify that only MinCIR of bandwidth is allocated to CBWFQ under map-class do the following:

Rack1R1(config)#policy-map CBWFQ
Rack1R1(config-pmap)# class VOICE
Rack1R1(config-pmap-c)#  priority 64
Rack1R1(config-pmap-c)# class class-default
Rack1R1(config-pmap-c)#  bandwidth 192
I/f Serial0/0/0:0 DLCI 112 Class class-default requested bandwidth 192 (kbps) Only 128 (kbps) available

As first and very basic option, you may use Generic Traffic Shaping to implement FRTS. This is a common technique, not unique to Frame-Relay, with the following properties:

- Configured by using traffic-shape interface command
- As with standard GTS, internal shaper queue is basic WFQ
- Configured per inteface/subinteface (no PVC granularity)
- GTS may adapt to BECNs & reflect FECNs (BECN received at any interface/subinteface PVC will cause shaper rate to throttle back to minCIR)
- FECN reflection activates sending BECNs in responce to incoming FECN frames. Note, that FECN/BECN responce requires provider to mark frames with FECN/BECN bits
- You can configure Fancy-Queueing (WFQ/PQ/CQ) at physical interface level with GTS.

Example:

!
! Physical Interface, fancy-queueing may apply here
!

interface Serial 0/0/0:0
 no ip address
 encapsulation frame-relay
 fair-queue

!
! Subinterface, apply GTS here
!

interface Serial0/0/0:0.1 point-to-point
 ip address 177.0.112.1 255.255.255.0
 !
 ! Shaping rate
 !
 traffic-shape rate 512000
 !
 ! "MinCIR", adapt to BECNs
 !
 traffic-shape adaptive 256000
 !
 ! Reflect FECNs as BECNs
 !
 traffic-shape fecn-adapt
 frame-relay interface-dlci 112

Verification:

Rack1R1#show traffic-shape 

Interface   Se0/0/0:0.1
       Access Target    Byte   Sustain   Excess    Interval  Increment Adapt
VC     List   Rate      Limit  bits/int  bits/int  (ms)      (bytes)   Active
-             512000    3200   12800     12800     25        1600      BECN

Rack1R1#show traffic-shape statistics
                  Acc. Queue Packets   Bytes     Packets   Bytes     Shaping
I/F               List Depth                     Delayed   Delayed   Active
Se0/0/0:0.1             0     157       10500     0         0         no

Rack1R1#show traffic-shape queue
Traffic queued in shaping queue on Serial0/0/0:0.1
  Queueing strategy: weighted fair
  Queueing Stats: 0/1000/64/0 (size/max total/threshold/drops)
     Conversations  0/0/32 (active/max active/max total)
     Reserved Conversations 0/0 (allocated/max allocated)
     Available Bandwidth 512 kilobits/sec